Bayes's Net

Description:

• Help express and visualize Conditional Independence
• Given n variables and each has domain of k, then we need $k^n$ entries to model the Joint Distribution
  • We can use bayes net to break down using and make the table smaller
  • → Encoding joint distribution
• The idea is group the independent variables together, like karnaugh map
• Then Bayes’s net is a Topology Graph with local conditional probabilities
• $P(x_i|x_1,...,x_n)=P(x_i|parents(X_i))$
• $P(x_1,x_2,...,x_n)=\prod _{i=1}^{n}P(x_i|parents(X_i))$

Representation:

• Nodes are variables
  • can be assigned (observed) or unassigned (unobserved)
• Arcs denotes influences
  • sometimes the influences can be causal but
• Size of Bayes’s Net:
  • The joint distribution over N boolean variables: $2^n$
  • N-node net if nodes have up to $k$ parents: $O(N\ast 2^{k+1})$

D-separation Triplet:

• As Bayes’s net are not drawn by causal but influences, we need a way to separate them
• We do so by analyzing the triplets and give a type of each, rmb to shade the evidence(given) variable gray:
  • 
  • Active triples:
    1. Causal chain: if given the middle variable, the path is inactive as the evidence along the chain stop the indirect influence
    2. Common cause: if given cause variable, the path is inactive
    3. Common effect: if given effect variable, the path becomes active as one of them is the blame for it
    4. General case
• Any complex graph can be broken into repetitions of 3 cases then a path is active if all triples on that path are active
• A path is inactive → we can guarantee they are independence but not dependence
• With this, we can build a complete list of conditional independences that are necessarily true to the form $X_i\perp \perp X_j|\{X_{k1},...,X_{kn}\}$ which determines the probability distributions that can be represented

Inference by Enumeration vs Variable Elimination vs Sample in Bayes’ Net

Decision Network

Naive Bayes Net