Articles

Part 7: Structure Learning

Part 7: Structure Learning

DJ Rich
Posted: October 08, 2018

Structure learning precedes parameter learning, whereby a graph or similarly abstract structure must be learnd from data. Doing so presents a formidable integration, but with techniques and approximations, a fruitful search over structures can be performed. For theoretical reasons, the task is considerably easier in the Bayesian Network case than in the alternative.
Part 6: Learning Parameters of a Markov Network

Part 6: Learning Parameters of a Markov Network

DJ Rich
Posted: October 01, 2018

The theory of Markov Network parameter learning is intuitive and instructive, but it exposes an intractable normalizer, forbidding the task from reducing to easier ones. Ultimately, the task is hard.
Part 5: Learning Parameters of a Bayesian Network

Part 5: Learning Parameters of a Bayesian Network

DJ Rich
Posted: September 27, 2018

Learning parameters of a Bayesian Network enjoys a decomposition that it makes a much friendly endeavor than that of it's cousin, the Markov Network.
Part 4: Monte Carlo Methods

Part 4: Monte Carlo Methods

DJ Rich
Posted: September 24, 2018

Monte Carlo methods answer the inference task with a set of samples, sampled approximately from the target distribution. In total, they provide a supremely general toolset. However, to use them requires a skill for managing complexities of distributional convergence and autocorrelation.
Part 3: Variational Inference

Part 3: Variational Inference

DJ Rich
Posted: September 10, 2018

Variational Inference, a category of approximate inference algorithms, achieves efficiency by restricting inference to a computationally friendly set of distributions. Using tools from information theory, we may find the distribution that best approximates results of exact inference.
Part 2: Exact Inference

Part 2: Exact Inference

DJ Rich
Posted: September 03, 2018

Given a Probabilistic Graphical Model, exact inference algorithms exploit factorization and caching to answer questions about the system it represents.
Part 1: An Introduction to Probabilistic Graphical Models

Part 1: An Introduction to Probabilistic Graphical Models

DJ Rich
Posted: August 27, 2018

Probabilistic Graphical Models are born from a remarkable synthesis of probability theory and graph theory.
Bias-Variance Trade-Off

Bias-Variance Trade-Off

DJ Rich
Posted: July 08, 2018

The bias-variance trade-off is a rare insight into the challenge of generalization.
Information Theory and Entropy

Information Theory and Entropy

DJ Rich
Posted: June 18, 2018

Entropy and its related concepts quantify the otherwise abstract concept of information. A tour reveals its relationship to information, binary encodings and uncertainty. Most intuitively, we're left with a simple analogy to 2D areas.
Generalized Linear Models

Generalized Linear Models

DJ Rich
Posted: June 12, 2018

A Generalized Linear Model, if viewed without knowledge of their motivation, can be a confusing tool. It's easier to understand if seen as a two knob generalization of linear regression.