UP.A — Probability for Generative Models
Session UP.A · async primer · date: see calendar-map
How to read this page. This is the async primer of the course, and it is a document rather than a lecture. There are no slides, and nothing here was said out loud in a room you might have missed. It is written to be read alone, at your own pace, before the first live session.
It is also self-contained at the level required for the course. The probability identities used later are either derived here or introduced here in the form in which we will need them. No proof below sends you to a textbook to fill a gap. Two books are named at the end of each section as a slower treatment of the same material — (Bishop and Bishop 2024) chapters 2–3, and (Murphy 2023) chapters 2 and 5 — and they are optional, every one of them. Each section names the chapter that matches it.
The primer serves two populations at once. If you come from computer science, plan to grind: read with a pen, and do the exercises. If you come from mathematics, plan to skim: every section opens with a short self-test, and if you answer it comfortably you may go to the next section. The self-test answers are in a footnote, so you can check yourself without reading ahead.
Every section ends with exercises tagged [PS1.1.x]. Those nine exercises are problem set PS1.1, and PS1.1 is part of PS1, which carries about 12% of the course grade. They are not optional practice. Answers are not printed here, because you will hand them in.
PS1 is released in stages. What is released with this primer is the PS1 shell and PS1.1. PS1.2 is appended at UP.T1, and U1 appends its own items when they are assigned. So “PS1 is out” means, today, exactly these nine exercises — not every item the set will eventually contain.
One more thing to know before you start. Each section ends with a box titled “Where this reappears”, naming the sessions that consume it by session ID. Nothing on this page is here for general culture. Every item was put here because a specific later session needs it, and the box tells you which one.
The notation of this course is fixed, and this page is your first exposure to it. Section 5 drills it, and Appendix A reproduces the full symbol table.
§1 — Densities and conditioning
- A joint density \(p(x, y)\) is given. Write the marginal \(p(y)\) and the conditional \(p(x \mid y)\) in terms of it.
- \(X\) and \(Y\) are each fair coin flips, and \(Z = X \oplus Y\) is their exclusive or. Are \(X\) and \(Y\) independent? Are they independent given \(Z\)?
- You draw \(Y \sim \mathrm{Unif}\{1, 2\}\), then \(X \sim \mathcal{N}(Y, 1)\). What is \(\mathbb{E}_{x \sim p_X}[x]\), where \(p_X\) is the marginal law of \(X\)?
Answers in the footnote.1
Joint, marginal, conditional
Everything in this course is built out of three operations on one object. The object is a joint density, and the operations are marginalizing, conditioning, and taking expectations.
A function \(p \colon \mathbb{R}^{d_x} \times \mathbb{R}^{d_y} \to [0, \infty)\) is a joint density for the pair \((X, Y)\) if \[ \mathbb{P}\big[(X, Y) \in A\big] \;=\; \int_A p(x, y)\,\mathrm{d}x\,\mathrm{d}y , \qquad A \subseteq \mathbb{R}^{d_x} \times \mathbb{R}^{d_y} , \] for every measurable set \(A\), and \[ \int_{\mathbb{R}^{d_x} \times \mathbb{R}^{d_y}} p(x,y)\,\mathrm{d}x\,\mathrm{d}y \;=\; 1 . \] The normalization follows from the first display with \(A\) the whole space; it is stated on its own because it is the condition you check in practice.
When the joint law needs a name distinct from its marginals, we write \(p_{XY}\); its marginals are \(p_X\) and \(p_Y\). Arguments say which density is meant; subscripts say which law is sampled.
A density is not a probability. It is a probability per unit volume, and it may exceed \(1\). What is bounded by \(1\) is its integral over a region.
One distinction, kept throughout this primer. A probability law assigns probabilities to sets. A density is a representation of such a law relative to volume, and exists when the law is absolutely continuous with respect to Lebesgue measure. Throughout this section, continuous random variables are described by densities; when a discrete variable appears, we use probabilities and replace integrals over that variable by sums.
This distinction will matter again in §2. A Gaussian with singular covariance, for example, is a perfectly valid probability law but has no density on the ambient space \(\mathbb{R}^d\).
Marginalizing removes a variable by integrating it away: \[ p(y) \;=\; \int_{\mathbb{R}^{d_x}} p(x, y)\,\mathrm{d}x . \tag{1}\]
Read Equation 1 as summing out \(x\): you no longer care which value \(X\) took, so you add up the mass over all of them. The result is a density in \(y\) alone. From here on, the integration domain is omitted when it is clear from context. We use the same symbol \(p\) for joint, marginal, and conditional densities; the arguments make clear which one is meant. This is a common convention in probabilistic modeling.
Conditioning fixes a variable and renormalizes what is left:
\[ p(x \mid y) \;=\; \frac{p(x, y)}{p(y)} , \qquad \text{wherever } p(y) > 0 . \tag{2}\]
For continuous \(Y\), this should not be interpreted as dividing by \(\mathbb{P}[Y = y]\), which is typically zero. The equation above is a relation between densities.
Figure 1 is the picture behind Equation 2. Conditioning can be visualized in two steps. First, fix \(y = y_0\) and take the slice \(x \mapsto p(x, y_0)\). This slice is generally not normalized: its integral over \(x\) is \(p(y_0)\). Dividing the slice by this quantity gives \[ p(x \mid y_0) \;=\; \frac{p(x, y_0)}{p(y_0)} , \] whose integral over \(x\) is one.
The three panels of Figure 1 show the results of those two steps. The animation below shows the same construction dynamically. At \(y = y_0\), the joint density produces the slice \(x \mapsto p(x, y_0)\), whose integral is \(p(y_0)\). The slice is then multiplied by \(1/p(y_0)\) to obtain the conditional density.
Notice that normalization changes only the scale, not the shape of the slice. Thus \(p(x \mid y_0)\) has the same shape as \(p(x, y_0)\) as a function of \(x\), but only the former is normalized to integrate to one.
Three consequences of Equation 2 deserve to be stated, because each one is used later without comment.
The chain rule. Multiply Equation 2 out and you get \(p(x, y) = p(x \mid y)\, p(y)\). This factorization is also a standard way to construct joint distributions: choose a marginal \(p(y)\), then specify a conditional \(p(x \mid y)\). We will use this pattern repeatedly for latent-variable constructions.
Bayes’ rule. Apply the chain rule both ways and divide:
\[ p(y \mid x) \;=\; \frac{p(x \mid y)\, p(y)}{p(x)} , \qquad p(x) \;=\; \int p(x \mid y)\, p(y) \,\mathrm{d}y . \tag{3}\]
Conditioning on the wrong thing gives the wrong answer. \(p(x \mid y)\) and \(p(y \mid x)\) are different functions of different arguments, and the whole content of Equation 3 is the exchange rate between them.
Let \(Y \in \{0, 1\}\) be a class label with \(\mathbb{P}[Y = 1] = \pi\), and let the feature \(X \in \mathbb{R}\) satisfy \(X \mid Y = k \sim \mathcal{N}(\mu_k, \sigma^2)\). The two variables are of different types, so the two conditionals are different kinds of object, and they are written as such: \(p(x \mid Y = k)\) is a density in \(x\), and \(\mathbb{P}[Y = k \mid X = x]\) is a probability, a number in \([0, 1]\). For a discrete \(Y\), the integral in Equation 3 becomes a sum, and the two directions read \[ \begin{gathered} p(x) \;=\; \sum_{k \in \{0, 1\}} p(x \mid Y = k)\, \mathbb{P}[Y = k] , \\[4pt] \mathbb{P}[Y = k \mid X = x] \;=\; \frac{p(x \mid Y = k)\, \mathbb{P}[Y = k]}{p(x)} . \end{gathered} \] This keeps densities for the continuous variable \(X\) and probabilities for the discrete variable \(Y\) clearly separated.
Forward, with the Gaussian classes, the density of \(X\) is a mixture: \[ p(x) \;=\; (1 - \pi)\, \mathcal{N}(x; \mu_0, \sigma^2) \;+\; \pi\, \mathcal{N}(x; \mu_1, \sigma^2) . \]
Backward, the posterior on the label is a number in \([0,1]\): \[ \mathbb{P}[Y = 1 \mid X = x] \;=\; \frac{\pi\, \mathcal{N}(x; \mu_1, \sigma^2)} {(1-\pi)\, \mathcal{N}(x; \mu_0, \sigma^2) + \pi\, \mathcal{N}(x; \mu_1, \sigma^2)} . \]
Put \(\mu_0 = -1\), \(\mu_1 = +1\), \(\sigma = 1\), \(\pi = 1/2\) and simplify: the Gaussian normalizers cancel, and what survives is \(\mathbb{P}[Y=1 \mid X=x] = \sigma_{\mathrm{logistic}}(2x)\), the logistic function of \(2x\).
More generally, binary Gaussian classes with equal covariance produce posterior log-odds that are affine in \(x\): \[ \log \frac{\mathbb{P}[Y = 1 \mid X = x]}{\mathbb{P}[Y = 0 \mid X = x]} \;=\; a^\top x + b , \] for suitable \(a\) and \(b\) when the two Gaussian classes share the same covariance matrix. This is precisely the functional form used by logistic regression. In the numbers above, \(a = 2\) and \(b = 0\).
Independence, and why the conditional version is different
\(X\) and \(Y\) are independent, written \(X \perp Y\), when \[ p(x, y) \;=\; p(x)\,p(y) \] almost everywhere. Equivalently, \(p(x \mid y) = p(x)\) for almost every relevant \(x\) and \(y\): learning \(Y\) does not change the distribution of \(X\).
They are conditionally independent given \(Z\), written \(X \perp Y \mid Z\), when \[ p(x, y \mid z) \;=\; p(x \mid z)\, p(y \mid z) \] for \(p_Z\)-almost every \(z\), and almost every \(x, y\). In less technical language, the factorization may fail on sets of values having probability zero. Once \(Z\) is known, learning \(Y\) provides no additional information about \(X\).
Here and below, “almost everywhere” means that equality may fail on a set of probability (or Lebesgue measure, for densities) zero. Such changes do not alter the underlying probability law.
Neither statement implies the other. The self-test at the beginning of this section gives one direction: if \(X\) and \(Y\) are independent fair coin flips and \(Z = X \oplus Y\), then \(X \perp Y\) but \(X \not\perp Y \mid Z\). Figure 2 gives the opposite direction: \(X \perp Y \mid Z\), while marginalizing over \(Z\) makes \(X\) and \(Y\) dependent.
One number is used below to summarize how two coordinates move together, and it is worth defining before the picture. For random variables \(X\) and \(Y\) with finite, nonzero variances, their correlation is \[ \rho \;=\; \frac{\operatorname{Cov}(X, Y)}{\sigma_X \sigma_Y}, \qquad \operatorname{Cov}(X, Y) = \mathbb{E}_{(x,y) \sim p_{XY}}\big[(x - \mu_X)(y - \mu_Y)\big] , \tag{4}\] where \(\mu_X\) and \(\mu_Y\) are the means of \(X\) and \(Y\). The correlation satisfies \(-1 \le \rho \le 1\). A value \(\rho = 0\) means that \(X\) and \(Y\) are uncorrelated: covariance detects no linear association between them. This does not rule out nonlinear dependence. Under the same finite, nonzero variance assumptions, \(|\rho| = 1\) if and only if \[ Y \;=\; aX + b \] almost surely for some \(a \ne 0\) and \(b\). The sign records the slope: \(\rho = 1\) when \(a > 0\) and \(\rho = -1\) when \(a < 0\). This is the equality case of the Cauchy–Schwarz inequality, and it is not proved here. The sample correlation is the same formula with the two expectations replaced by averages over a finite sample, and it is what Figure 2 reports.
Zero correlation is weaker than independence. When the relevant second moments exist, independence implies zero covariance and hence zero correlation. The converse is false: uncorrelated variables may still be strongly dependent through a nonlinear relationship.
Figure 2 shows the direction that will be particularly useful later. A latent variable \(Z\) selects one of two clusters. Conditional on \(Z\), the coordinates are drawn independently, so within each cluster knowing \(X\) provides no information about \(Y\). After \(Z\) is hidden, the two clusters form a mixture in which \(X\) and \(Y\) are dependent. The marginal dependence therefore arises from mixing over the unobserved variable \(Z\).
This latent-variable pattern will reappear throughout the course: introduce a variable that makes a conditional distribution simple, then recover a more complicated marginal distribution by integrating that variable out. Figure 2 is a simple example of this mechanism.
Expectations, and the subscripts this course requires
For an integrable function \(g\) and a random variable \(X\) with density \(p\), \[ \mathbb{E}_{x \sim p}\big[g(x)\big] \;=\; \int g(x)\, p(x)\, \mathrm{d}x , \] where integrability means \[ \int |g(x)|\, p(x)\, \mathrm{d}x \;<\; \infty . \] For a discrete random variable, the corresponding expectation is defined by replacing the integral with a sum.
If \(g(X)\) and \(h(X)\) are integrable, expectation is linear: \[ \mathbb{E}_{x\sim p}\big[a\,g(x) + b\,h(x)\big] \;=\; a\,\mathbb{E}_{x\sim p}[g(x)] + b\,\mathbb{E}_{x\sim p}[h(x)] . \] No independence assumption is required. We will use this property repeatedly throughout the course.
Throughout the course, whenever more than one random quantity or distribution is involved, expectations carry explicit sampling subscripts naming the variable and the distribution it is drawn from. Write \(\mathbb{E}_{x \sim p}\) rather than a bare \(\mathbb{E}\).
The purpose is to make clear which variables are averaged over and under which distribution. For example, \[ \mathbb{E}\big[\,\|f_\theta(y) - x\|^2\,\big] \] does not specify whether \(x\) and \(y\) are sampled jointly, independently, or whether one of them is held fixed. It also does not specify whether \(\theta\) is random or a fixed parameter. Writing \[ \mathbb{E}_{(x, y) \sim p_{XY}}\big[\,\|f_\theta(y) - x\|^2\,\big] \] removes this ambiguity: \((x, y)\) is sampled jointly from \(p_{XY}\), while \(\theta\) is treated as a parameter. Section 5 drills this convention.
The same convention applies to time. From U3 onward, many losses include a random \(t \in [0, 1]\), and its sampling law must also be stated explicitly: \[ \mathbb{E}_{t \sim \mathrm{Unif}[0,1],\; x_1 \sim q}\big[\,\cdot\,\big] . \] A subscript containing only \(t\) does not specify how time is sampled. The course default is \(t \sim \mathrm{Unif}[0,1]\); whenever a different time distribution is used, it will be written explicitly.
The law of total expectation, informally. Suppose you draw \(Y\) first, then draw \(X\) given \(Y\), with \(p_X\) and \(p_Y\) the marginal laws of \(X\) and \(Y\) as above. An expectation under \(p_X\) can be computed in one pass, or in two: average within each value of \(Y\) first, then average those averages over \(Y\). For any integrable function \(g\),
\[ \mathbb{E}_{x \sim p_X}\big[g(x)\big] \;=\; \mathbb{E}_{y \sim p_Y}\Big[\, \mathbb{E}_{x \sim p(\cdot \mid y)}\big[g(x)\big] \,\Big] . \tag{5}\]
For \(g(x) = x\), this gives \[ \mathbb{E}_{x \sim p_X}[x] \;=\; \mathbb{E}_{y \sim p_Y}\Big[\, \mathbb{E}_{x \sim p(\cdot \mid y)}[x] \,\Big] . \] For each fixed \(y\), the inner expectation is a number or vector. As \(y\) varies, these conditional expectations define a function of \(y\), which is then averaged under the marginal law \(p_Y\).
Equation 5 is introduced here to build intuition and is used only in examples where it can be checked directly. UP.T1 gives the formal statement, its hypotheses, and its proof.
Let \(Y \sim \mathrm{Unif}\{1, 2, 3\}\) and \(X \mid Y = k \sim \mathcal{N}(k^2, 1)\).
Directly. The marginal density of \(X\) is \[ p_X(x) \;=\; \frac{1}{3} \sum_{k=1}^{3} \mathcal{N}(x; k^2, 1) . \] Therefore \[ \mathbb{E}_{x \sim p_X}[x] \;=\; \frac{1}{3}(1 + 4 + 9) \;=\; \frac{14}{3} . \]
By Equation 5. For fixed \(y\), \[ m(y) \;:=\; \mathbb{E}_{x \sim p(\cdot \mid y)}[x] \;=\; y^2 . \] Hence \[ \mathbb{E}_{y \sim \mathrm{Unif}\{1,2,3\}}\big[m(y)\big] \;=\; \frac{1}{3}(1^2 + 2^2 + 3^2) \;=\; \frac{14}{3} . \]
The two calculations agree. The conditional route is particularly useful when the conditional mean is easier to compute than the corresponding marginal expectation. For each fixed \(y\), the inner expectation is a number (or a vector when \(X\) is vector-valued); as \(y\) varies, these values define the function \[ m(y) \;=\; \mathbb{E}_{x \sim p(\cdot \mid y)}[x] . \] This viewpoint will be important in UP.T1.
The variance companion. The same two-stage bookkeeping splits a covariance into the part that comes from the inner draw and the part that comes from the outer one. PS1.1.4 uses it, so it is stated here; the proof is a few lines of Equation 5 and is left as optional. With \(m(y) = \mathbb{E}_{x \sim p(\cdot \mid y)}[x]\) as above and \(\mu_X = \mathbb{E}_{x \sim p_X}[x]\),
\[ \begin{aligned} \operatorname{Cov}(X) \;=\;&\; \mathbb{E}_{y \sim p_Y}\big[\operatorname{Cov}(X \mid Y = y)\big] \\ &+\; \mathbb{E}_{y \sim p_Y}\Big[\big(m(y) - \mu_X\big)\big(m(y) - \mu_X\big)^\top\Big] . \end{aligned} \tag{6}\]
For scalar \(X\), the corresponding identity is \[ \operatorname{Var}(X) \;=\; \mathbb{E}_{y \sim p_Y}\big[\operatorname{Var}(X \mid Y = y)\big] \;+\; \mathbb{E}_{y \sim p_Y}\Big[\big(m(y) - \mu_X\big)^2\Big] . \] The first term is the average variation of \(X\) within each conditional distribution. The second measures how the conditional mean \(m(y)\) varies as \(y\) changes. This is the usual within plus between decomposition.
In the example above, \[ \mathbb{E}_{y \sim \mathrm{Unif}\{1,2,3\}}\big[\operatorname{Var}(X \mid Y = y)\big] \;=\; 1 , \] since every conditional distribution has variance \(1\). The conditional mean is \(m(y) = y^2\), so \[ \mathbb{E}_{y \sim \mathrm{Unif}\{1,2,3\}}\Big[\Big(m(y) - \tfrac{14}{3}\Big)^2\Big] \;=\; \frac{98}{9} . \] Therefore \[ \operatorname{Var}(X) \;=\; 1 + \frac{98}{9} \;=\; \frac{107}{9} . \] As with Equation 5, the identity is used here only in an example; its derivation can be obtained from the law of total expectation.
- UP.T1 proves Equation 5 properly, and then proves the projection Lemma — the central technical device of this course — out of it.
- U3.T1 marginalizes a family of conditional densities exactly as Equation 1 does here, over a variable that will be a data point.
- The mandatory subscripts of Section 1.3 appear in every loss in every remaining session, without exception.
Exercises for §1
[PS1.1.1] Let \(p(x, y) = c\) on the region \(\{(x,y) : 0 \le y \le 1,\ 0 \le x \le 1 + y\}\) and \(p(x,y) = 0\) elsewhere.
(a) Find \(c\).
(b) Compute the marginals \(p_Y(y)\) and \(p_X(x)\); note that they are not of the same form.
(c) Compute the conditional \(p(x \mid y)\) and state, in one sentence, how it depends on \(y\).
(d) Compute the conditional density \(p(y \mid x)\) for \(x = 1/2\) and for \(x = 3/2\), and explain why the two answers have different supports.
[PS1.1.2] Draw \(Y \sim \mathrm{Exp}(1)\). For \(y > 0\), let \(X \mid Y = y \sim \mathcal{N}(y, y^2)\).
(a) Compute \(\mathbb{E}_{x \sim p_X}[x]\) directly. First write the marginal density \[ p_X(x) \;=\; \int_{0}^{\infty} p(x \mid y)\, p_Y(y)\, \mathrm{d}y , \] and then exchange the order of integration. You may assume the required absolute integrability for this exchange.
(b) Compute \(\mathbb{E}_{x \sim p_X}[x]\) using Equation 5.
(c) Apply Equation 5 with \(g(x) = x^2\) to compute \(\mathbb{E}_{x \sim p_X}[x^2]\), and hence compute \(\operatorname{Var}(X)\).
(d) Suppose the conditional law is still \(X \mid Y = y \sim \mathcal{N}(y, y^2)\), but the marginal law of \(Y\) can only be sampled from, rather than integrated against analytically. Which of the two routes above remains directly usable? Explain briefly how you would approximate the outer expectation from samples of \(Y\).
Slower treatment of §1, optional: (Bishop and Bishop 2024) ch. 2, (Murphy 2023) ch. 2.
§2 — Gaussian algebra
- \(x \sim \mathcal{N}(\mu, \Sigma)\) and \(A\) is a matrix. What is the law of \(Ax + b\)?
- \((x_a, x_b)\) is jointly Gaussian. Is \(x_a \mid x_b\) Gaussian? Is its covariance a function of \(x_b\)?
- Let \(A\) be symmetric and invertible. Write \(x^\top A x - 2 b^\top x\) as a single quadratic term plus a constant.
Answers in the footnote.2
Gaussian identities account for many of the closed-form calculations used later in the course. This section collects five results that will be reused repeatedly. The boxed formulas also appear separately in the two-page gaussian-algebra-toolkit sheet (see the toolkit page).
2.1 The Gaussian, and the first pushforward
We define a Gaussian random vector through an affine transformation of a standard Gaussian. This definition includes both nondegenerate and degenerate Gaussian laws; a Lebesgue density exists only in the nondegenerate case.
Let \(z \sim \mathcal{N}(0, I_k)\), where the components of \(z\) are independent standard normal random variables. For \(\mu \in \mathbb{R}^d\) and \(L \in \mathbb{R}^{d \times k}\), define \[ x \;=\; \mu + L z . \tag{7}\] Then \(x\) is a Gaussian random vector with \[ \mathbb{E}[x] \;=\; \mu , \qquad \operatorname{Cov}(x) \;=\; L L^\top \;=:\; \Sigma \;\succeq\; 0 . \] Its law is denoted by \(\mathcal{N}(\mu, \Sigma)\). Although different matrices \(L\) may produce the same covariance \(\Sigma\), they produce the same Gaussian law; thus the distribution is determined by \(\mu\) and \(\Sigma\). This primer uses that fact and does not prove it (a slower treatment: (Bishop and Bishop 2024) ch. 3).
If \(\Sigma\) is singular, the Gaussian law is still well defined, but it is supported on the affine subspace \[ \mu + \operatorname{range}(L) . \] It therefore has no density with respect to \(d\)-dimensional Lebesgue measure on \(\mathbb{R}^d\). Such degenerate Gaussian laws are allowed throughout the course as probability laws, but Equation 8 below, which uses \(\Sigma^{-1}\) and \(\det \Sigma\), applies only when \(\Sigma \succ 0\).
For \(\mu \in \mathbb{R}^d\) and a symmetric positive definite \(\Sigma \in \mathbb{R}^{d \times d}\) — the nondegenerate case — the law \(\mathcal{N}(\mu, \Sigma)\) has the Lebesgue density \[ \mathcal{N}(x; \mu, \Sigma) \;=\; \frac{1}{(2\pi)^{d/2} \,(\det \Sigma)^{1/2}} \exp\!\left( -\frac{1}{2} (x - \mu)^\top \Sigma^{-1} (x - \mu) \right) . \tag{8}\] The standard Gaussian is \(\mathcal{N}(0, I_d)\). In this course it is the source distribution, written \(p_0\), and its samples are written \(x_0\).
The exponent of Equation 8 is a quadratic form in \(x\), and the prefactor is whatever makes the integral equal \(1\). That is the entire structure. Every manipulation in this section is a manipulation of the quadratic form, with the prefactor recovered at the end by inspection.
Here is the first, and the one that names the pattern. For \(\Sigma \succ 0\) it is Equation 7 read as a recipe.
Let \(z \sim \mathcal{N}(0, I_d)\) and let \(\Sigma^{1/2}\) be any matrix with \(\Sigma^{1/2} (\Sigma^{1/2})^\top = \Sigma\), \(\Sigma \succ 0\). Then \[ x \;=\; \mu + \Sigma^{1/2} z \;\sim\; \mathcal{N}(\mu, \Sigma) . \tag{9}\]
Equation 9 gives a sampling recipe. Define \[ T(z) \;=\; \mu + \Sigma^{1/2} z . \] If \(z \sim \mathcal{N}(0, I_d)\), then \(T(z) \sim \mathcal{N}(\mu, \Sigma)\). Equivalently, the law \(\mathcal{N}(\mu, \Sigma)\) is the pushforward of \(\mathcal{N}(0, I_d)\) by the deterministic map \(T\). UP.T1 will define pushforwards formally.
The pattern to remember is: a simple source distribution, followed by a deterministic map, produces a new distribution as its image.
2.2 Linear transforms
\[ x \sim \mathcal{N}(\mu, \Sigma) \quad\Longrightarrow\quad A x + b \;\sim\; \mathcal{N}\!\big(A\mu + b,\; A \Sigma A^\top\big) . \tag{10}\]
Two things have to be shown, and they are shown differently.
The moments. By linearity of expectation, \(\mathbb{E}_{x \sim \mathcal{N}(\mu, \Sigma)}[Ax + b] = A\mu + b\). For the covariance, write \(y = Ax + b\) and \(\bar y = A\mu + b\), so that \(y - \bar y = A(x - \mu)\) and \[ \begin{aligned} \operatorname{Cov}(y) &= \mathbb{E}_{x \sim \mathcal{N}(\mu, \Sigma)}\big[A(x-\mu)(x-\mu)^\top A^\top\big] \\ &= A\,\mathbb{E}_{x \sim \mathcal{N}(\mu, \Sigma)}\big[(x-\mu)(x-\mu)^\top\big]\,A^\top = A \Sigma A^\top . \end{aligned} \] The matrix \(A\) comes out of the expectation because it is constant, and what remains inside is the definition of \(\Sigma\).
The family. Moments alone are not enough: a non-Gaussian variable can share the first two moments of a Gaussian. What closes the argument is that the Gaussian family is closed under affine maps, and Equation 7 is the reason. If \(x = \mu + L z\) with \(z\) standard, then \[ Ax + b \;=\; (A\mu + b) \;+\; (A L)\, z , \] which is again an affine map of a standard Gaussian, hence again Gaussian by definition, with covariance \((AL)(AL)^\top = A \Sigma A^\top\).
Together, these observations prove Equation 10. The affine representation shows that \(Ax + b\) remains Gaussian, while the moment calculation gives its parameters. Notice that \[ A \Sigma A^\top \] may be singular even when \(\Sigma\) is positive definite; a projection is a simple example. This is one reason it is useful to include degenerate Gaussian laws in the definition.
As an immediate corollary, selecting a subset of coordinates with an appropriate matrix \(A\) — for instance \(A = \begin{bmatrix} 0 & I \end{bmatrix}\) — shows that every marginal of a jointly Gaussian vector is again Gaussian, with mean and covariance read off the corresponding blocks. Section 2.5 uses this.
2.3 Sums of independent Gaussians
\[ \begin{aligned} x \sim \mathcal{N}(\mu_x, \Sigma_x),\quad y \sim \mathcal{N}(\mu_y, \Sigma_y),\quad x \perp y \\ \Longrightarrow\quad x + y \sim \mathcal{N}(\mu_x + \mu_y,\; \Sigma_x + \Sigma_y) . \end{aligned} \tag{11}\]
Because \(x\) and \(y\) are independent Gaussian vectors, the stacked vector \((x, y)\) is jointly Gaussian, with mean and block-diagonal covariance \[ \begin{bmatrix} \mu_x \\ \mu_y \end{bmatrix} , \qquad \begin{bmatrix} \Sigma_x & 0 \\ 0 & \Sigma_y \end{bmatrix} . \] Applying Equation 10 with \(A = \begin{bmatrix} I & I \end{bmatrix}\) gives Equation 11.
Independence is sufficient both to make the stacked vector jointly Gaussian and to eliminate the cross-covariance terms. Without independence, Equation 11 is not guaranteed. If \(x\) and \(y\) are jointly Gaussian but not necessarily independent, then \(x + y\) is still Gaussian, but \[ \operatorname{Cov}(x + y) \;=\; \Sigma_x + \Sigma_y + \operatorname{Cov}(x, y) + \operatorname{Cov}(y, x) . \]
Now a special case, stated separately because U3.T1 reuses it.
Let \(x_0 \sim \mathcal{N}(0, I_d)\), let \(x_1 \in \mathbb{R}^d\) be fixed, and let \(\alpha, \sigma \in \mathbb{R}\) with \(\sigma \ge 0\). Then \[ \alpha x_1 + \sigma x_0 \;\sim\; \mathcal{N}\big(\alpha x_1,\; \sigma^2 I_d\big) . \tag{12}\] If \(x_1\) is instead random, with \(x_1 \sim q\) and \(x_0 \perp x_1\), then Equation 12 describes the conditional law: \[ x \mid x_1 \;\sim\; \mathcal{N}\big(\alpha x_1,\; \sigma^2 I_d\big), \qquad x \;=\; \alpha x_1 + \sigma x_0 . \] Marginalizing over \(x_1\) produces a Gaussian mixture whose mixing distribution is \(q\). Unless additional assumptions are imposed on \(q\), this marginal distribution need not itself be Gaussian.
The endpoint \(\sigma = 0\). At \(\sigma = 0\), \(x \mid x_1 = \alpha x_1\) deterministically, so \[ x \mid x_1 \;\sim\; \delta_{\alpha x_1} . \] In the Gaussian-law notation of Section 2.1, this may also be written \(\mathcal{N}(\alpha x_1, 0)\). It is a degenerate Gaussian law and has no density with respect to Lebesgue measure on \(\mathbb{R}^d\). Equation 8 must therefore not be used with \(\Sigma = 0\).
The proof is one line of Equation 10 with \(A = \sigma I_d\) and \(b = \alpha x_1\). The content is entirely in the conditional qualifier, and in Section 1.2’s picture: simple given the latent, complicated once the latent is hidden.
Equation 12 will reappear in U3.T1 with time-dependent coefficients \(\alpha_t\) and \(\sigma_t\). The resulting conditional Gaussian family is one of the main constructions used in the flow-matching part of the course.
2.4 Completing the square
Every remaining derivation in this section is one application of the following identity. It is worth learning by heart, because it is the only trick.
Let \(A\) be symmetric and invertible, and let \(b\) be a vector. Then \[ x^\top A x \;-\; 2\, b^\top x \;=\; (x - A^{-1} b)^\top A\, (x - A^{-1} b) \;-\; b^\top A^{-1} b . \tag{13}\]
Proof. Expand the first term on the right: \[ (x - A^{-1}b)^\top A (x - A^{-1}b) = x^\top A x - x^\top A A^{-1} b - b^\top A^{-1} A x + b^\top A^{-1} A A^{-1} b . \] The two middle terms are each the scalar \(b^\top x\), because \(A\) is symmetric and so is \(A^{-1}\); the last term is \(b^\top A^{-1} b\). Subtracting \(b^\top A^{-1} b\) from both sides gives Equation 13. \(\square\)
To identify a Gaussian density, first collect all terms depending on \(x\) into \[ -\frac{1}{2} \big( x^\top A x - 2\, b^\top x \big) + \text{const} . \] If \(A \succ 0\), Equation 13 gives \[ -\frac{1}{2}\, (x - A^{-1} b)^\top A\, (x - A^{-1} b) \;+\; \frac{1}{2}\, b^\top A^{-1} b \;+\; \text{const} . \] The terms independent of \(x\) are absorbed into the normalizing constant. Comparing with the Gaussian density shows that \[ \mu \;=\; A^{-1} b , \qquad \Sigma \;=\; A^{-1} . \]
The absorbed constant is what makes the method cheap: you never track the prefactor. You identify the quadratic form, and the prefactor is whatever normalizes it.
Equation 13 is an algebraic identity for symmetric invertible \(A\). To interpret the completed expression as a normalizable Gaussian density, however, one needs \(A \succ 0\). In that case \(A\) is the precision matrix and \(A^{-1}\) is the covariance matrix. Every use below has \(A\) a precision matrix, and the hypothesis is stated where it is used.
2.5 Conditionals and marginals of a joint Gaussian
This is the centerpiece of the primer. Let \(z = \begin{bmatrix} x_a \\ x_b \end{bmatrix}\) be jointly Gaussian, with the blocks \[ \mu = \begin{bmatrix} \mu_a \\ \mu_b \end{bmatrix}, \qquad \Sigma = \begin{bmatrix} \Sigma_{aa} & \Sigma_{ab} \\ \Sigma_{ba} & \Sigma_{bb} \end{bmatrix}, \qquad \Sigma_{ba} = \Sigma_{ab}^\top . \]
The marginals are free. By the corollary of Section 2.2, \(x_a \sim \mathcal{N}(\mu_a, \Sigma_{aa})\) and \(x_b \sim \mathcal{N}(\mu_b, \Sigma_{bb})\). You read them off the blocks, and there is nothing to compute.
The conditional takes one application of Equation 13. Assume, for the derivation, that the joint covariance is nondegenerate, \(\Sigma \succ 0\), so that the joint density exists and the precision \(\Lambda = \Sigma^{-1}\) is defined; then every principal block, \(\Sigma_{bb}\) in particular, is positive definite and invertible too. Write \(\Lambda\) in blocks, \(\Lambda_{aa}, \Lambda_{ab}, \Lambda_{bb}\). The exponent of the joint density is \(-\frac{1}{2} (z - \mu)^\top \Lambda (z - \mu)\), and expanding the blocks with \(u = x_a - \mu_a\) and \(w = x_b - \mu_b\) gives \[ -\frac{1}{2}\Big( u^\top \Lambda_{aa} u \;+\; 2\, u^\top \Lambda_{ab} w \;+\; w^\top \Lambda_{bb} w \Big). \] Now fix \(x_b\), so that \(w\) is a constant. The last term is constant, and drops into the normalizer. What remains is quadratic in \(u\), in exactly the shape Equation 13 wants, with \(A = \Lambda_{aa}\) and \(b = -\Lambda_{ab} w\). Reading off the mean and covariance: \[ \begin{aligned} x_a \mid x_b \;\sim\; \mathcal{N}\big( m,\; C \big), \qquad m &= \mu_a - \Lambda_{aa}^{-1} \Lambda_{ab}(x_b - \mu_b), \\ C &= \Lambda_{aa}^{-1} . \end{aligned} \tag{14}\]
Equation 14 is already the answer, but it is written in the blocks of the inverse covariance, and you are usually handed \(\Sigma\). The translation is one identity.
Write \(S = \Sigma_{aa} - \Sigma_{ab} \Sigma_{bb}^{-1} \Sigma_{ba}\), the Schur complement of \(\Sigma_{bb}\). If \(\Sigma\) and \(\Sigma_{bb}\) are invertible, so is \(S\), and \[ \Lambda_{aa} \;=\; S^{-1} , \qquad \Lambda_{ab} \;=\; -\,S^{-1} \Sigma_{ab} \Sigma_{bb}^{-1} . \tag{15}\]
Proof. It is enough to exhibit a matrix and check that it is the inverse. Take \(\Lambda_{aa} = S^{-1}\), \(\Lambda_{ab} = -S^{-1}\Sigma_{ab}\Sigma_{bb}^{-1}\) and \(\Lambda_{bb} = \Sigma_{bb}^{-1} + \Sigma_{bb}^{-1}\Sigma_{ba} S^{-1}\Sigma_{ab}\Sigma_{bb}^{-1}\). Multiplying out \(\Sigma \Lambda\) block by block, the top-left block is \(\Sigma_{aa} S^{-1} - \Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba}S^{-1} = S S^{-1} = I\), and the top-right block is \(-\Sigma_{aa}S^{-1}\Sigma_{ab}\Sigma_{bb}^{-1} + \Sigma_{ab}\Sigma_{bb}^{-1} + \Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba}S^{-1}\Sigma_{ab}\Sigma_{bb}^{-1} = (-S S^{-1} + I)\,\Sigma_{ab}\Sigma_{bb}^{-1} = 0\). The two lower blocks are the same computation transposed. \(\square\)
Substituting Equation 15 into Equation 14 gives the formula this course uses. Note that \(\Lambda_{aa}^{-1}\Lambda_{ab} = -\Sigma_{ab}\Sigma_{bb}^{-1}\), so the two minus signs cancel.
\[ \boxed{\; x_a \mid x_b \;\sim\; \mathcal{N}\Big(\; \mu_a + \Sigma_{ab}\Sigma_{bb}^{-1}(x_b - \mu_b), \;\; \Sigma_{aa} - \Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba} \;\Big) \; } \tag{16}\]
Three observations, each of which is used later.
The conditional mean is affine in \(x_b\). It is the marginal mean, corrected by a linear function of how far \(x_b\) landed from its mean. The correction matrix \(\Sigma_{ab}\Sigma_{bb}^{-1}\) is a regression coefficient. If \(\Sigma_{ab} = 0\), the conditional mean reduces to \(\mu_a\) and the conditional covariance reduces to \(\Sigma_{aa}\). Thus jointly Gaussian blocks with zero cross-covariance are independent. This implication is special to the Gaussian family; zero covariance does not imply independence in general.
The conditional covariance does not depend on \(x_b\) at all. Observing \(x_b\) reduces your uncertainty about \(x_a\) by the same amount no matter what value you observe. This is peculiar to the Gaussian family, and it is the single property that makes everything downstream tractable.
The conditional covariance is no larger than the marginal one. In this jointly Gaussian setting, the conditional covariance satisfies \[ \Sigma_{aa} - \Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba} \;\preceq\; \Sigma_{aa} , \] because \(\Sigma_{ab}\Sigma_{bb}^{-1}\Sigma_{ba} \succeq 0\). Thus the conditional covariance is no larger than the marginal covariance in the Loewner order. The statement is about the jointly Gaussian case; for general distributions, a conditional covariance at a particular conditioning value need not be smaller than its marginal counterpart.
The scalar corollary
You will use the one-dimensional case constantly, so it is written out separately. Let \((X, Y)\) be jointly Gaussian with means \(\mu_X, \mu_Y\), variances \(\sigma_X^2 > 0\), \(\sigma_Y^2 > 0\) and covariance \(\sigma_{XY}\). Then
\[ X \mid Y = y \;\sim\; \mathcal{N}\Big(\; \mu_X + \frac{\sigma_{XY}}{\sigma_Y^2}\,(y - \mu_Y), \;\; \sigma_X^2 - \frac{\sigma_{XY}^2}{\sigma_Y^2} \;\Big) . \tag{17}\]
Writing \(\rho = \sigma_{XY}/(\sigma_X \sigma_Y)\) for the correlation, the conditional variance becomes \[ \operatorname{Var}(X \mid Y = y) \;=\; \sigma_X^2 (1 - \rho^2) . \] Thus the conditional variance is the fraction \(1 - \rho^2\) of the marginal variance of \(X\).
2.6 A worked example, to be remembered
Let \(X \sim \mathcal{N}(\mu_0, \sigma_0^2)\) with \(\sigma_0^2 > 0\), and suppose that we observe \[ Y \;=\; X + \sigma \varepsilon , \qquad \varepsilon \sim \mathcal{N}(0, 1), \qquad \varepsilon \perp X , \qquad \sigma > 0 . \] What is the conditional distribution of \(X\) given \(Y = y\)? In particular, what are its conditional mean and variance?
Step 1 — the pair is jointly Gaussian. The vector \((X, Y)\) is an affine map of the independent Gaussian pair \((X, \varepsilon)\), namely \(\begin{bmatrix} X \\ Y \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 1 & \sigma \end{bmatrix} \begin{bmatrix} X \\ \varepsilon \end{bmatrix}\), so Equation 10 applies and the pair is Gaussian.
Step 2 — read off the moments. Both variables have mean \(\mu_0\). Their variances are \(\operatorname{Var}(X) = \sigma_0^2\) and \[ \operatorname{Var}(Y) \;=\; \operatorname{Var}(X + \sigma\varepsilon) \;=\; \sigma_0^2 + \sigma^2 , \] where independence eliminates the cross-covariance term. Similarly, \[ \operatorname{Cov}(X, Y) \;=\; \operatorname{Cov}(X, X + \sigma\varepsilon) \;=\; \operatorname{Var}(X) + \sigma \operatorname{Cov}(X, \varepsilon) \;=\; \sigma_0^2 . \]
Step 3 — apply Equation 17. With \(\sigma_{XY} = \operatorname{Cov}(X, Y) = \sigma_0^2\) and \(\sigma_Y^2 = \operatorname{Var}(Y) = \sigma_0^2 + \sigma^2\): \[ \begin{aligned} \mathbb{E}\big[X \mid Y = y\big] &\;=\; \frac{\sigma_0^2}{\sigma_0^2 + \sigma^2}\, y \;+\; \frac{\sigma^2}{\sigma_0^2 + \sigma^2}\, \mu_0 , \\[2pt] \operatorname{Var}\big(X \mid Y = y\big) &\;=\; \frac{\sigma_0^2 \sigma^2}{\sigma_0^2 + \sigma^2} . \end{aligned} \tag{18}\]
Read the answer. The conditional mean in Equation 18 interpolates between the observation \(y\) and the prior mean \(\mu_0\). Equivalently, it can be written in precision-weighted form as \[ \mathbb{E}\big[X \mid Y = y\big] \;=\; \frac{\sigma^{-2}\, y + \sigma_0^{-2}\, \mu_0}{\sigma^{-2} + \sigma_0^{-2}} . \] The conditional variance can be written as \[ \operatorname{Var}\big(X \mid Y = y\big) \;=\; \frac{\sigma_0^2 \sigma^2}{\sigma_0^2 + \sigma^2} \;=\; \big( \sigma_0^{-2} + \sigma^{-2} \big)^{-1} . \] As \(\sigma \to 0\), the observation becomes effectively noise-free: the conditional mean approaches \(y\) and the conditional variance approaches zero. As \(\sigma \to \infty\), the observation becomes uninformative: the conditional mean approaches \(\mu_0\) and the conditional variance approaches \(\sigma_0^2\).
Equation 18 will reappear later in a different form. For now, the important point is that it follows directly from the conditional formula for a jointly Gaussian pair. Keep the result available for later reference.
- U3.T1 item 4 builds its conditional families out of Equation 12, with \(\alpha_t\) and \(\sigma_t\) in place of the two constants.
- Detour D1 studies which choices of those two functions of time recover which classical constructions.
- Detour D2 rests on Equation 18, in the different form mentioned above.
- UP.T1 reuses the example of Section 2.6 for problem set PS1.2.
- The boxed formulas of this section are the
gaussian-algebra-toolkitsheet, consumed by name in the U3 sessions.
Exercises for §2
[PS1.1.3] Let \((X_1, X_2)\) be jointly Gaussian with mean \((1, -2)\) and covariance \(\Sigma = \begin{bmatrix} 4 & 2 \\ 2 & 3 \end{bmatrix}\).
(a) Write \(X_1 \mid X_2 = x_2\) explicitly, using Equation 17.
(b) Compute the correlation \(\rho\) and verify that the conditional variance equals \(\sigma_{X_1}^2 (1 - \rho^2)\).
(c) Invert \(\Sigma\) by hand and confirm Equation 15 on this example, that is, confirm \(\Lambda_{11} = S^{-1}\) with \(S = \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}\).
(d) By what percentage, relative to the marginal variance of \(X_1\), does observing \(X_2\) reduce the conditional variance?
[PS1.1.4] Let \(x_0 \sim \mathcal{N}(0, I_d)\) and let \(x_1 \sim q\) be independent of \(x_0\), where \(q\) is any distribution on \(\mathbb{R}^d\) with mean \(m\) and covariance \(C\). Fix constants \(\alpha\) and \(\sigma\), and define \(x = \alpha x_1 + \sigma x_0\).
(a) Show that, conditional on \(x_1\), \(x \mid x_1 \sim \mathcal{N}(\alpha x_1, \sigma^2 I_d)\).
(b) Compute \(\mathbb{E}[x]\) using Equation 5.
(c) Compute \(\operatorname{Cov}(x)\) using the law of total covariance Equation 6, and identify which term comes from the Gaussian noise and which comes from the variability of the data distribution \(q\).
(d) Is the marginal distribution of \(x\) guaranteed to be Gaussian from the assumptions above? Explain briefly. What additional assumption on \(q\) would guarantee that \(x\) is Gaussian?
(e) Set \(\sigma = 0\). What is the conditional law of \(x\) given \(x_1\)? Does it have a density on \(\mathbb{R}^d\)? Do your answers to parts (b) and (c) still hold?
[PS1.1.5] A drill on Equation 13.
(a) Let \(A = \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}\) and \(b = \begin{bmatrix} 1 \\ 4 \end{bmatrix}\). Write \(x^\top A x - 2 b^\top x\) in the completed form, giving \(A^{-1}b\) and \(b^\top A^{-1} b\) numerically.
(b) A density is known to be proportional to \(\exp(-\frac{1}{2} x^\top A x + b^\top x)\) with the same \(A\) and \(b\). Name the distribution: give its mean and covariance, without computing any normalizing constant.
(c) Explain in one sentence why part (b) needed no integral.
Slower treatment of §2, optional: (Bishop and Bishop 2024) ch. 3, (Murphy 2023) ch. 2.
§3 — Divergences
- Write \(\mathrm{KL}(p \Vert q)\) as an expectation, with the subscript.
- Is \(\mathrm{KL}(p \Vert q) = \mathrm{KL}(q \Vert p)\)? Is either ever negative?
- Fitting a single Gaussian \(q\) to a two-humped target \(p\): which direction of KL puts \(q\) on one hump, and which spreads it across both?
Answers in the footnote.3
We need a way to compare probability distributions. The main divergence used in this course is the Kullback–Leibler (KL) divergence. This section introduces its definition, basic properties, and the closed form for Gaussian distributions.
3.1 Definition
Let \(p\) and \(q\) be densities with respect to the same reference measure. When \(q(x) > 0\) for \(p\)-almost every \(x\) such that \(p(x) > 0\), define \[ \begin{aligned} \mathrm{KL}(p \,\Vert\, q) \;=\; \mathbb{E}_{x \sim p}\big[\log p(x) - \log q(x)\big] \;=\; \int p(x) \log \frac{p(x)}{q(x)} \,\mathrm{d}x . \end{aligned} \tag{19}\] The double bar is the course’s notation, and the order of the arguments matters: the expectation is taken with respect to the first distribution.
At the level of probability laws, the corresponding support condition is absolute continuity, written \(P \ll Q\). If \(Q\) assigns zero probability to a set to which \(P\) assigns positive probability, we define \[ \mathrm{KL}(P \,\Vert\, Q) \;=\; +\infty . \] Thus finite KL divergence requires \(P \ll Q\). The converse is not automatic: even when \(P \ll Q\), the integral defining KL may still diverge to \(+\infty\). KL is therefore naturally an extended-valued divergence, taking values in \([0, +\infty]\).
Two features of Equation 19 are worth naming at once.
It is not a distance. It is not symmetric, and it does not satisfy a triangle inequality. It is called a divergence for precisely that reason.
The expectation is under the first argument. Equation 19 is an expectation under the first distribution. Consequently, a Monte Carlo estimate of \(\mathrm{KL}(p \Vert q)\) uses samples from \(p\): \(x^{(i)} \sim p\). This distinction will matter when the first distribution is the data distribution: a dataset provides samples from that distribution even when its density is not available analytically.
3.2 Non-negativity
Let \(\varphi\) be convex and let \(Z\) be an integrable random variable such that \(\varphi(Z)\) is also integrable. Jensen’s inequality states \[ \varphi\big(\mathbb{E}[Z]\big) \;\le\; \mathbb{E}\big[\varphi(Z)\big] . \tag{20}\]
Geometric argument. For the differentiable convex functions used here, convexity implies that the graph lies above its tangent line. Let \(\bar z = \mathbb{E}[Z]\). The tangent at \(\bar z\) has the form \[ \ell(z) \;=\; \varphi(\bar z) + \varphi'(\bar z)\,(z - \bar z) . \] Since \(\varphi(z) \ge \ell(z)\), \[ \mathbb{E}\big[\varphi(Z)\big] \;\ge\; \mathbb{E}\big[\ell(Z)\big] \;=\; \varphi(\bar z) , \] because \(\mathbb{E}[Z - \bar z] = 0\). \(\square\)
The geometric argument assumes a tangent line exists at \(\bar z\), which is true for the differentiable convex functions we use. The general nondifferentiable case follows from the subgradient form of the same argument; it is standard and not proved here (see (Bishop and Bishop 2024) ch. 2 or (Murphy 2023) §5.1.2.2).
\(\mathrm{KL}(p \,\Vert\, q) \ge 0\), with equality if and only if \(p = q\) almost everywhere.
Proof. If \(\mathrm{KL}(p \,\Vert\, q) = +\infty\), the inequality is immediate. Otherwise \(q(x) > 0\) for \(p\)-almost every \(x\) with \(p(x) > 0\). On the set \(A = \{x : p(x) > 0\}\), define \[ Z(x) \;=\; \frac{q(x)}{p(x)} . \] Then \[ \mathrm{KL}(p \,\Vert\, q) \;=\; \mathbb{E}_{x \sim p}\big[ -\log Z(x) \big] . \] Since \(-\log z\) is convex, Equation 20 gives \[ \mathrm{KL}(p \,\Vert\, q) \;\ge\; -\log \mathbb{E}_{x \sim p}\big[ Z(x) \big] . \] But \[ \mathbb{E}_{x \sim p}\big[ Z(x) \big] \;=\; \int_A p(x)\, \frac{q(x)}{p(x)} \,\mathrm{d}x \;=\; \int_A q(x)\,\mathrm{d}x \;\le\; 1 . \] Therefore \[ \mathrm{KL}(p \,\Vert\, q) \;\ge\; -\log\Big( \int_A q(x)\,\mathrm{d}x \Big) \;\ge\; 0 . \] For equality to hold, both inequalities must be equalities. Strict convexity of \(-\log\) requires \(q(x)/p(x)\) to be constant \(p\)-almost everywhere, while the second inequality requires \(\int_A q(x)\,\mathrm{d}x = 1\). Since both \(p\) and \(q\) integrate to one, the constant must be \(1\), so \(p = q\) almost everywhere. \(\square\)
The support condition is essential. If \(q\) vanishes on a region having positive probability under \(p\), then \(\mathrm{KL}(p \,\Vert\, q) = +\infty\). Thus this direction of KL strongly penalizes assigning zero density to events that can occur under the first distribution.
3.3 Asymmetry, and what each direction punishes
Figure 3 considers a deliberately restricted approximation problem: a bimodal target distribution \(p\), one whose density has two separate humps, is approximated by a single Gaussian \(q\). Because the Gaussian family cannot represent both modes exactly, the two directions of KL produce different compromises.
In this setting it is common to describe the resulting behaviors as mass-covering and mode-seeking. In the figure, minimizing \(\mathrm{KL}(p \Vert q)\) over \(q\) gives the first and minimizing \(\mathrm{KL}(q \Vert p)\) gives the second; the paragraphs after the figure say why. These labels describe typical behavior under a restricted approximation family; they are not general theorems about the two KL directions. If the approximation family contains the target distribution itself, both divergences are minimized by the same solution.
The mechanism behind the heuristic is visible in Equation 19, and it is worth seeing, because it says what each direction charges for — the family being fitted then decides what the minimizer can afford. The integrand is weighted by the first argument.
\(\mathrm{KL}(p \Vert q)\) — mass-covering. The expectation is taken under \(p\). Regions in which \(p\) has appreciable probability therefore contribute strongly to the objective. If \(q\) assigns very small density to such a region, the term \(\log\big(p(x)/q(x)\big)\) becomes large. In restricted approximation families, this often favors solutions that allocate density across the regions occupied by \(p\), which motivates the term mass-covering.
\(\mathrm{KL}(q \Vert p)\) — mode-seeking. The expectation is instead taken under \(q\). The objective therefore penalizes regions where the approximation \(q\) places substantial probability but the target \(p\) has very small density. In multimodal problems with a restricted unimodal approximation family, this can favor concentrating on one high-density region of \(p\) rather than placing probability in the low-density region between modes. This motivates the term mode-seeking. It is a behavior of the approximation problem, not a theorem that reverse KL must select one mode.
This distinction becomes relevant in generative modeling because maximum-likelihood training of a model \(p_\theta\) is equivalent, up to a term independent of \(\theta\), to minimizing \(\mathrm{KL}(q \,\Vert\, p_\theta)\), where \(q\) denotes the data distribution. The data distribution therefore occupies the first argument. UP.T1 derives this equivalence.
3.4 KL between Gaussians
We now specialize Equation 19 to two nondegenerate Gaussian distributions. Let \[ p \;=\; \mathcal{N}(\mu_0, \Sigma_0), \qquad q \;=\; \mathcal{N}(\mu_1, \Sigma_1), \qquad \Sigma_0 \succ 0,\ \ \Sigma_1 \succ 0 . \] Then
\[ \begin{aligned} \mathrm{KL}\big(\mathcal{N}(\mu_0, \Sigma_0) \,\big\Vert\, \mathcal{N}(\mu_1, \Sigma_1)\big) = \frac{1}{2}\Big[ &\operatorname{tr}(\Sigma_1^{-1}\Sigma_0) \\ + &(\mu_1 - \mu_0)^\top \Sigma_1^{-1} (\mu_1 - \mu_0) \\ - &\; d + \log \tfrac{\det \Sigma_1}{\det \Sigma_0} \Big] . \end{aligned} \tag{21}\]
The positive-definiteness assumptions ensure that both Gaussian laws have densities on \(\mathbb{R}^d\) and that the inverses and log-determinants in Equation 21 are well defined.
Derivation. Taking logs of Equation 8, the two log-normalizers are constants and the two exponents are quadratic forms, so \[ \log \frac{p(x)}{q(x)} = \frac{1}{2} \log \frac{\det \Sigma_1}{\det \Sigma_0} - \frac{1}{2} (x - \mu_0)^\top \Sigma_0^{-1} (x - \mu_0) + \frac{1}{2} (x - \mu_1)^\top \Sigma_1^{-1} (x - \mu_1) . \] Take \(\mathbb{E}_{x \sim p}\) of each term. The first is constant. For the other two, use the identity that turns a quadratic form into a trace: for any matrix \(M\) of compatible size and vector \(v\), \[ v^\top M v \;=\; \operatorname{tr}\big(M v v^\top\big) . \] Therefore \[ \mathbb{E}\big[ v^\top M v \big] \;=\; \operatorname{tr}\big( M\, \mathbb{E}[v v^\top] \big) , \] whenever the expectation exists.
Second term. With \(v = x - \mu_0\) we have \(\mathbb{E}_{x \sim p}[v v^\top] = \Sigma_0\) by definition, so \[ \mathbb{E}_{x \sim p}\big[(x-\mu_0)^\top \Sigma_0^{-1} (x - \mu_0)\big] = \operatorname{tr}(\Sigma_0^{-1}\Sigma_0) = \operatorname{tr}(I_d) = d . \]
Third term. Write \(x - \mu_1 = (x - \mu_0) + (\mu_0 - \mu_1)\) and expand. The cross terms carry \(\mathbb{E}_{x \sim p}[x - \mu_0] = 0\) and vanish, so \[ \mathbb{E}_{x \sim p}\big[(x-\mu_1)^\top \Sigma_1^{-1} (x-\mu_1)\big] = \operatorname{tr}(\Sigma_1^{-1}\Sigma_0) + (\mu_0 - \mu_1)^\top \Sigma_1^{-1}(\mu_0 - \mu_1) . \]
Collecting the three pieces with their signs and the factor \(\frac{1}{2}\) gives Equation 21; the quadratic term is symmetric in the sign of \(\mu_1 - \mu_0\). \(\square\)
Scalar corollary. With \(d = 1\), Equation 21 reads \[ \mathrm{KL}\big(\mathcal{N}(\mu_0, \sigma_0^2) \,\big\Vert\, \mathcal{N}(\mu_1, \sigma_1^2)\big) = \log \frac{\sigma_1}{\sigma_0} + \frac{\sigma_0^2 + (\mu_0 - \mu_1)^2}{2\sigma_1^2} - \frac{1}{2} . \tag{22}\]
As a check, setting \(\mu_0 = \mu_1\) and \(\sigma_0 = \sigma_1\) in Equation 22 gives \(0 + \frac{1}{2} - \frac{1}{2} = 0\), as required by Gibbs’ inequality.
3.5 Two more names, in passing
Two other ways of comparing distributions will appear later in the course. Neither is developed here; you need to recognize the names.
Total variation. For densities \(p\) and \(q\), \[ \mathrm{TV}(p, q) \;=\; \frac{1}{2} \int |p(x) - q(x)|\,\mathrm{d}x . \] Equivalently, for the laws, \(\mathrm{TV}(P, Q) = \sup_A |P(A) - Q(A)|\) over measurable sets \(A\). It is a metric on probability distributions and satisfies \(0 \le \mathrm{TV}(P, Q) \le 1\).
Wasserstein distance. For distributions \(P\) and \(Q\) with finite second moments, \[ W_2(P, Q) \;=\; \Big( \inf_{\pi \in \Pi(P, Q)} \mathbb{E}_{(X, Y) \sim \pi}\big[ \|X - Y\|^2 \big] \Big)^{1/2} , \] where \(\Pi(P, Q)\) is the set of couplings having marginals \(P\) and \(Q\). Thus \(W_2^2\) is the minimum expected squared transport distance.
Unlike KL divergence, \(W_2\) uses the geometry of the underlying space. For example, two distributions with disjoint supports can have infinite KL divergence while their Wasserstein distance still reflects how far apart those supports are. \(W_2\) will reappear with optimal-transport couplings in U3.T4.
- UP.T1 §3 proves that maximum likelihood is minimization of \(\mathrm{KL}(q \Vert p_\theta)\), the direction discussed in Section 3.3.
- U0.T3 derives a lower bound whose slack is exactly a KL term. Students who took the bootcamp have met it; students joining at the main course have not, and do not need to have.
- U3.T4 uses \(W_2\) and optimal-transport couplings.
- Equation 21 itself is the closed form behind several of the comparisons you will make in the labs.
Exercises for §3
[PS1.1.6] Work with Equation 22.
(a) Derive Equation 22 from Equation 21, showing the specialization of each of the four terms.
(b) Fix \(\mu_0 = \mu_1 = 0\) and \(\sigma_0 = 1\), and plot or tabulate Equation 22 as a function of \(\sigma_1 \in (0, 4]\). Identify the minimum and verify it is where Gibbs’ inequality says it must be.
(c) Study the two limits \(\sigma_1 \to 0\) and \(\sigma_1 \to \infty\). Both make \(\mathrm{KL}\big(\mathcal{N}(0, 1) \,\Vert\, \mathcal{N}(0, \sigma_1^2)\big)\) diverge to \(+\infty\), but for different reasons. Explain the two limits in terms of how much density the approximating distribution assigns to regions where the first distribution has substantial probability.
(d) Now fix \(\sigma_0 = \sigma_1 = 1\) and let \(\mu_1 - \mu_0 = \delta\). How does the divergence grow with \(\delta\)?
[PS1.1.7] Let \(g = \mathcal{N}(0, 1)\) and let \(m = \frac{1}{2} \mathcal{N}(-3, 1) + \frac{1}{2} \mathcal{N}(3, 1)\).
(a) Compute numerically \(\mathrm{KL}(g \Vert m)\) and \(\mathrm{KL}(m \Vert g)\) on a grid fine enough that halving the spacing does not change the third decimal place. Report the grid and spacing you used.
(b) The two values are different. Explain which regions of the real line contribute most strongly to each divergence and relate this to the fact that KL is asymmetric.
(c) Now treat \(m\) as the target distribution and let \(r\) range over all one-dimensional Gaussian distributions. Describe the Gaussian that minimizes \(\mathrm{KL}(m \Vert r)\) and the qualitative form of a Gaussian minimizing \(\mathrm{KL}(r \Vert m)\). Relate the two solutions to the mass-covering and mode-seeking heuristics of Section 3.3, making clear that these descriptions concern the restricted single-Gaussian family.
(d) If \(m\) represents a data distribution and \(r_\theta\) is a Gaussian generative model trained by maximum likelihood, which of the two KL objectives above corresponds to maximum-likelihood fitting?
Slower treatment of §3, optional: (Murphy 2023) ch. 5 (§5.1 is KL); (Bishop and Bishop 2024) ch. 2.
§4 — Sampling, and change of variables in 1D
- You have a generator of \(\mathrm{Unif}(0,1)\) and a CDF \(F\) you can invert. How do you sample from \(F\)?
- \(X \sim \mathcal{N}(0,1)\) and \(Y = e^X\). What is the density of \(Y\)?
- In one sentence: what does “reparameterization” mean?
Answers in the footnote.4
4.1 Inverse-CDF sampling
Let \(F\) be any cumulative distribution function and define its generalized inverse, or quantile function, by \[ F^{-1}(u) \;=\; \inf\{\, x \in \mathbb{R} : F(x) \ge u \,\}, \qquad 0 < u < 1 . \] If \(U \sim \mathrm{Unif}(0,1)\), then \[ X \;=\; F^{-1}(U) \tag{23}\] has CDF \(F\).
Proof. For every \(x\), \[ F^{-1}(u) \le x \quad\Longleftrightarrow\quad u \le F(x) \] (the forward implication uses the right-continuity of \(F\)). Therefore \[ \mathbb{P}[X \le x] \;=\; \mathbb{P}\big[F^{-1}(U) \le x\big] \;=\; \mathbb{P}\big[U \le F(x)\big] \;=\; F(x) , \] because \(U\) is uniform on \((0,1)\). Hence \(X\) has CDF \(F\). \(\square\)
When \(F\) is continuous and strictly increasing on its effective support, this generalized inverse coincides with the usual inverse, and Equation 23 reduces to the familiar inverse-CDF sampling rule. The formulation above covers both continuous and discrete distributions.
The exponential distribution with rate \(\lambda\) has CDF \[ F(x) \;=\; \begin{cases} 0, & x < 0, \\[4pt] 1 - e^{-\lambda x}, & x \ge 0 . \end{cases} \] For \(0 < u < 1\), \(F^{-1}(u) = -\log(1-u)/\lambda\). Hence \[ X \;=\; -\frac{\log(1 - U)}{\lambda}, \qquad U \sim \mathrm{Unif}(0,1) , \] has the exponential distribution with rate \(\lambda\). Since \(1 - U\) is also uniformly distributed on \((0,1)\), one may equivalently sample \(X = -\log(U)/\lambda\).
Equation 23 expresses sampling as a deterministic transformation of a fixed source of randomness. In one dimension, the quantile function provides such a map directly.
In several dimensions there is no equally simple canonical scalar inverse-CDF construction. Multivariate constructions based on conditional CDFs do exist, but they are generally not the kind of tractable maps needed for high-dimensional generative modeling. The central problem therefore becomes how to construct or learn useful maps from a simple source distribution to a complicated target distribution.
4.2 Reparameterization
A related construction becomes especially important when a distribution depends on parameters. A reparameterization writes a random variable with a parameter-dependent law as a deterministic function of parameter-free noise: \[ x \;=\; T_\theta(\varepsilon), \qquad \varepsilon \sim p_\varepsilon , \] where the distribution \(p_\varepsilon\) does not depend on \(\theta\). The Gaussian example is the scalar case of Equation 9:
\[ x \;=\; \mu + \sigma \varepsilon , \qquad \varepsilon \sim \mathcal{N}(0,1) \qquad\Longrightarrow\qquad x \sim \mathcal{N}(\mu, \sigma^2) . \tag{24}\]
Informally: sampling is pushing a fixed noise source through a deterministic map. Equation 24 illustrates a recurring viewpoint in generative modeling: start from a fixed source distribution and transform its samples through a deterministic map. If the map contains learned parameters, changing those parameters changes the resulting distribution while the source distribution remains fixed. Normalizing flows and flow-based models will develop this idea using increasingly structured maps.
Reparameterization is also useful computationally. Because the distribution of \(\varepsilon\) does not depend on \(\mu\) or \(\sigma\), a sample can be viewed as the deterministic function \[ x(\mu, \sigma, \varepsilon) \;=\; \mu + \sigma \varepsilon \] for a fixed realization of \(\varepsilon\). Therefore \[ \frac{\partial x}{\partial \mu} \;=\; 1 , \qquad \frac{\partial x}{\partial \sigma} \;=\; \varepsilon . \] More generally, if \(x = T_\theta(\varepsilon)\) with parameter-independent noise \(\varepsilon\) and differentiable \(T_\theta\), gradients can be propagated through the sampled value by differentiating the deterministic map \(T_\theta\). This is the pathwise-gradient, or reparameterization, idea.
Reparameterization is not the only way to estimate gradients of expectations involving random variables; score-function estimators provide an alternative. Its advantage is that, when available, it usually gives a direct differentiable path through the sample and often lower-variance gradient estimates.
4.3 Change of variables in one dimension
Let \(X\) have density \(p_X\) on an interval \(I \subseteq \mathbb{R}\), and let \(g \colon I \to J\) be a continuously differentiable, strictly monotone bijection onto an interval \(J\), with \(g'(x) \ne 0\) on \(I\). Set \(Y = g(X)\). Then \(Y\) has density \[ p_Y(y) \;=\; p_X\big(g^{-1}(y)\big)\, \left| \frac{\mathrm{d}}{\mathrm{d}y} g^{-1}(y) \right| , \qquad y \in J , \tag{25}\] and \(p_Y(y) = 0\) outside \(J\). As usual for densities, Equation 25 is understood almost everywhere.
Derivation, via the CDF. For increasing \(g\), \(g(X) \le y\) holds exactly when \(X \le g^{-1}(y)\), so \[ F_Y(y) \;=\; \mathbb{P}[Y \le y] \;=\; \mathbb{P}\big[X \le g^{-1}(y)\big] \;=\; F_X\big(g^{-1}(y)\big) . \] Differentiating gives \[ p_Y(y) \;=\; p_X\big(g^{-1}(y)\big)\, \frac{\mathrm{d}}{\mathrm{d}y} g^{-1}(y) . \] For decreasing \(g\), write \[ F_Y(y) \;=\; \mathbb{P}\big[X \ge g^{-1}(y)\big] \;=\; 1 - F_X\big(g^{-1}(y)\big) , \] up to endpoint conventions that do not affect the density. Differentiating gives the negative of the inverse derivative, which is positive because \(g^{-1}\) is decreasing. Both cases are therefore summarized by Equation 25. \(\square\)
The factor \[ \left| \frac{\mathrm{d}}{\mathrm{d}y} g^{-1}(y) \right| \;=\; \frac{1}{\big| g'\big(g^{-1}(y)\big) \big|} \] accounts for local stretching or compression by the map. If \(g\) stretches a small interval around \(x\), its image occupies more length, so the same probability mass must be represented by a smaller density. If \(g\) compresses the interval, the density increases. The key principle is conservation of probability mass: \[ p_X(x)\,\mathrm{d}x \;\approx\; p_Y(y)\,\mathrm{d}y , \qquad y = g(x) . \] In the infinitesimal limit this relation gives Equation 25.
Figure 4 illustrates this local mass conservation for \(y = e^x\). A small interval in \(x\) is mapped to an interval in \(y\). Where the map stretches, the image interval is wider and the transformed density is correspondingly smaller; where it compresses, the opposite occurs. Integrating either density over the corresponding intervals gives the same probability mass.
The animation repeats this comparison at different locations along the map. The stretching factor varies with position, while the probability carried by corresponding intervals remains equal.
In several dimensions, the scalar stretching factor is replaced by the absolute determinant of a Jacobian matrix. U1 develops that result.
Let \(X \sim \mathcal{N}(0, 1)\) and \(Y = e^X\), so \(g^{-1}(y) = \log y\) for \(y > 0\) and \(\frac{\mathrm{d}}{\mathrm{d}y} g^{-1}(y) = 1/y\). Then Equation 25 gives \[ p_Y(y) \;=\; \mathcal{N}(\log y; 0, 1) \cdot \frac{1}{y} \;=\; \frac{1}{y\sqrt{2\pi}} \exp\!\left(-\frac{1}{2} (\log y)^2\right) , \qquad y > 0 , \] which is the standard log-normal density. The factor \(1/y\) is essential: omitting it would violate the change-of-variables formula and the resulting expression would not define the correct transformed density.
The behavior near \(y = 0\) is worth noting carefully. As \(y \to 0^+\), we have \(\log y \to -\infty\) and \(1/y \to +\infty\). The two factors therefore compete. The Gaussian term \(\exp\big(-\frac{1}{2} (\log y)^2\big)\) decays fast enough to dominate the growth of \(1/y\), and consequently \(p_Y(y) \to 0\) as \(y \to 0^+\). Similarly, \(p_Y(y) \to 0\) as \(y \to \infty\).
In \(d\) dimensions, the scalar factor \(\big|\frac{\mathrm{d}}{\mathrm{d}y} g^{-1}(y)\big|\) is replaced by \(|\det J_{g^{-1}}(y)|\). This is the multivariate change-of-variables theorem. U1 proves it and PS1.3 asks you to work through that proof. The one-dimensional derivation above provides intuition but is not a proof of the multivariate result.
4.4 Two directions, one map
An invertible generative map is used in two different directions.
Sampling. If \(x\) is drawn from the simple source distribution, a sample from the transformed distribution is obtained by evaluating \(y = g(x)\). Only the forward map is needed.
Density evaluation. Given a point \(y\), Equation 25 requires the corresponding source point \(x = g^{-1}(y)\) together with the local volume correction \(\big|\frac{\mathrm{d}}{\mathrm{d}y} g^{-1}(y)\big|\). Thus an explicit invertible model must make both the transformation and the required Jacobian information computationally tractable.
In one dimension, monotonicity makes inversion straightforward in principle. In high dimensions, however, an expressive map may be expensive to invert, and computing a determinant of a dense Jacobian generally has cubic cost in the dimension. U1 studies normalizing-flow architectures whose structure makes inversion and Jacobian determinants tractable, together with the expressive tradeoffs introduced by that structure. U2 then introduces continuous-time flows, which replace the discrete Jacobian determinant by a different density-evolution calculation and therefore change the computational tradeoff.
- U1.T1 proves the multivariate version of Equation 25, with the Jacobian determinant, and PS1.3 assigns the proof.
- U1.T2 develops the tradeoff between efficient sampling, inversion, and density evaluation in normalizing-flow architectures.
- The reparameterization Equation 24, and more generally reparameterized sampling from fixed noise, is used throughout the course whenever gradients must propagate through sampled quantities.
Exercise for §4
[PS1.1.8] Let \(X \sim \mathcal{N}(0, 1)\) and \(Y = \tanh X\).
(a) Derive the density \(p_Y\) of \(Y\) on \((-1, 1)\) using Equation 25, giving both \(g^{-1}\) and its derivative explicitly.
(b) Evaluate your density at \(y = 0\) and at \(y = 0.9\), and confirm numerically that it integrates to \(1\) over \((-1,1)\).
(c) Analyze the behavior of the density from part (a) as \(|y| \to 1\). Describe separately the behavior of \(\operatorname{artanh}(y)\), the Gaussian factor, and the inverse-Jacobian factor \(1/(1 - y^2)\), and determine which asymptotic behavior dominates in their product.
(d) In floating-point arithmetic, evaluating the density near \(|y| = 1\) is numerically delicate. Explain what happens to \(\operatorname{artanh}(y)\) and to \(1/(1 - y^2)\) as \(|y| \to 1\). Then write the log of the density from part (a) as a sum of two terms, and explain why evaluating the density in log-space is preferable to multiplying the two factors directly near the endpoints.
Slower treatment of §4, optional: (Murphy 2023) §2.5, “Transformations of random variables”. (Lipman et al. 2024) states the multivariate change of variables, with the Jacobian matrix, already in the notation of this course — that is where Section 4.3 is heading, and it is U1’s material rather than this section’s.
§5 — Notation drill
- In \(\mathbb{E}_{x \sim p}[f_\theta(x)]\), which symbols are integrated over and which are held fixed?
- Rewrite \(\mathbb{E}[\|f(y) - x\|^2]\) so that a reader cannot mistake what is random.
- In this course, is \(t = 0\) noise or data?
Answers in the footnote.5
This section introduces the expectation notation used throughout the course. The goal is to make explicit which quantities are random, how they are sampled, and which symbols are held fixed.
5.1 Reading an expectation with subscripts
Three examples, each one step more nested than the last.
One distribution. \[ \mathbb{E}_{x \sim q}\big[\, \|x\|^2 \,\big] . \] One variable is drawn, from the data distribution \(q\). The result is a number.
Two independent draws. \[ \mathbb{E}_{x_0 \sim p_0,\; x_1 \sim q}\big[\, \|x_1 - x_0\|^2 \,\big] . \] Here \(x_0\) is drawn from the source distribution \(p_0\) and \(x_1\) is drawn independently from the data distribution \(q\). The expectation is therefore taken under the product law \(p_0 \otimes q\). As a course convention, separate sampling statements in the same expectation subscript, with no joint or conditional law specified, denote independent draws. When variables must be sampled jointly, the joint draw will be written explicitly, for example \((x, y) \sim p_{XY}\). The result is again a number.
A chain: draw, then draw conditionally on what you drew. \[ \mathbb{E}_{t \sim \mathrm{Unif}[0,1],\; x_1 \sim q,\; x \sim p_t(\cdot \mid x_1)}\big[\, \ell(x, x_1, t) \,\big] . \tag{26}\] Read the sampling specification of Equation 26 from left to right. First, \(t \sim \mathrm{Unif}[0,1]\) and \(x_1 \sim q\) are sampled independently according to the course convention above. Conditional on the realized values of \(t\) and \(x_1\), draw \(x \sim p_t(\cdot \mid x_1)\). The notation \(p_t(\cdot \mid x_1)\) denotes a density in its argument, indexed by \(t\) and conditioned on \(x_1\).
Equation 26 is included only as a notation pattern. The family \(p_t\) is not defined in this primer; its construction and role belong to U3. At this stage, the point is only to practice reading nested sampling instructions correctly.
5.2 The rewriting game
Below are three expressions of the kind that appear in real papers. Each is ambiguous. Each is followed by a standard-form rewriting and a one-line diagnosis of what the ambiguity was.
Sloppy 1. \[ \mathcal{L}(\theta) = \mathbb{E}\big[\, \|f_\theta(y) - x\|^2 \,\big] . \]
Standard form. \[ \mathcal{L}(\theta) = \mathbb{E}_{(x, y) \sim p_{XY}}\big[\, \|f_\theta(y) - x\|^2 \,\big] . \]
Diagnosis. The bare \(\mathbb{E}\) does not specify whether \(X\) and \(Y\) are sampled jointly or independently. Under the intended joint law \(p_{XY}\), the optimal predictor under squared loss is the conditional mean. If \(X\) and \(Y\) were instead sampled independently from their marginals, the optimal predictor would reduce to the constant \(\mathbb{E}_{x \sim p_X}[x]\), assuming the required second moments exist.
Sloppy 2. \[ \mathcal{L}(\theta) = \mathbb{E}_{x, t}\big[\, \|u_t^\theta(x) - u_t(x)\|^2 \,\big] . \]
Standard form. \[ \mathcal{L}(\theta) = \mathbb{E}_{t \sim \mathrm{Unif}[0,1],\; x \sim p_t}\big[\, \|u_t^\theta(x) - u_t(x)\|^2 \,\big] . \]
Diagnosis. The notation \(\mathbb{E}_{x, t}\) names variables but does not specify their laws. The standard form makes two pieces of information explicit: the sampling law of time and the fact that the distribution of \(x\) depends on the sampled value of \(t\).
Sloppy 3. \[ \mathcal{L}(\theta) = \mathbb{E}_{x_1}\big[\, \|u_t^\theta(x) - u_t(x \mid x_1)\|^2 \,\big] . \]
Standard form. \[ \begin{aligned} \mathcal{L}(\theta) = \mathbb{E}_{t \sim \mathrm{Unif}[0,1],\; x_1 \sim q,\; x \sim p_t(\cdot \mid x_1)} \big[\, \|u_t^\theta(x) - u_t(x \mid x_1)\|^2 \,\big] . \end{aligned} \]
Diagnosis. In the original expression, \(t\) and \(x\) appear inside the integrand but are not bound by the expectation. The displayed quantity therefore still depends on those free variables and is not, as written, a scalar objective depending only on \(\theta\). The repair is the rule below.
Every symbol appearing inside an expectation is either
- a bound random variable whose sampling law is given in the subscript, or
- a parameter or other fixed quantity on which the expectation is allowed to depend.
Before writing an expectation, identify which role each symbol plays.
5.3 The four rules
These follow the course notation standard, and they are binding on everything you write in this course, including problem sets.
- Write explicit sampling subscripts whenever an expectation could otherwise be ambiguous. Every loss must use them. The subscript must identify both the random variables and their sampling laws, including the law of time whenever \(t\) is random. The course default is \(t \sim \mathrm{Unif}[0,1]\), but the law is still written explicitly.
- Never use the diffusion time arrow outside the Rosetta Stone.
- When restating a result from a source that uses different notation, translate it to the course standard. For example, a source may use \(v_t\) for a velocity field, \(x_T\) for noise, or \(\rho\) for a density. When a notation difference is itself pedagogically relevant, state the mapping explicitly. A direct quotation should preserve the source’s original notation.
- When the projection Lemma from UP.T1 is invoked, cite it by name.
Rules 2 and 4 refer to material introduced later in the course. Rule 1 applies immediately and will be enforced in PS1. In particular, writing a random variable such as \(t\) in an expectation subscript without specifying its law does not satisfy the convention.
These conventions are used from UP.T1 onward. Rule 2 becomes relevant when diffusion notation is introduced in U3, and Rule 4 applies once the projection Lemma has been proved in UP.T1.
Exercise for §5
[PS1.1.9] Rewrite each expression below into the standard form of this course. For each, list every symbol inside the brackets and classify it as bound (naming the distribution you chose, and justifying the choice in a few words) or fixed.
(a) Consider the maximum-likelihood objective \[ J(\theta) \;=\; \mathbb{E}\big[\log p_\theta(x)\big] , \] where the dataset consists of samples from the data distribution \(q\). Rewrite the objective in the course standard, and classify each symbol appearing inside the expectation as either bound or fixed.
(b) For a fixed observed example \(x\), consider \[ \mathcal{L}(\phi, \theta; x) \;=\; \mathbb{E}_{z}\big[\log p_\theta(x \mid z)\big] \;-\; \mathrm{KL}\big(q_\phi(z \mid x) \,\Vert\, p(z)\big) . \] Rewrite the expectation in the standard notation of the course. Classify every symbol appearing inside the expectation as bound or fixed.
(c) After rewriting part (b), is \(\mathcal{L}(\phi, \theta; x)\) a scalar independent of \(x\), or a function of the observed example \(x\)? Then write an outer expectation that turns it into a dataset-level objective.
Appendix A — The course notation standard
This table is the canonical symbol table for every artifact of this course. It follows the course notation standard (L1 §4), and it is the first place to look whenever a symbol is unfamiliar.
The table is a reference for the whole course, not a list of concepts already introduced in this primer. Symbols such as \(p_t\), \(\psi_t\), \(u_t\), \(u_t^\theta\), and \(s_t\) are defined in later units. They are included here so that the notation used across the course can be consulted in one place.
Time convention. \(t \in [0,1]\), with \(t = 0\) corresponding to noise and \(t = 1\) corresponding to data. Diffusion sources that use the opposite time direction are translated to this convention. The Rosetta Stone records the corresponding mappings, including \(s = 1 - t\) when appropriate.
| Object | Symbol | Notes |
|---|---|---|
| Data distribution | \(q\), samples \(x_1 \sim q\) | on \(\mathbb{R}^d\) |
| Encoder / variational distributions | \(q_\phi(z \mid x)\) | subscript mandatory. Outside local derivations in which symbols such as \(p\) and \(q\) are explicitly introduced as generic probability distributions, the reserved course symbol \(q\) denotes the data distribution (ruling from U0.T3) |
| Source / noise | \(p_0 = \mathcal{N}(0, I_d)\), samples \(x_0\) | |
| Probability path | \(p_t\) | marginal probability path with endpoints \(p_{t=0} = p_0\), \(p_{t=1} = q\) |
| Conditional path | \(p_t(x \mid x_1)\) | designer’s choice |
| Flow map | \(\psi_t\) | \(\psi_0 = \mathrm{id}\) |
| Marginal velocity | \(u_t(x)\) | generates \(p_t\) via the continuity equation |
| Conditional velocity | \(u_t(x \mid x_1)\) | generates \(p_t(\cdot \mid x_1)\) |
| Learned field | \(u_t^\theta(x)\) | network; never \(v_\theta\) in our materials |
| Score | \(s_t(x) = \nabla_x \log p_t(x)\) | |
| Gaussian path schedule | \(\alpha_t, \sigma_t\) | \(x_t = \alpha_t x_1 + \sigma_t x_0\) with \(x_0 \sim \mathcal{N}(0, I_d)\), \(x_1 \sim q\), \(x_0 \perp x_1\); for a path from noise at \(t = 0\) to data at \(t = 1\) the endpoints are \(\alpha_0 = 0\), \(\sigma_0 = 1\) and \(\alpha_1 = 1\), \(\sigma_1 = 0\) |
| Noise parametrization | \(\varepsilon\)-prediction \(\varepsilon_\theta\) | appears in D2 and the Rosetta Stone |
| Diffusion \(v\)-target | \(v\) (Rosetta-qualified) | diffusion prediction target; never the learned transport field \(u_t^\theta\) |
| Jacobian | \(\partial f / \partial x\) or \(J_f(x)\) | matrix of partials |
| Divergence | \(\nabla \cdot f = \operatorname{tr}(\partial f/\partial x)\) | both notations declared equal once, then \(\nabla \cdot\) |
| KL divergence | \(\mathrm{KL}(p \,\Vert\, q)\) | double bar |
| Gaussian law | \(X \sim \mathcal{N}(\mu, \Sigma)\) | may be degenerate (\(\Sigma \succeq 0\)) |
| Gaussian density | \(\mathcal{N}(x; \mu, \Sigma)\) | exists only when \(\Sigma \succ 0\) |
| Expectations | \(\mathbb{E}\) with explicit sampling subscripts | whenever the sampling law would otherwise be ambiguous; every loss states all random variables and their laws explicitly, for example \(\mathbb{E}_{t \sim \mathrm{Unif}[0,1],\, x_1 \sim q,\, x \sim p_t(\cdot \mid x_1)}\). Separate unconditional sampling statements denote independent draws unless a joint or conditional law is written explicitly |
| Losses | \(\mathcal{L}_{\mathrm{FM}}(\theta)\), \(\mathcal{L}_{\mathrm{CFM}}(\theta)\), \(\mathcal{L}_{\mathrm{DSM}}(\theta)\) | |
| Dimension | \(d\) | |
| ODE / SDE | \(\mathrm{d}x_t = u_t(x_t)\,\mathrm{d}t\); \(\mathrm{d}x_t = f_t(x_t)\,\mathrm{d}t + g_t\,\mathrm{d}W_t\) |
\(W_t\) standard Wiener process |
What to do next
Before UP.T1, complete the following.
- Complete the nine PS1.1 exercises. They are graded.
- Keep the Gaussian Algebra Toolkit available for later units. The two-page extract of §2 is the sheet you will reach for during U3.
- Keep Equation 18 available for reference. It will reappear in later derivations.
- Review the four notation rules in Section 5.3. Rule 1 will be enforced on every problem set.
- Come to UP.T1 comfortable with the intuition behind Equation 5. That session gives its formal statement and proof and then develops the projection Lemma.
References
Footnotes
- \(p(y) = \int p(x, y)\,\mathrm{d}x\) and \(p(x \mid y) = p(x,y)/p(y)\) wherever \(p(y) > 0\). (2) Independent, yes; independent given \(Z\), no — knowing \(Z\) makes \(X\) determine \(Y\). This is the reverse of the example in Section 1.2, and both directions happen. (3) \(\mathbb{E}_{x \sim p_X}[x] = 1.5\), by averaging the two conditional means.
- \(\mathcal{N}(A\mu + b,\; A \Sigma A^\top)\). (2) Yes, and no — the conditional covariance does not depend on the value of \(x_b\), which is a special property of Gaussians and the reason they are so convenient. (3) \((x - A^{-1}b)^\top A (x - A^{-1}b) - b^\top A^{-1} b\).
- \(\mathrm{KL}(p \Vert q) = \mathbb{E}_{x \sim p}[\log p(x) - \log q(x)]\).
- No, and no — it is asymmetric, and it is non-negative, which Section 3.2 proves. (3) For \(q\) a single Gaussian: minimizing \(\mathrm{KL}(q \Vert p)\) over \(q\) picks one hump; minimizing \(\mathrm{KL}(p \Vert q)\) spreads across both. A heuristic for that family, not a theorem; see Figure 3.
- \(X = F^{-1}(U)\) with \(U \sim \mathrm{Unif}(0,1)\); see Equation 23.
- \(p_Y(y) = \mathcal{N}(\log y; 0, 1)/y\) for \(y > 0\), the log-normal; see Equation 25. (3) Writing a random draw as a deterministic function of a fixed, simple noise source.
- In \(\mathbb{E}_{x \sim p}[f_\theta(x)]\), \(x\) is bound by the expectation and sampled from \(p\); \(\theta\) is held fixed, so the resulting quantity is generally a function of \(\theta\). (2) Any unambiguous version is acceptable, for example \(\mathbb{E}_{(x,y) \sim p_{XY}}[\|f(y) - x\|^2]\) with an explicitly named joint law — what is not acceptable is leaving it as given. (3) Under the course convention, \(t = 0\) corresponds to noise and \(t = 1\) to data.