Series

We open the ELBO, compute each term, and meet the reparameterization trick — the idea that lets us backpropagate through randomness.

Variational Inference transforms the impossible task of computing intractable integrals into a solvable optimization problem, providing the mathematical foundation for modern generative models like VAEs.

From PCA to Probabilistic PCA and general Latent Variable Models: the probabilistic lens that seeds VAEs.

A summary of explicit, implicit and score-based generative models.

Information theory essentials: entropy, cross-entropy, joint/conditional entropy, KL divergence, mutual information.

Diffusion Models (DMs) include two processes: forward and backward. Forward process General idea Degrading input data using noise iteratively, forward in time (i.e., $t$ increases). Given image $x_0 \sim q(x_0)$, which called data distribution, forward process gradually adds Gauss noise thru $T$ time steps and produces latent $x_T$. At each time step $t$, we sample Gauss noise that following the distribution $\mathcal{N}(\sqrt{1 - \beta_t} x_{t-1}, \beta_t)$, where the hyper-parameters $0 < \beta_{1:T} < 1$ represent the variance of noise incorporated at each time step....

Determinants, eigenvalues, eigenvectors: geometric meaning, finding methods, and linear transformation essence.

Linear independence, span, basis, dimension: fundamental concepts for vector spaces and subspaces.

Four fundamental subspaces: row space, column space, nullspace, left nullspace with dimensions and relationships.

Solving Ax=b: conditions for solutions, complete solution (particular + nullspace), rank relationships.