Generative-Models

A survey of where the VAE idea went after 2014 — VQ-VAE, hierarchical VAEs, adversarial hybrids, flow-based posteriors — and what the VAE really gave us beyond a specific architecture.

A single Greek letter in front of the KL term changes what the VAE learns. We look at β-VAE as a rate-distortion trade-off, an information bottleneck, and a simple probe into disentangled representations.

We extend the VAE into a controllable generative model by adding a condition y into every term of the ELBO.

We open the ELBO, compute each term, and meet the reparameterization trick — the idea that lets us backpropagate through randomness.

Variational Inference transforms the impossible task of computing intractable integrals into a solvable optimization problem, providing the mathematical foundation for modern generative models like VAEs.

From PCA to Probabilistic PCA and general Latent Variable Models: the probabilistic lens that seeds VAEs.

A summary of explicit, implicit and score-based generative models.

Diffusion Models (DMs) include two processes: forward and backward. Forward process General idea Degrading input data using noise iteratively, forward in time (i.e., $t$ increases). Given image $x_0 \sim q(x_0)$, which called data distribution, forward process gradually adds Gauss noise thru $T$ time steps and produces latent $x_T$. At each time step $t$, we sample Gauss noise that following the distribution $\mathcal{N}(\sqrt{1 - \beta_t} x_{t-1}, \beta_t)$, where the hyper-parameters $0 < \beta_{1:T} < 1$ represent the variance of noise incorporated at each time step....