Posts

Sparse attention patterns, head specialization, rotary embeddings, gated attention, and the modern efficiency tricks that make large transformers actually trainable.

A single Greek letter in front of the KL term changes what the VAE learns. We look at β-VAE as a rate-distortion trade-off, an information bottleneck, and a simple probe into disentangled representations.

A first-principles walkthrough of the Transformer — self-attention, positional encoding, multi-head attention — with the math that makes it work.

We extend the VAE into a controllable generative model by adding a condition y into every term of the ELBO.

Mechanistic interpretability meets alignment — how researchers found that a tiny fraction of neurons are responsible for almost all safety behavior in LLMs, and what that means.

We open the ELBO, compute each term, and meet the reparameterization trick — the idea that lets us backpropagate through randomness.

Books, papers, conferences, and researchers — a personal resource list for anyone going deep into LLMs, RL, and AI safety.

Variational Inference transforms the impossible task of computing intractable integrals into a solvable optimization problem, providing the mathematical foundation for modern generative models like VAEs.

From PCA to Probabilistic PCA and general Latent Variable Models: the probabilistic lens that seeds VAEs.

A summary of explicit, implicit and score-based generative models.