https://learning-notes-dz2.pages.dev/categories/series/ Recent content in Series on My Learning Notes Hugo -- 0.124.0 en Tue, 16 Jun 2026 07:19:01 +0000 https://learning-notes-dz2.pages.dev/posts/2026-03-30/ Mon, 30 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-30/ DPO has problems — preference reversals, reward degradation, and a safety-helpfulness trade-off. Here’s how SafeDPO, RePO, and other recent variants are fixing them. https://learning-notes-dz2.pages.dev/posts/2026-03-22/ Sun, 22 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-22/ A unifying view of RLHF, DPO, and Constitutional AI — they’re all estimating the divergence between safe and unsafe output distributions. Plus a clean derivation of why DPO works. https://learning-notes-dz2.pages.dev/posts/2026-03-18/ Wed, 18 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-18/ How trust regions stabilize policy optimization, why PPO became the default for RLHF, and how GRPO eliminates the critic entirely. https://learning-notes-dz2.pages.dev/posts/2026-03-05/ Thu, 05 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-05/ MDPs, value functions, the REINFORCE algorithm, actor-critic methods, and generalized advantage estimation — the RL foundations you need before understanding RLHF. https://learning-notes-dz2.pages.dev/posts/2026-02-25/ Wed, 25 Feb 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-02-25/ A survey of where the VAE idea went after 2014 — VQ-VAE, hierarchical VAEs, adversarial hybrids, flow-based posteriors — and what the VAE really gave us beyond a specific architecture. https://learning-notes-dz2.pages.dev/posts/2026-02-20/ Fri, 20 Feb 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-02-20/ Sparse attention patterns, head specialization, rotary embeddings, gated attention, and the modern efficiency tricks that make large transformers actually trainable. https://learning-notes-dz2.pages.dev/posts/2026-02-10/ Tue, 10 Feb 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-02-10/ A single Greek letter in front of the KL term changes what the VAE learns. We look at β-VAE as a rate-distortion trade-off, an information bottleneck, and a simple probe into disentangled representations. https://learning-notes-dz2.pages.dev/posts/2026-02-08/ Sun, 08 Feb 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-02-08/ A first-principles walkthrough of the Transformer — self-attention, positional encoding, multi-head attention — with the math that makes it work. https://learning-notes-dz2.pages.dev/posts/2026-01-25/ Sun, 25 Jan 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-01-25/ We extend the VAE into a controllable generative model by adding a condition y into every term of the ELBO. https://learning-notes-dz2.pages.dev/posts/2026-01-18/ Sun, 18 Jan 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-01-18/ Mechanistic interpretability meets alignment — how researchers found that a tiny fraction of neurons are responsible for almost all safety behavior in LLMs, and what that means. https://learning-notes-dz2.pages.dev/posts/2026-01-10/ Sat, 10 Jan 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-01-10/ We open the ELBO, compute each term, and meet the reparameterization trick — the idea that lets us backpropagate through randomness. https://learning-notes-dz2.pages.dev/posts/2025-12-20/ Sat, 20 Dec 2025 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2025-12-20/ Variational Inference transforms the impossible task of computing intractable integrals into a solvable optimization problem, providing the mathematical foundation for modern generative models like VAEs. https://learning-notes-dz2.pages.dev/posts/2025-10-28/ Tue, 28 Oct 2025 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2025-10-28/ From PCA to Probabilistic PCA and general Latent Variable Models: the probabilistic lens that seeds VAEs. https://learning-notes-dz2.pages.dev/posts/2024-12-24/ Tue, 24 Dec 2024 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2024-12-24/ A summary of explicit, implicit and score-based generative models. https://learning-notes-dz2.pages.dev/posts/2024-10-05/ Sat, 05 Oct 2024 11:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2024-10-05/ Information theory essentials: entropy, cross-entropy, joint/conditional entropy, KL divergence, mutual information. https://learning-notes-dz2.pages.dev/posts/2024-06-11/ Tue, 11 Jun 2024 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2024-06-11/ Diffusion Models (DMs) include two processes: forward and backward. Forward process General idea Degrading input data using noise iteratively, forward in time (i.e., $t$ increases). Given image $x_0 \sim q(x_0)$, which called data distribution, forward process gradually adds Gauss noise thru $T$ time steps and produces latent $x_T$. At each time step $t$, we sample Gauss noise that following the distribution $\mathcal{N}(\sqrt{1 - \beta_t} x_{t-1}, \beta_t)$, where the hyper-parameters $0 < \beta_{1:T} < 1$ represent the variance of noise incorporated at each time step. https://learning-notes-dz2.pages.dev/posts/2021-08-21/ Sat, 21 Aug 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-08-21/ Determinants, eigenvalues, eigenvectors: geometric meaning, finding methods, and linear transformation essence. https://learning-notes-dz2.pages.dev/posts/2021-08-07/ Sat, 07 Aug 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-08-07/ Linear independence, span, basis, dimension: fundamental concepts for vector spaces and subspaces. https://learning-notes-dz2.pages.dev/posts/2021-08-14/ Sat, 07 Aug 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-08-14/ "This is really the heart of this approach to linear algebra, to see these four subspaces, how they are related." - Prof. Gilbert Strang https://learning-notes-dz2.pages.dev/posts/2021-08-01/ Sun, 01 Aug 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-08-01/ Solving Ax=b: conditions for solutions, complete solution (particular + nullspace), rank relationships. https://learning-notes-dz2.pages.dev/posts/2021-07-27/ Tue, 27 Jul 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-07-27/ Nullspace and solving Ax=0: special solutions, free variables, reduced row echelon form. https://learning-notes-dz2.pages.dev/posts/2021-07-25/ Sun, 25 Jul 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-07-25/ Echelon form and matrix rank: row elimination, leading elements, and solving linear systems. https://learning-notes-dz2.pages.dev/posts/2021-07-23/ Fri, 23 Jul 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-07-23/ Vector spaces, subspaces, column space: 8 axioms, subspace properties, and linear combinations. https://learning-notes-dz2.pages.dev/posts/2021-07-21/ Wed, 21 Jul 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-07-21/ What is vanishing gradients? How come it happens and what is the solution to resolve? https://learning-notes-dz2.pages.dev/posts/2021-07-20/ Tue, 20 Jul 2021 01:12:07 +0700 https://learning-notes-dz2.pages.dev/posts/2021-07-20/ Basic concepts of Linear Algebra: data types, notations, and so on