Reinforcement-Learning on My Learning Notes

Reinforcement-Learning on My Learning Notes https://learning-notes-dz2.pages.dev/tags/reinforcement-learning/ Recent content in Reinforcement-Learning on My Learning Notes Hugo -- 0.124.0 en Tue, 16 Jun 2026 07:19:01 +0000 Paper Roundup: LLM Safety & RLHF at NeurIPS 2025 and ICLR 2026 https://learning-notes-dz2.pages.dev/posts/2026-04-29/ Wed, 29 Apr 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-04-29/ A curated list of papers on alignment, preference optimization, mechanistic interpretability, and reasoning from the two biggest ML conferences this cycle — with personal takes on the ones that matter most. SafeDPO and Friends: Preference Optimization That Doesn't Sacrifice Safety https://learning-notes-dz2.pages.dev/posts/2026-03-30/ Mon, 30 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-30/ DPO has problems — preference reversals, reward degradation, and a safety-helpfulness trade-off. Here’s how SafeDPO, RePO, and other recent variants are fixing them. Does RL Actually Make LLMs Reason Better? https://learning-notes-dz2.pages.dev/posts/2026-03-28/ Sat, 28 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-28/ The evidence is more complicated than the hype suggests. RL improves sampling efficiency but may not expand reasoning capacity — and longer chains of thought don’t always help. RLHF Is Just Divergence Estimation in Disguise https://learning-notes-dz2.pages.dev/posts/2026-03-22/ Sun, 22 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-22/ A unifying view of RLHF, DPO, and Constitutional AI — they’re all estimating the divergence between safe and unsafe output distributions. Plus a clean derivation of why DPO works. From Policy Gradient to PPO — Part 2: Trust Regions, PPO, and GRPO https://learning-notes-dz2.pages.dev/posts/2026-03-18/ Wed, 18 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-18/ How trust regions stabilize policy optimization, why PPO became the default for RLHF, and how GRPO eliminates the critic entirely. From Policy Gradient to PPO — Part 1: Foundations https://learning-notes-dz2.pages.dev/posts/2026-03-05/ Thu, 05 Mar 2026 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2026-03-05/ MDPs, value functions, the REINFORCE algorithm, actor-critic methods, and generalized advantage estimation — the RL foundations you need before understanding RLHF. A Curated Guide to LLMs, Reinforcement Learning, and AI Safety https://learning-notes-dz2.pages.dev/posts/2025-12-28/ Sun, 28 Dec 2025 00:00:00 +0700 https://learning-notes-dz2.pages.dev/posts/2025-12-28/ Books, papers, conferences, and researchers — a personal resource list for anyone going deep into LLMs, RL, and AI safety.