Skip to content
RL, from scratch
to LLMs

Part I Foundations

  1. 01What is RL?
  2. 02Bandits
  3. 03MDPs
  4. 04Dynamic programming
  5. 05MC & TD learning

Part II Deep RL

  1. 06DQN
  2. 07Policy gradients
  3. 08Actor-critic & GAE
  4. 09PPO
  5. 10The wider landscape

Part III RL for LLMs

  1. 11LLM as a policy
  2. 12RLHF
  3. 13DPO
  4. 14Reward sources
  5. 15RLVR & GRPO
  6. 16Algorithm zoo
  7. 17Process rewards
  8. 18Agentic RL
  9. 19Failure modes
  10. 20RL systems
  11. 21The frontier
  1. ▶Narrated lessons
  2. ∑Notation
  3. ✎Exercises & setup
  4. ¶Paper map
Search

Narrated lessons

Sit back and listen. Each lesson is a slide deck with a spoken voiceover and captions, covering several chapters in one go. Every slide links back to its chapter for the demos, exercises and full derivations.

  • ▶ RL foundations, narratedA narrated slide lesson through Chapters 1–9: from a gridworld to PPO in about 30 minutes.29 slides · ~31 min

The narration is synthesized speech, generated from the transcript under each lesson.