How Small Models Learn to Think Like Giants | On-Policy Distillation

How do we train a small yet powerful AI model? This video explains knowledge distillation from first principles. We start with token-level and sequence-level distillation, discuss exposure bias, and why reinforcement learning helps but gives sparse feedback. Then we get to the main idea: On-Policy Distillation, in which a student learns from its own trajectories while still receiving dense teacher guidance. 00:00 Why Distillation Matters 01:28 Soft Targets and Dark Knowledge 04:11 From One Token to Many Tokens 09:30 Exposure Bias 12:11 On-Policy Reinforcement Learning 14:08 On-Policy Distill