How LLMs Get Faster Without Changing Their Outputs | Speculative Decoding

Speculative decoding is a clever trick for speeding up LLM inference. It uses a fast draft model to guess ahead, while the large model verifies many tokens in parallel, preserving the same output distribution. In this video, we explain the core trick behind speculative decoding, why rejection sampling keeps the answers unchanged, and how newer methods like draft trees, Medusa, MTP, EAGLE, DFlash, and DFlash 2 push LLM inference even faster. Chapters: 0:00 Autoregressive decoding 0:51 Speculative decoding: Draft and verify 2:08 Rejection sampling 7:08 Draft trees 9:43 Medusa 12:44 MTP: Mult