DeepSeek Just Open-Sourced How It Serves 155T Tokens a Day

DeepSeek just open-sourced DeepInfer, the inference engine behind its own API, along with a detailed write-up of how it serves DeepSeek V4.1 Flash at scale: 155 trillion tokens a day from a single cluster. In this video we follow one request through the system: deepseek-recipe tokenization and prompt-injection protection, a KV cache that DeepSeek keeps for days (96% of input tokens are cache hits), prefill and decode on different machines, expert parallelism across 64+ GPUs, DSpark speculative decoding, and batch-invariant inference. Then I test deepseek-recipe and the temperature-0 batching e