DeepSeek Splits the Transformer in Two. Here’s Why.

DeepSeek-V4.1-Flash cuts the global KV cache to 890 bytes per token, 3.9× smaller than V4-Flash, and shrinks the total persistent cache to about one-eighth, all while staying competitive on agentic benchmarks! How did they do that? Let's explore the architecture from first principles. We start with a standard decoder-only transformer and then discuss the key techniques of Causal Encoder-Decoder, Compressed Sparse Attention, and Sliding-Window Bounded Replay. Chapters 0:00 DeepSeek V4.1-Flash 0:14 Decoder-only transformer 2:16 Causal encoder-decoder (CED) 4:00 Compressed Sparse Attention (C