DeepSeek-V4.1-Flash on vLLM: 5x agentic throughput three weeks after launch

Why it matters

【Key points】

On October 7, the vLLM team and Inferact reported that three weeks after DeepSeek-V4.1-Flash launched, the model runs 1.9× faster on vLLM at low concurrency and delivers 5.3× the day-0 throughput at 150 tokens per second per user, measured on SemiAnalysis's AgentX agentic serving benchmark.

The gains come from three places. First, "SWA bounded replay": vLLM recomputes only the last 128 tokens of the sliding window instead of caching it, and layers 21–39 handle only each request's last 128 tokens; with CUDA graphs, prefill compute drops 30–40%. Second, vLLM integrated the kernels DeepSeek open-sourced alongside the model, including MegaAttention, Mega-mHC, Mega-Gate and DeepSelect. Third, more kernel fusion and parallelism.

【Verifiable】

Primary sources are the vLLM blog post and the @vllm_project announcement on X (about 23K views, reposted by SemiAnalysis).

All figures are vLLM's own measurements against its own day-0 implementation, not against other inference engines, on NVIDIA Blackwell GPUs such as GB200 and GB300. Bounded replay is not bit-exact; vLLM's checks on GSM8K and GPQA found accuracy gaps within about 1.5 standard errors. No independent reproduction has appeared yet.

【Our read】

An open-weight model's launch-day speed and cost are not its final price. DeepSeek shipped its kernels with the weights, and the serving community turned them into 5× throughput in three weeks; vLLM did 2.8× for Kimi K3 last month. Once weights are out, cost reduction becomes a shared effort.

It answers this week's worry over Haiku 5.5's price cut. Closed vendors lower list prices; open models lower the actual compute per token, and anyone who deploys them gets the benefit. Teams self-hosting should judge open models by whether the maker releases kernels and how fast the community follows, not just launch benchmarks.

Caveat: the gains were measured on new NVIDIA hardware and may not carry over to older or Chinese accelerators.