vLLM: serving Kimi K3 on 8x AMD Instinct MI355X beats NVIDIA HGX B300 by 24.4% throughput per GPU and 87.4% more tokens per dollar on the AgentX benchmark

Why it matters

【Key points】

On Oct 16 the vLLM community (with AMD and Embedded LLM) reported that, on the public AgentX coding-agent benchmark, serving Kimi K3 with vLLM on an 8-GPU AMD Instinct MI355X system reaches about 7,567 total tokens/s/GPU at the 50 tokens/s/user P90 interactivity point, vs about 6,083 for NVIDIA HGX B300 — a 24.4% throughput edge. Using SemiAnalysis's modeled TCO ($1.50 vs $2.26 per GPU-hour), that is 87.4% more tokens per dollar (18.16M vs 9.69M).

Both GPUs have 288GB HBM3E and 8 TB/s bandwidth; published dense peaks are 10.1 PFLOP/s MXFP4 (MI355X) vs 13.5 PFLOP/s FP4 (B300) — i.e. higher measured throughput despite lower peak FLOPs. The post attributes gains to cache reuse (DCP + CPU offload), MoE dedup across GPUs, fused ops to cut copies, AMD AITER assembly for MLA long-context decode, KDA parallelizing prefill and speculative verification, and tunable speculative-decoding draft length.

【Verifiable】

vLLM post: https://blog.vllm.ai/blog/2026-10-16-kimi-k3-mi355x

Data source: SemiAnalysis InferenceX (Oct 9 snapshot, per the post)

【Our read】

1. Another case of software squeezing frontier-class serving out of nominally weaker silicon: with lower peak FP4, MI355X overtakes B300 on a realistic coding-agent replay (>96% prefix reuse), which bears directly on inference price/performance choices.

2. Mind the framing: the comparison is against SemiAnalysis's modeled TCO at a P90 interactivity point, not an absolute ceiling, and per-optimization micro-tests use separate configs; vendor-reported numbers warrant independent replication.

3. It continues vLLM's track of turning Chinese open models (MiniMax M3, DeepSeek, now Moonshot's Kimi K3) into first-class inference showcases (see e/793, e/729) — useful for open-weight deployers.