EAGLE-4 speculative decoding goes public: one draft model that adapts to batch size and hardware, up to 3.6x faster on Qwen3.8-27B in the team's tests

Why it matters

【Key points】

Eagle AI Corp, the team behind the widely used EAGLE speculative-decoding methods, published EAGLE-4 v0.1.0 on Oct 20: inference code, a paper and a Qwen3.8-27B draft model, as plugins for SGLang, vLLM and experimental TokenSpeed.

1. Idea: one draft model measures drafting and verification costs for the current batch size, hardware and workload, and picks how many tokens to guess per step, keeping the target model's output distribution.

2. Paper (team's tests, Qwen3.8-27B, 4x B300, T=0): at one request, 3.59x vanilla decoding, 1.28x DFlash 2, 1.40x DSpark, 1.62x the built-in MTP head; at 512 concurrent requests, 1.70x DFlash 2 and 1.99x DSpark. 2.71x on a 4-vCPU CPU.

3. Release notes: vLLM at concurrency 1 goes from 241.6 to 722.8 tok/s (2.99x); SGLang 1.91x at 8, 1.35x at 64; AMD MI355X 2.07x at 8, ROCm wheels still being verified.

4. Conditions: needs the team's patched engine builds. The license is source-available, not open source: organizations with over $20M revenue in 12 months need a commercial license; competing offerings are barred.

【Verifiable】

Release: https://github.com/eagle-hq/EAGLE-4/releases/tag/v0.1.0

Paper: https://github.com/eagle-hq/EAGLE-4/blob/main/EAGLE4_Paper.pdf

Draft model: https://huggingface.co/EAGLE-Corp/Qwen3.8-27B-EAGLE4

License: https://github.com/eagle-hq/EAGLE-4/blob/main/LICENSE

【Our read】

1. Speculative decoding is a main lever for cutting serving cost; adapting to batch size and hardware targets real swings in production load.

2. All comparisons are the team's own and cover one target model; gains shrink at high concurrency, and new models need their own drafts.

3. Patched engines and a revenue-gated license make large-company production use harder than with earlier EAGLE versions.