eagle-hq/EAGLE-4

EAGLE-4 One Flexible Drafter for All Batch Sizes and Hardware Paper (EAGLE-4) · EAGLE-4 draft for Qwen3.8-27B · Documentation Leave speculation on. EAGLE-4 adapts to your hardware, batch and workload. For high load, serve SGLang on four B300 GPUs with the high-load recipe, which keeps pace with plain decoding up to 512 concurrent requests. EAGLE-4 (Extrapolation Algorithm for Greater Language-model Efficiency) speeds up LLM inference with speculative decoding. EAGLE-4 draft models are built to run with eagle-harness, which decides how far to speculate from measured costs on your setup, so every draft tree fits the batch size, the hardware and the workload. The output stays exactly what the target model would generate on its own: up to 3.4× faster generation on Qwen3.8-27B, with the same answers. Measured speedups This repository holds eagle-harness, the package that brings EAGLE-4 to SGLang, vLLM and TokenSpeed. Install it, point your engine at an EAGLE-4…