DFlash Speculative Decoding on AMD Instinct MI355X: Up to 5× Faster Qwen3.5 Inference
Autoregressive decode generates one token per model forward pass, so single-request latency is bounded by how fast you can stream weights through the GPU — not by compute. Speculative decoding attacks that bottleneck by letting a small draft model propose several tokens that the large target model then verifies in a single parallel pass. We bring DFlash — a block-diffusion drafter — to AMD Instinct MI355X through vLLM on ROCm, benchmark it against Qwen3.5's built-in MTP drafter, and quantize the target model to mxfp4 to demonstrate that speculation and quantization stack.
The result: up to...
