Glossary › Speculative Decoding
Speculative Decoding
A serving technique where a small draft model proposes several tokens and the large model verifies them in one pass, accepting the matching prefix. Same output distribution, two to three times more tokens per expensive forward pass.