Glossary › Prompt Caching
Prompt Caching
Reusing the computed state (the KV cache) for a prompt prefix across requests. Providers bill cached tokens at a discount; serving frameworks do it automatically. Requires stable content to come first in the prompt. Cache hit rate is an operational metric.
Where Prompt Caching shows up
All matching postsPrompt Caching: The New Optimization Target
Providers cache the prefix of your prompt and bill cached tokens at a steep discount, which turns prompt layout into a cost decision. Here is how caching works, how to structure prompts for it, the mistakes that silently bust the cache, and why cache hit rate belongs on your dashboard.
#prompt-caching#context-engineering#inference#llm#serving