Prompt Caching
Also called Context Caching
A feature that lets an AI provider reuse its processing of a repeated prompt prefix, which reduces cost and delay for requests that begin the same way.
What is prompt caching?
Prompt caching lets a model provider remember the processed form of the start of a prompt, so that later requests with an identical start do not pay to process it again. Output is generated as normal, so responses are the same as without caching.
How does it work?
The reusable part is the beginning of the prompt, such as tool definitions, the system prompt and long reference documents. A cache hit needs an exact match of that prefix. In Anthropic's implementation, reading from the cache costs much less than processing tokens normally, while writing to it costs somewhat more, and entries last 5 minutes by default, refreshed each time they are used. Other providers offer similar features with their own rules.
What is a simple example?
A support bot sends the same 20-page policy document with every customer question. With caching, the document is processed once and reused for later questions, which lowers latency and cost.
Why does it matter?
Apps that repeat large instructions or documents can save money and respond faster. To benefit, put the stable content first and the changing question last.