What has to stay the same for a prefix to be reused?

The prefix itself. A change to that prefix, the model, or the tool list can mean it is not reused.

Does a cache hit answer the new question?

No. The new part of the request is still processed. Caching can change the cost of the reused prefix.

Map · Foundations · 04 · Prompt Caching

Concept card · Foundations

Prompt Caching

Prompt caching reuses an unchanged prefix of a prompt so a later request can skip processing that part again.

Reusing an unchanged prefix so a repeated prompt can cost less than sending it from scratch.

Last checked 2026-10-17 Free card 4 min

Ask about Prompt Caching

What changed

Oct 17: First card. The picture compares an unchanged prefix with a prefix that changed. Sources are the OpenAI, Anthropic, and Gemini caching guides.

1 What it is

A prompt often starts with the same instructions or the same document, then ends with a new question. Prompt caching keeps that shared beginning, the prefix, so a later request can reuse it instead of processing those tokens from scratch.

A picture of one prompt. The prefix is the part that stays the same. The question is the part that changes. A real cache also depends on the model and on the provider's size and time rules.

Prefix Instructions and the document Can be reused

This turn The new question Sent fresh

The prefix matches the previous request, so a provider can reuse it. The new question is still sent fresh.

2 Why it matters

Cost

A reused prefix can be billed at a lower input rate than processing that text again. A cache hit is not guaranteed, and each provider sets its own size rule and time limit.

Quality

The new question is still processed. Caching does not answer it for you.

Risk

The prefix is still sent to the provider. A change to the prefix, the model, or the tool list can mean the earlier prefix is not reused.

3 How to apply · try today

Take one prompt you send often. Put the stable instructions and the stable document first, and the part that changes last. Send it twice and look for a cached-input count in the usage report.

Common mistake

Editing the start of the prompt, or switching the model or the tools, and expecting the previous prefix to be reused.

Self-check

The instructions and the document stay the same. Only the question changes. What can be reused?

Questions

What is prompt caching?: Prompt caching reuses an unchanged prefix of a prompt so a later request can skip processing that part again.
What has to stay the same for a prefix to be reused?: The prefix itself. A change to that prefix, the model, or the tool list can mean it is not reused.
Does a cache hit answer the new question?: No. The new part of the request is still processed. Caching can change the cost of the reused prefix.