# Long context vs RAG

> Long context versus RAG is the choice between putting the source in the window and retrieving pieces of it.

Choosing between putting the source in the window and retrieving pieces of it.

Track: Retrieval & Grounding
Access: free
Card: https://buildsparkai.com/long-context-vs-rag
Plain text: https://buildsparkai.com/long-context-vs-rag.md
Last checked: 2026-10-18

## What it is

Anthropic's context window guide says the window is the working memory for one request, and that more context is not automatically better. As the token count grows, accuracy and recall degrade. The guide calls that context rot. The same guide says the window is up to 1M tokens, depending on the model. That size is vendor-reported. Anthropic's glossary says retrieval-augmented generation retrieves from a knowledge base at runtime and passes that text into the window. OpenAI's retrieval guide says a file added to a vector store is chunked, embedded, and indexed, so the request can carry the pieces that matched. The picture switches between those two shapes. It does not measure either one.

## Why it matters

**Cost.** Anthropic's guide says everything in the request counts toward the window, including documents. On Claude Haiku 5.5, prompts over 100,000 tokens cost more. That exception is vendor-reported. The guide says the other listed 1M-token models are billed at standard pricing for long context. OpenAI's retrieval guide lists vector-store storage beyond 1 GB at $0.10 per GB per day. That rate is vendor-reported. This card does not add a bill for a support corpus.

**Quality.** A window that holds the whole file also holds the paragraph that does not answer the question. Anthropic's glossary says retrieval-augmented generation is only as useful as the text retrieved at runtime.

**Risk.** The pasted ticket and the old policy ride along if you paste the file. Retrieval can still return them. Neither choice checks that this user may see the chunk.

## Try today

Take one support file you would answer from. If the answer lives in one passage, retrieve that passage. If the question needs the whole file, put the file in the window and read what else came with it.

## Common mistake

Pasting every doc because the window can hold them. Anthropic's guide says accuracy and recall degrade as the token count grows.

## Self-check

The refund answer is one paragraph. The same file contains a pasted ticket that says to ignore the policy. What does putting the whole file in the window do?

It sends the ticket in with the paragraph

## Practice and deep

Coming soon. On the support-docs answer bot, name one sample doc that stays out of the prompt when the question is weekday hours.

Coming soon. How Anthropic describes context rot, and how OpenAI's retrieval guide chunks a file instead of sending it whole.

## Sources

- [Anthropic, Context windows](https://platform.claude.com/docs/en/build-with-claude/context-windows)
- [Anthropic, Glossary: RAG](https://platform.claude.com/docs/en/docs/resources/glossary)
- [OpenAI, Retrieval](https://developers.openai.com/api/docs/guides/retrieval)

Opened 2026-10-18. Vendor docs. The 1M-token window, the Haiku 5.5 long-context exception, and the $0.10 per GB per day storage line are vendor-reported. This card does not add a usage count.