Map · Retrieval & Grounding · 02 · Chunking
Concept card · Retrieval & Grounding
Chunking splits a document into pieces that retrieval can store and return.
Splitting documents into pieces that retrieval can store and return.
Oct 17: First card. The picture cuts one note two ways. Sources are the OpenAI retrieval guide, the Gemini File Search guide, and LangChain's text splitter guide.
Chunking splits a document into pieces small enough to store and return. LangChain's text splitter guide says those pieces should be retrievable on their own and fit in a context window. OpenAI's retrieval guide says a file added to a vector store is chunked, embedded, and indexed. Gemini's File Search guide says an imported file is broken into chunks, then embedded.
Each stored piece can be embedded and kept. Overlap repeats text across pieces, so the same sentence can be stored more than once. OpenAI's retrieval guide documents a default of 800-token chunks with 400 tokens of overlap, and a chunk size from 100 to 4096 tokens. Those are that API's settings, not a universal size.
LangChain's text splitter guide says splitters break large documents into smaller chunks that can be retrieved on their own and that fit in a context window. A piece that starts mid-sentence can be returned without the heading it needed.
OpenAI's retrieval guide says files added to a vector store are chunked automatically unless you set a chunking strategy. A default cut can separate a fact from the name it refers to. Gemini's File Search guide also chunks on import, and documents a config for a maximum number of tokens per chunk and an overlap.
Take one page you would search. Mark the cuts so each piece still names the thing it is about. If a sentence uses "it" or "the company," keep the name in that piece.
Cutting on a fixed length so a heading, a sentence, or a table row is split, then retrieving the half that does not say what it refers to.
A sentence says "It closes at 18:00." The heading "Warehouse hours" is in the previous piece. What is wrong with that cut?
Coming soon. One page you already search, cut so each piece still names its subject.
Coming soon. How OpenAI and Gemini document chunk size and overlap, and how a framework splitter differs.