Map · Retrieval & Grounding · 03 · Embeddings for retrieval
Concept card · Retrieval & Grounding
Embeddings for retrieval turn a query and stored chunks into lists of numbers, then return the chunks closest to the query.
Using embeddings to find stored chunks that are close to a query.
Oct 17: First card. The picture ranks three fixed chunks against a query. It is not an embedding model. Foundations Embeddings stays without a card, and this card links to that entry. Sources are the OpenAI embeddings guide, the Gemini embeddings guide, and the OpenAI retrieval guide.
This card is the retrieval use. You embed the stored chunks, embed the query, and return the chunks that land closest. OpenAI's embeddings guide says an embedding is a list of floating point numbers, that the distance between two lists measures relatedness, and that search ranks results by relevance to a query. The Foundations entry Embeddings is that general idea, and it does not have a card yet. This page does not repeat it.
OpenAI's embeddings guide says requests are billed by the number of tokens in the input. You embed chunks when you store them, and you embed the query when you search. Gemini's File Search guide bills that tool differently: indexing embeddings are charged, and query-time embeddings for File Search are free. This card does not copy a rate.
OpenAI's embeddings guide says to retrieve documents with cosine similarity and return the highest scored ones. A close chunk can still be the wrong fact. Gemini's embeddings guide says to use RETRIEVAL_QUERY for a query and RETRIEVAL_DOCUMENT for documents on gemini-embedding-001, and a search prompt prefix rather than a sentence-similarity prefix on gemini-embedding-2. The same task format has to be used on the query and the documents.
OpenAI's retrieval guide says semantic search can surface a result that shares few or no keywords with the query. Nearness is not a check that the chunk is true, current, or permitted. Gemini's embeddings guide says the sentence-similarity task is not for search or retrieval.
Take one query and three chunks you already have. If the provider has a retrieval task type or a retrieval prompt prefix, use it on the query and the matching document form on the chunks. See whether the nearest chunk is the one you would have picked by reading.
Comparing a query embedding from one model with chunk embeddings from another, or ranking by shared words and calling that an embedding search.
The query is "When does the warehouse close?" What does this search use to pick a chunk?
Coming soon. One query and three chunks, ranked with the provider's retrieval setting.
Coming soon. How a query embedding and a document embedding are compared, and why the task setting has to match.