Cost
OpenAI's guide says embedding requests are billed by input tokens. You pay once to embed what you store, and again for each new text you compare. This card does not copy a rate.
Home · Foundations · 10 · Embeddings
Concept card · Foundations
Embeddings
What this gets you. Find text that means the same thing, even in different words, and know when a close score still isn't the answer.
Next on this goal: Chunking.
An embedding is a list of numbers an embedding model gives a piece of text, so texts with related meaning get lists that sit close together.
What changed
Oct 25: First card. A worked example shows what a close score means, and what it doesn't.
See why two texts with no word in common can still score as close, and what to check before you trust the score.
1 What it is
OpenAI's embeddings guide says an embedding is a list of floating point numbers, and that the distance between two lists measures relatedness: small distances mean related, large ones mean not. You don't read the numbers. You compare them. The usual score is cosine similarity: near 1 means the two lists point the same way. Sentence Transformers lets you choose the similarity function, such as cosine or dot product. Use the one your model's docs name. The same idea powers search, grouping similar texts, and classification, which OpenAI's guide lists. Searching your documents is its own card: Embeddings for retrieval.
Example: made up to show the idea.
Real models give hundreds or thousands of numbers per text. This one uses three, so you can see them. The lines are shortened from the sample docs of Northline Desk, a made-up product.
Question: "Can I get my money back?" → [0.8, 0.1, 0.3]
| Text (Northline Desk sample docs) | Numbers | Cosine with the question |
|---|---|---|
| "The refund window is 14 days from the charge." | [0.9, 0.1, 0.2] | 0.99 |
| "After 14 days, you get account credit only." | [0.7, 0.2, 0.4] | 0.98 |
| "Weekday support is open 09:00 to 17:00." | [0.1, 0.9, 0.2] | 0.29 |
The refund sentence shares no word with the question, and it still comes first. The credit sentence is almost as close, but on its own it only covers what happens after 14 days. Close means related, not the full answer. Reading the text is still your job.
2 Why it matters
Quality
Settings must match. Cohere and Voyage AI ask you to mark text as a query or a document for search. Gemini's guide sets a task type or a task instruction, depending on the model. Gemini also lets you pick a smaller size with output_dimensionality, which saves storage for a small loss in quality.
Risk
Gemini's guide says the spaces of gemini-embedding-001 and gemini-embedding-2 are incompatible. You can't compare across them, and an upgrade means re-embedding all your data. A score from mismatched models looks normal and means nothing.
Before you compare two embeddings
| Check | Why |
|---|---|
| Same model and version on both sides? | Different models' numbers don't line up |
| Same task or input type, or the query/document pair your provider asks for? | Some models embed queries and documents differently |
| Same size (dimensions)? | Lists of different lengths can't be compared |
| The similarity measure your model's docs name? | Cosine and dot product can rank differently on vectors that aren't normalised |
| Changed the model? | Re-embed everything you stored |
3 How to apply · try today
Take one question and three sentences from your own docs. Embed all four with the same model and settings. Rank the sentences by cosine score. Is the top one the sentence you'd pick by reading? If not, note which setting differs before you change anything else.
Common mistake
Comparing a new model's embeddings with ones you stored from an older model. The scores still come out as numbers. Re-embed what you stored first.
Self-check
Two texts score 0.98 cosine similarity. What does that tell you?
Self-check
You switch to a newer embedding model. What do you do with the vectors you stored?
Self-check
A provider asks you to mark text as a query or a document. Why?
Example: made up to show the idea.
Work out one score by hand, rank the texts for a new question, then find what broke a search.
Question: "Can I get my money back?" → [0.8, 0.1, 0.3]
| Text (Northline Desk sample docs) | Numbers | Cosine with the question |
|---|---|---|
| "The refund window is 14 days from the charge." | [0.9, 0.1, 0.2] | 0.99 |
| "After 14 days, you get account credit only." | [0.7, 0.2, 0.4] | 0.98 |
| "Weekday support is open 09:00 to 17:00." | [0.1, 0.9, 0.2] | 0.29 |
Refund row: multiply the matching numbers and add them up: 0.8 times 0.9, plus 0.1 times 0.1, plus 0.3 times 0.2, is 0.79. Divide by the two lists' lengths multiplied together (about 0.80), and you get 0.99.
New question
Question: "When can I email support?" → [0.2, 0.8, 0.1]
| Text | Numbers | Cosine with the question |
|---|---|---|
| "The refund window is 14 days from the charge." | [0.9, 0.1, 0.2] | 0.36 |
| "After 14 days, you get account credit only." | [0.7, 0.2, 0.4] | 0.49 |
| "Weekday support is open 09:00 to 17:00." | [0.1, 0.9, 0.2] | ? |
Which row comes first?
Work out the last row, or look at which way its numbers point.
Spot the setting
Your stored docs were embedded as documents with model X. Today's questions are embedded with model Y, and the top results look random. What do you fix first?
Understand it: what has to match before a score means anything
A high cosine says the lists point the same way. It does not say the text is the answer.
Cosine, dot product, and distance
Sentence Transformers lets you score pairs with cosine, dot product, or a distance. On vectors that are already normalised, dot product ranks the same as cosine. On vectors that aren't, cosine and dot product can rank differently. OpenAI's guide says its embeddings are normalised to length 1, so those two measures rank the same way there, and it recommends cosine. Use the measure your model's docs name.
Dimensions and shorter vectors
Gemini's guide lets you set output_dimensionality so the list is shorter. A shorter list saves storage for a small loss in quality. Cohere and Voyage AI also offer smaller sizes. Lists of different lengths can't be compared. See each maker's docs for the sizes that model allows.
Query and document forms
Cohere asks for input_type search_query on a question and search_document on the text you store. Voyage AI asks for input_type query or document for retrieval. Gemini's guide sets a task type on gemini-embedding-001, and a task instruction in the text on gemini-embedding-2. Use the query form on questions and the document form on what you store.
Not every maker sells an embedding model
Anthropic's embeddings page says Anthropic does not offer its own embedding model, and points to Voyage AI. Pick one model for both sides of a comparison, whichever maker you use.
Uses beyond search
OpenAI's guide lists search, clustering (grouping similar texts), recommendations, and classification. Mistral's embeddings page calls embeddings vector representations that capture meaning, and lists duplicate detection among the uses. Search against stored chunks is Embeddings for retrieval.
Upgrading the model
Gemini's guide says the spaces of gemini-embedding-001 and gemini-embedding-2 are incompatible, so an upgrade means re-embedding all your data. After you re-embed, run your retrieval checks again. Production evaluation shows how to keep a saved set of checks you can re-run.
Check yourself
Name two settings that must match before you compare two embeddings.
Show the answer
The model and version, and the size (dimensions). The task or input type, and the similarity measure your model's docs name, have to match too.
Questions
- What is an embedding?
- An embedding is a list of numbers an embedding model gives a piece of text, so texts with related meaning get lists that sit close together.
- Is this the same as Embeddings for retrieval?
- No. This card is the idea: text as numbers, and what close means. Embeddings for retrieval is the search use.
- Can I compare embeddings from two providers?
- No. Each model's numbers live in their own space. Pick one model for both sides.