Map · Retrieval & Grounding · 05 · Reranking
Concept card · Retrieval & Grounding
Reranking is a second pass that reorders retrieved chunks before they go into the prompt.
A second pass that reorders retrieved chunks before they go into the prompt.
Oct 18: First card. The picture reorders three chunks that retrieval already returned. Sources are Azure AI Search's semantic ranking overview, Cohere's Rerank overview, and Elasticsearch's retrievers reference.
Reranking takes a list retrieval already returned and scores it again, so a better chunk can move up before you put text in the prompt. Azure AI Search's semantic ranking overview says semantic ranker adds a secondary ranking over an initial result set that was scored with BM25 or with reciprocal rank fusion. It does not search the whole corpus again. The same page says that even when the first pass returns more, only the top 50 results are sent to the ranker, and that the ranker cannot create new text. Cohere's Rerank overview says the endpoint takes a query and a list of documents and orders those documents from most to least semantically relevant to the query. Elasticsearch's retrievers reference describes a text similarity reranker that re-ranks documents by semantic similarity to a given text, using a machine learning model.
The second pass is extra work on chunks you already retrieved. Azure's semantic ranking overview says semantic ranker is a premium feature billed by usage, with a free plan that includes a monthly allowance and a standard plan after that. This card does not copy a price.
The first pass can rank a shared word above the chunk that answers the question. A second pass can move that chunk up. Azure's overview says the ranker promotes results that fit the query, and that it only reranks the existing result set. If the right chunk never made the first pass, reranking cannot find it.
A higher rank is not a fact check. Azure's overview says captions and answers from the semantic ranker are verbatim text from the index, and that the ranker cannot create new information. The chunk can still be wrong, stale, or something this user should not see.
Retrieve ten chunks for one real question. Read the top three, then read the rest. If a better chunk is lower in the list, that is the case a reranker is for. If the right chunk is missing, fix retrieval first.
Reranking a first pass that never retrieved the right document, and expecting it to appear. Azure's semantic ranking overview says the ranker does not rerun the query over the whole corpus.
The right chunk was not in the first ten results. What can a reranker do about that chunk?
Coming soon. Ten chunks from one question, with the better chunk marked if it was not in the top three.
Coming soon. How Azure's semantic ranker rescores a first pass, and what it refuses to do.