Cohere's Rerank 5 turns a relevance score into a probability. That lets an agent know when it found nothing
Rerank 5 Pro and Fast are generally available with calibrated scores and Context Focus, which returns only the relevant passages. The useful change for enterprise RAG is a reliable way to say no.
Listen to this article · 7 min
AI-generated narration of the full article.
Most retrieval pipelines in production contain a number that everyone treats as meaningful and nobody has checked: the reranker’s relevance score. A document comes back at 0.82, another at 0.41, and somewhere a threshold, often 0.5 because it looks reasonable, decides which ones the language model will read.
Until now, that number was mostly good for sorting. A 0.8 from one model did not mean what a 0.8 meant from another, or from the next version of the same one. Teams tuned the cutoff by hand and tuned it again on every upgrade, if they remembered.
Cohere’s Rerank 5, generally available since October 21, is built around fixing that number. Its claim is specific: a score of 0.7 means a 70% chance the document is useful.
What shipped
Two models, Rerank 5 Pro for quality and Rerank 5 Fast for volume, available on Cohere’s API, its single-tenant Model Vault, Microsoft Foundry, Amazon SageMaker and Cohere’s North platform, with private VPC and on-premises deployments arranged through Cohere. API pricing is $0.05 per million tokens for Pro and $0.02 for Fast. Both read up to 32,000 tokens, cover more than 100 languages and handle JSON and tables. Because a reranker reads raw text at query time, upgrading does not require re-indexing or re-embedding.
Two capabilities are new.
Calibrated relevance scores. Cohere checked its scores against two sets of human judgments the model was not trained on: a blind panel of graders and NIST’s TREC Deep Learning assessors. It reports an expected calibration error of about 0.06, and says competing rerankers it tested had nearly four times that error or more. For Pro, “more likely useful than not” sits at 0.50; for Fast, at 0.45; and Pro’s default threshold stays between 0.47 and 0.53 across the benchmarks Cohere ran.
Context Focus, off by default, returns the most relevant passages inside each document as scored character spans, in the same call. Pass only those spans to the model and the prompt shrinks. Across 30 question-answering datasets, Cohere says removing half of the retrieved text kept 96% of answer quality, and removing three quarters kept 87%.
The real feature is the empty result
The obvious benefit is cost. In Cohere’s test of a search agent, adding Rerank 5 Pro to keyword search raised accuracy from 46% to 49%; turning on Context Focus cut tokens per task by 19% at the default setting and by 34% at a stricter one, while accuracy rose to 57% and 58%. Fewer tokens and better answers is a rare combination, and worth testing.
The more important benefit is quieter. When a score means a probability, a threshold can mean a policy. Instead of always passing the top five results to the model, an application passes everything above the line, which might be twelve documents for a broad question and none for a question the corpus cannot answer.
“None” is the result that matters in an enterprise. An agent that retrieves nothing relevant and says so, or rewrites its query, behaves very differently from one that answers confidently from whatever happened to rank first. Our view is that many wrong answers in enterprise RAG are not the model inventing facts from nowhere; they are the model doing its best with the wrong documents. A calibrated threshold gives the system a principled way to notice that it has the wrong documents.
Read the numbers with three cautions
These are vendor results. The ranking benchmarks use RCP-nDCG@10, a metric Cohere developed to handle incomplete relevance judgments. We looked at it when Cohere first published it, in our piece on measuring retrieval as its own system. The method is reasonable; it is still Cohere grading Cohere, and the context-reduction results were scored by an LLM judge.
Useful is not correct. A calibrated score estimates the chance that a grader would find the document useful for the query. It says nothing about whether the document is current, authorized for this user or true. Permissions and freshness still have to be enforced elsewhere in the pipeline.
Compression can remove the caveat. Context Focus keeps the passages most similar to the question. In regulated content, the sentence that matters is often the qualification three paragraphs away: the exclusion in a policy, the contraindication in a label, the exception in a contract. Test aggressive settings on documents where the answer depends on an exception before using them where it counts.
What to change in a RAG pipeline
- Replace fixed top-k with a threshold on calibrated scores, and log how many documents pass for each query. A sudden change in that distribution is an early warning.
- Design the no-result path explicitly: query rewrite, a clarifying question or an honest “I could not find this,” and measure how often each happens.
- Treat the threshold as configuration under change control. Moving it from 0.5 to 0.4 changes what users are told. It deserves the same review as a prompt change.
- Evaluate Context Focus per document type, with a test set that includes questions whose answers live in exceptions and footnotes.
Retrieval quality has always decided enterprise RAG quality; we said as much when Cohere launched Compass Cloud and Embed 5. What Rerank 5 adds is the ability to say how confident retrieval is, in a number that means the same thing tomorrow.