Who it's for
Teams with documents people search all day, such as policies, manuals, support articles and contracts, who want answers they can check.
What we deliver
- Ingestion and chunking that respects document structure.
- Hybrid search (vector plus keyword) with reranking.
- Answers with citations, and a clear "not in the documents" when the answer isn't there.
- An evaluation set with known evidence, so every change is measured.
- Document-level access control.
How we measure it
- Retrieval hit rate and recall.
- Answer correctness and faithfulness to sources.
- Correct abstention when the answer isn't in the documents.
- Latency and cost per question.
What it looks like in practice
The RAG retrieval quality benchmark compares three setups on the same Kubernetes documentation (a pinned snapshot of 389 pages) and the same 55 evaluation questions. Baseline uses fixed-size chunks and dense search. Intermediate uses heading-aware chunks. Advanced adds keyword search, fuses the two result lists and reranks them.
| Metric | Baseline | Intermediate | Advanced |
|---|---|---|---|
| Hit Rate@5 | 0.804 | 0.848 | 0.935 |
| MRR@10 | 0.636 | 0.651 | 0.870 |
| Answer Correctness | 0.804 | 0.804 | 0.913 |
Hybrid search and reranking gave the clearest gain, with confidence intervals that exclude zero. Better chunking alone showed no clear quality gain but used about a third less context. The gain cost latency.
The limits matter as much as the numbers: 55 questions (9 multi-section and 9 unanswerable), one corpus and one run, and the same model generated, answered and judged.