Skip to content

RAG and knowledge assistants

Answers from your documents, with sources you can check.

A knowledge assistant answers questions from your policies, manuals or support articles and shows where each answer came from. When the answer isn't in the documents, it says so.

Who it's for

Teams with documents people search all day, such as policies, manuals, support articles and contracts, who want answers they can check.

What we deliver

  • Ingestion and chunking that respects document structure.
  • Hybrid search (vector plus keyword) with reranking.
  • Answers with citations, and a clear "not in the documents" when the answer isn't there.
  • An evaluation set with known evidence, so every change is measured.
  • Document-level access control.

How we measure it

  • Retrieval hit rate and recall.
  • Answer correctness and faithfulness to sources.
  • Correct abstention when the answer isn't in the documents.
  • Latency and cost per question.

What it looks like in practice

The RAG retrieval quality benchmark compares three setups on the same Kubernetes documentation (a pinned snapshot of 389 pages) and the same 55 evaluation questions. Baseline uses fixed-size chunks and dense search. Intermediate uses heading-aware chunks. Advanced adds keyword search, fuses the two result lists and reranks them.

MetricBaselineIntermediateAdvanced
Hit Rate@50.8040.8480.935
MRR@100.6360.6510.870
Answer Correctness0.8040.8040.913
Run 20261105-0038-d71e. Retrieval and correctness are scored on the 46 answerable questions.

Hybrid search and reranking gave the clearest gain, with confidence intervals that exclude zero. Better chunking alone showed no clear quality gain but used about a third less context. The gain cost latency.

The limits matter as much as the numbers: 55 questions (9 multi-section and 9 unanswerable), one corpus and one run, and the same model generated, answered and judged.

Related work

Built in the open

Questions

Common questions

How do you know the AI is working?
Each project gets a test set from your real work and agreed metrics (for example, answer correctness for an assistant or hours saved for an automation), measured before and after every change.
Where does our data go?
It stays in your accounts wherever possible. AI models are used through business API plans whose terms don't allow training on your data, or through your own cloud (for example, AWS Bedrock) when it must stay there. [TODO: verify current provider terms before publishing]
Do we need clean data before we start?
No. Discovery looks at the data you have and says what's usable now and what needs fixing. Some projects start with that fix.

Have a process that eats hours every week? Let's look at it together.