RAG: retrieval-augmented generation
RAG is not a magical knowledge base. It first retrieves passages from a corpus, then supplies them to a model to produce an evidence-led answer. The final accuracy depends on the whole pipeline.
Build a usable corpus
Select documents with reuse permission, date, version and stable identifiers. Chunking helps retrieval but may remove context; retain original locations and references.
Retrieve, then synthesize
Lexical, vector or hybrid retrieval selects candidate passages. A second-stage ranking can reorder them. The model then drafts an answer and may still omit nuance even when the retrieved material was relevant.
Evaluate the components separately
A trustworthy outcome requires separate measures for corpus coverage, retrieval relevance, citation accuracy, answer faithfulness and the ability to report missing evidence. A hyperlink alone does not validate its contents.
Teaching example
A school library collects ten official documents about a historical period. In answering a question, an assistant should present the passages and dates, then avoid a firm conclusion if sources conflict.
Architectural choices and corpus quality
A RAG system may use lexical, vector or hybrid retrieval. Selection depends on vocabulary, language, document updates and needed precision. Preserve document identifiers and passage locations after chunking, and measure relevant retrieval before judging generated answers. Hybrid retrieval can help when names or technical terms need literal matching.
Controlled case: university documentation
Build a question set against dated university regulations. Some questions have direct answers, others lack sufficient evidence or contain conflicting versions. For each answer, expose the exact passage and separately evaluate factual correctness, citation support and abstention. When regulations change, rerun the same questions to detect update failures.
Core steps
- 01Collect and attribute
- 02Retrieve passages
- 03Generate with references
- 04Verify claims
Three questions to ask
Do the cited passages answer the exact question?
How does the system handle missing relevant documents?
Are dates and versions of the sources available?
Read the source documents
Versions, reuse terms and document status can change. Check them on the original sites.