AI & Research
What Is Retrieval Augmented Generation (RAG)?
Giving a language model the source documents it needs at question time, instead of relying on what it memorised during training.
Definition
Retrieval augmented generation, usually shortened to RAG, is an architecture in which a system retrieves relevant documents from an external store and passes them to a language model as context before it generates an answer. It was introduced by Lewis and colleagues in 2020 as a way to give models access to knowledge without enlarging the model itself.
The stages of a pipeline
- Ingest: collect source documents and normalise them to clean text
- Chunk: split documents into passages small enough to retrieve precisely
- Embed: convert each chunk to a vector and store it in an index
- Retrieve: embed the query and return the closest chunks, often combined with keyword search
- Rerank: reorder candidates with a more expensive model for better precision
- Generate: pass the selected passages to the model with an instruction to answer from them
What it fixes and what it does not
RAG addresses stale knowledge and lets a system cite its sources, because the passages used are known. It does not guarantee accuracy. If retrieval returns the wrong passage the model will answer confidently from it, which is why retrieval quality is usually the limiting factor rather than the model.
Why chunking is the part people underestimate
Chunk boundaries decide what can be retrieved as a unit. Splitting on a fixed character count cuts sentences and separates a claim from its qualifier. Splitting on document structure, using headings and sections, tends to produce passages that stand on their own. This is one reason well structured source HTML matters beyond accessibility.
Given a query, a RAG system first retrieves the top-k most relevant documents from a corpus, then conditions a language model on this retrieved context to generate an answer.