RAG (retrieval-augmented generation)
Also called retrieval-augmented generation, grounded generation.
A pattern that finds relevant passages from your own data and gives them to a language model along with the question, so it answers from your content.
Last updated
What it is.
A RAG system has two parts. Retrieval searches your documents, tickets or records for the passages most relevant to the question, often using embeddings in a vector database, keyword search, or both. Generation sends those passages and the question to an LLM, which writes an answer based on them.
Because the knowledge lives in your data store, not in the model, you can update it any time without retraining, and the answer can cite the exact passages it used.
Why it matters
RAG is the usual way to make an AI assistant answer from company knowledge: policies, product docs, contracts, past cases. It reduces hallucinations on factual questions and lets you enforce permissions by only retrieving what the user is allowed to see.
Common mistakes.
- 01
Blaming the model for bad answers when retrieval found the wrong passages. Test retrieval on its own.
- 02
Splitting documents into chunks that cut tables and clauses in half.
- 03
Ignoring permissions, so the assistant can quote documents the user should not see.
Work it out with a tool.
Related terms.
- Embedding (vector embedding)A list of numbers that represents the meaning of a piece of text, an image or other data, so that similar items have similar numbers.
- Vector databaseA database built to store embeddings and quickly find the ones closest to a query vector.
- Fine-tuningTraining an existing model further on your own examples so it behaves the way you want, such as following a format, tone or task more reliably.
- Hallucination (AI)When an AI model produces information that is false or made up, presented as if it were true.
Guides and pages
Browse the full glossary or read answers to common questions.
