RAG vs fine-tuning
Use RAG when the model needs to know your facts, especially facts that change. Use fine-tuning when the model needs to behave differently: a format, a style or a narrow task done more reliably. Many products use RAG first and add fine-tuning only for a specific behaviour.
Last updated
RAG
Retrieve relevant passages from your data and give them to the model with each question.
RAG keeps knowledge outside the model, in a search index or vector database. At question time it finds the most relevant passages and sends them with the question, so the answer is based on your current content and can cite it.
Pick it when
- Answers must come from your documents, records or policies.
- The content changes often and must be current.
- Users need to see the source for each answer.
- Different users may see different documents, so permissions matter.
Fine-tuning
Train an existing model further on your own examples so its behaviour changes.
Fine-tuning changes the model itself. From a set of example inputs and ideal outputs, it learns a pattern: a label set, a writing style, a strict output format, domain wording. Updating it means preparing new examples and training again.
Pick it when
- A good prompt still cannot get the format, tone or labels right reliably.
- You have a solid set of high-quality examples of the behaviour you want.
- The task is narrow and high-volume, so a smaller tuned model could save cost or time.
- The knowledge needed is stable and the problem is behaviour, not facts.
How they differ.
| Aspect | RAG | Fine-tuning |
|---|---|---|
| What it changes | What the model is told for each request | How the model behaves |
| Best for | Facts, documents, current data | Format, style, labels, narrow tasks |
| Updating knowledge | Re-index the changed documents | Prepare examples and train again |
| Citations | Can point to the exact source passage | No built-in source |
| Permissions | Can filter what each user can retrieve | Whatever was trained in is available to everyone |
| What drives cost | Embeddings, storage, search, and longer prompts per call | Data preparation, training runs, and hosting or per-call rates for the tuned model |
| Effect on hallucinations | Reduces them on factual questions when retrieval is good | Does not stop the model inventing facts |
| Main risk | Poor retrieval returns the wrong passages | Poor examples teach the wrong behaviour |
Using them together.
They solve different problems, so they combine well. A common setup is a model fine-tuned to follow your answer format and tone, fed by RAG with the current facts. Start with RAG and good prompts, measure with evals, and add fine-tuning only when a specific behaviour still falls short.
Estimate it for your case.
Common follow-up questions.
Can fine-tuning teach a model our company knowledge?
It can absorb some of it, but it is a poor fit for facts that change, it cannot cite sources and it cannot respect per-user permissions. RAG is usually the better way to give a model company knowledge.
Which is faster to build?
A first RAG version is usually quicker, because it needs your documents and a search index rather than a curated training set. Getting retrieval quality high still takes careful work.
Do we need a vector database for RAG?
Not always. Keyword search can work for some content, and many teams use vector search inside a database they already run. Pick based on your data and how you need to filter it.
Related pages.
- AI agent vs workflow automationComparison
- How do I reduce LLM API costs?Question
- Is my AI system high-risk under the EU AI Act?Question
- All questionsEvery question and comparison.
- GlossaryEvery term, in plain words.
- AI by industryAI use cases, costs and risks per industry.
- AI automationHow OceanLab automates company workflows with AI.
- Human feedbackRLHF and SFT data from OceanLab Labs.
