In short
Use retrieval when the model needs facts that change or are specific to you: put them in a searchable index and fetch only the relevant passages. Use fine-tuning when the model needs a skill, format or style it lacks. Retraining to keep facts up to date repeats training costs, while pasting whole documents into every prompt wastes inference, so each approach has its place.
Steps
- List what the model is missing: facts, a skill, a format or a tone.
- Put changing or private facts in a searchable index and retrieve only relevant passages.
- Fine-tune a small adapter only for skills, formats or styles that prompting can't achieve.
- Update the index when content changes, processing only what changed.
- Measure tokens per request and quality for each option before deciding.
Retrieve facts; fine-tune skills. Choosing the wrong one means paying again and again, either for retraining or for oversized prompts.
What is the difference?
- Retrieval-augmented generation (RAG) looks up relevant passages from your own content and gives them to the model with the question. Lewis and colleagues introduced the approach in 2020 for knowledge-intensive tasks. The model's weights do not change, so updating knowledge means updating an index.
- Fine-tuning changes the model's weights, or a small adapter such as LoRA, so it learns a behaviour: a format, a classification scheme, a house style.
Which uses less energy?
It depends on how often things change and how much text each request carries.
| Situation | Usually better |
|---|---|
| Facts change weekly or monthly | Retrieval |
| Private documents, policies, product data | Retrieval |
| A fixed output format or labelling scheme | Fine-tuning a small model |
| Very high volume, narrow task | Fine-tuning, so prompts can be short |
GreenCode, an ITEA research project, describes using retrieval so its own AI reads only the data it needs, and processing only what changed between runs.
Common mistakes
- Retraining to add facts. Every update repeats the training cost.
- "Stuff everything in the prompt" RAG. Retrieving twenty long passages when two would do multiplies input tokens. See how to make vector search and RAG more efficient.
- Fine-tuning a big model when a small fine-tuned one would do.
For the wider principle, read orchestration over generation.
Sources
- Retrieval-augmented generation for knowledge-intensive NLP tasks, Lewis et al., NeurIPS (2020)
- Green AI and energy-aware computing, GreenCode (2026)
- LoRA: low-rank adaptation of large language models, Hu et al. (2021)
Related
How do I make vector search and RAG more efficient?
Ways to cut the compute of retrieval-augmented generation: approximate indexes, fewer and shorter passages, incremental indexing and smaller models.
How do I fine-tune a model without huge compute?
How to adapt a pre-trained model with parameter-efficient fine-tuning such as LoRA and QLoRA, so it fits on one GPU and uses far less energy.
Orchestration over generation
Why reusing vetted components and routing requests to the smallest capable model can cut AI's energy use, and what the evidence says so far.
Choose the right model for the job
How to right-size AI: when to use small or specialised models, when a frontier model earns its cost, how routing works, and when not to use AI.