# How do I choose between retrieval and fine-tuning to save energy?

> When to use retrieval (RAG) and when to fine-tune: retrieval for facts that change, fine-tuning for skills and style, and why mixing them up wastes energy.

Source: https://greenai.guide/how-do-i/choose-between-rag-and-fine-tuning/  
Updated: 2026-11-19

## In short

Use retrieval when the model needs facts that change or are specific to you: put them in a searchable index and fetch only the relevant passages. Use fine-tuning when the model needs a skill, format or style it lacks. Retraining to keep facts up to date repeats training costs, while pasting whole documents into every prompt wastes inference, so each approach has its place.

## Steps

1. List what the model is missing: facts, a skill, a format or a tone.
2. Put changing or private facts in a searchable index and retrieve only relevant passages.
3. Fine-tune a small adapter only for skills, formats or styles that prompting can't achieve.
4. Update the index when content changes, processing only what changed.
5. Measure tokens per request and quality for each option before deciding.

**Retrieve facts; fine-tune skills.** Choosing the wrong one means paying again and again, either for retraining or
for oversized prompts.

## What is the difference?

- **Retrieval-augmented generation (RAG)** looks up relevant passages from your own content and gives them to the
  model with the question. Lewis and colleagues introduced the approach in 2020 for knowledge-intensive tasks. The
  model's weights do not change, so updating knowledge means updating an index.
- **Fine-tuning** changes the model's weights, or a small adapter such as LoRA, so it learns a behaviour: a format,
  a classification scheme, a house style.

## Which uses less energy?

**It depends on how often things change and how much text each request carries.**

| Situation | Usually better |
|---|---|
| Facts change weekly or monthly | Retrieval |
| Private documents, policies, product data | Retrieval |
| A fixed output format or labelling scheme | Fine-tuning a small model |
| Very high volume, narrow task | Fine-tuning, so prompts can be short |

GreenCode, an ITEA research project, describes using retrieval so its own AI reads only the data it needs, and
processing only what changed between runs.

## Common mistakes

- **Retraining to add facts.** Every update repeats the training cost.
- **"Stuff everything in the prompt" RAG.** Retrieving twenty long passages when two would do multiplies input
  tokens. See [how to make vector search and RAG more efficient](/how-do-i/make-vector-search-and-rag-more-efficient/).
- **Fine-tuning a big model** when a small fine-tuned one would do.

For the wider principle, read [orchestration over generation](/articles/orchestration-over-generation/).

## Sources

- [Retrieval-augmented generation for knowledge-intensive NLP tasks](https://arxiv.org/abs/2005.11401), Lewis et al., NeurIPS (2020)
- [Green AI and energy-aware computing](https://greencode.ai/benefit/green-ai-and-energy-aware-computing/), GreenCode (2026)
- [LoRA: low-rank adaptation of large language models](https://arxiv.org/abs/2106.09685), Hu et al. (2021)
