In short
Start from a pre-trained model and train a small adapter instead of all its weights, using LoRA, or QLoRA with 4-bit weights. LoRA cut trainable parameters by 10,000 times and GPU memory by three times compared with full fine-tuning of GPT-3, and QLoRA fine-tuned a 65-billion-parameter model on a single 48 GB GPU. Compare the result with your quality bar before considering anything bigger.
Steps
- Check whether prompting or retrieval already meets your need before fine-tuning at all.
- Pick the smallest pre-trained model that comes close on your task.
- Fine-tune a LoRA adapter, or QLoRA if memory is tight.
- Use a small, clean, task-specific dataset and stop early when validation stops improving.
- Track energy with an emissions tracker and record it with the results.
Adapt a small part of a pre-trained model rather than retraining all of it. Parameter-efficient fine-tuning gets most of the benefit of full fine-tuning for a small fraction of the compute.
Why does it matter?
Full fine-tuning updates every weight, which needs large GPU clusters and long runs. Hu and colleagues showed that LoRA (low-rank adaptation), which trains small extra matrices alongside frozen weights, could reduce trainable parameters by 10,000 times and GPU memory by three times compared with fine-tuning GPT-3 175B. Dettmers and colleagues' QLoRA combines this with 4-bit weights and fine-tuned a 65-billion-parameter model on a single 48 GB GPU.
How do I do it?
- Question the need. Fine-tuning teaches skills and style; it is a poor way to add facts that change. See retrieval or fine-tuning.
- Start small. A small model fine-tuned on your task can match a much larger general one on that task.
- Use an adapter library. Popular open-source libraries support LoRA and QLoRA with a few lines of configuration.
- Keep data tight. Fewer, cleaner examples train faster. Remove duplicates.
- Measure. Wrap the run in a tracker such as CodeCarbon and record energy next to accuracy.
Common mistakes
- Fine-tuning the biggest model available before testing a smaller one.
- Long runs with no early stopping, which keep spending after quality has levelled off.
- Retraining for every data update instead of updating a retrieval index.
For more training techniques, read efficient training and inference.
Sources
- LoRA: low-rank adaptation of large language models, Hu et al. (2021)
- QLoRA: efficient finetuning of quantized LLMs, Dettmers, Pagnoni, Holtzman and Zettlemoyer (2023)
- Green Software Patterns: leverage pre-trained models and transfer learning, Green Software Foundation (2025)
- CodeCarbon, CodeCarbon (2026)
Related
How do I avoid training a model from scratch?
How to reuse existing models instead of pre-training your own: fine-tune, adapt with LoRA, merge fine-tuned models or retrieve knowledge.
How do I choose between retrieval and fine-tuning to save energy?
When to use retrieval (RAG) and when to fine-tune: retrieval for facts that change, fine-tuning for skills and style, and why mixing them up wastes energy.
How do I measure the carbon footprint of training a model?
Track the energy of a training run with a tool such as CodeCarbon, multiply by local grid carbon intensity, and report hardware, time and location.
Efficient AI training and inference
Techniques for developers and researchers to cut AI energy: pruning, sparsity, quantisation, distillation, KV cache, batching, mixed precision and scheduling.