# How do I reduce the energy of training a model?

> The main levers for cutting training energy: start pre-trained, right-size, use mixed precision, stop early, pick clean places and times, measure.

Source: https://greenai.guide/how-do-i/reduce-the-energy-of-training-a-model/  
Updated: 2026-11-19

## In short

Start from a pre-trained model where you can, choose the smallest architecture that could meet your bar, train in mixed precision, stop when validation stops improving, and run on efficient hardware in a low-carbon region and time. Patterson and colleagues found that the choice of model, data centre and processor can reduce the carbon footprint of training by up to around 100 to 1,000 times. Measure every run so the trade-offs are visible.

## Steps

1. Check whether a pre-trained model can be fine-tuned instead of training from scratch.
2. Pilot ideas at small scale before launching a full run.
3. Train in mixed precision and stop early when validation stops improving.
4. Run on efficient accelerators in a low-carbon region, and schedule flexible runs for cleaner hours.
5. Save checkpoints so runs can pause, resume and avoid starting over.
6. Track energy and emissions for every run, including failed ones.

**Training energy is set mostly by decisions made before the run starts.** What you train, on what, where and when
matters more than any tweak during the run.

## Why does it matter?

Strubell, Ganesh and McCallum's 2019 paper quantified the financial and environmental cost of training language
models and proposed ways to cut it. Patterson and colleagues later found that the choice of model architecture, data
centre and processor can reduce the carbon footprint of training by up to around 100 to 1,000 times.

## What makes the biggest difference?

1. **Not training from scratch.** Fine-tuning a pre-trained model, or an adapter on it, usually costs a tiny
   fraction of pre-training. See [how to avoid training from scratch](/how-do-i/avoid-training-a-model-from-scratch/).
2. **Right-sizing.** A smaller architecture that meets the bar beats a large one that exceeds it.
3. **Mixed precision.** Training in half precision with a full-precision master copy nearly halved memory use in
   Micikevicius and colleagues' work, letting larger batches fit on the same hardware.
4. **Early stopping.** End runs when validation loss plateaus.
5. **Where and when.** Pick efficient accelerators and a low-carbon region, and run flexible jobs when the grid is
   cleaner: see [how to schedule training](/how-do-i/schedule-training-when-the-grid-is-greener/).

## Common mistakes

- **Counting only the final run.** Tuning and failed runs often cost more; see
  [how to cut the energy of hyperparameter searches](/how-do-i/cut-the-energy-of-hyperparameter-searches/).
- **Not checkpointing**, so an interruption means starting over.
- **No measurement.** Put a tracker such as CodeCarbon in the training script.

For the full toolkit, read [efficient training and inference](/practice/efficient-training-and-inference/).

## Sources

- [Carbon emissions and large neural network training](https://arxiv.org/abs/2104.10350), Patterson et al. (2021)
- [Mixed Precision Training](https://arxiv.org/abs/1710.03740), Micikevicius et al. (arXiv 1710.03740) (2017)
- [Energy and policy considerations for deep learning in NLP](https://arxiv.org/abs/1906.02243), Strubell, Ganesh and McCallum, ACL (2019)
- [CodeCarbon](https://codecarbon.io/), CodeCarbon (2026)
