# How do I cut the energy of hyperparameter searches?

> How to tune models with less compute: smarter search than full grids, early stopping of poor trials, small pilots, good defaults and a budget.

Source: https://greenai.guide/how-do-i/cut-the-energy-of-hyperparameter-searches/  
Updated: 2026-11-19

## In short

Replace large grid searches with random or adaptive search, stop poor trials early, run pilots on small data or small models first, and start from published defaults for your architecture. Track energy for every trial, not just the winner, and set a compute budget before you start. Tuning runs often cost more than the final training run, and they rarely get reported.

## Steps

1. Set a compute budget for the search before you begin.
2. Start from published default hyperparameters for your model family.
3. Use random or adaptive search instead of a full grid.
4. Stop trials early when they are clearly worse than the best so far.
5. Pilot on a subset of data or a smaller model before scaling up.
6. Record energy and emissions for every trial alongside accuracy.

**Tune smarter, not wider.** A full grid over many hyperparameters multiplies the cost of training by the number of
combinations, and most of those runs are clearly worse early on.

## Why does it matter?

Strubell, Ganesh and McCallum's 2019 paper quantified the cost of developing language models, including the
experimentation around them, not just a single final training run. Schwartz and colleagues' "Green AI" argued that
researchers should report the computational cost of their results, including the experiments it took to reach them,
so that efficiency is valued alongside accuracy.

## What makes the biggest difference?

- **Early stopping of poor trials.** Successive-halving style methods give every trial a small budget and continue
  only the promising ones. Most tuning libraries support this.
- **Random or adaptive search.** Instead of running every combination, sample settings at random or let an
  adaptive method use earlier results to choose the next trial.
- **Small pilots.** Rule out ideas on a subset of data or a smaller model.
- **Defaults first.** Well-tested defaults for common architectures are often close to optimal.
- **Track every trial.** CodeCarbon's documentation shows how to track each configuration and compare accuracy
  against emissions.

## Common mistakes

- **Reporting only the best run.** Include all trials in your compute total; see
  [how to report the energy use of your research](/how-do-i/report-energy-use-in-a-research-paper/).
- **Tuning on the full dataset from day one.**
- **No budget.** Without one, searches expand to fill the cluster. See
  [how to set a compute budget](/how-do-i/set-a-compute-budget-for-a-research-project/).

For more, read the [guide for researchers](/guides/researchers/).

## Sources

- [Energy and policy considerations for deep learning in NLP](https://arxiv.org/abs/1906.02243), Strubell, Ganesh and McCallum, ACL (2019)
- [Green AI](https://doi.org/10.1145/3381831), Schwartz, Dodge, Smith and Etzioni, Communications of the ACM (2020)
- [Comparing model efficiency with carbon tracking](https://docs.codecarbon.io/latest/tutorials/comparing-model-efficiency/), CodeCarbon documentation (2025)
