greenai.guide

Home › Good practice › How to measure the energy and carbon of AI

Good practice

How to measure the energy and carbon of AI

Updated · 6 min read

In short

Measure AI energy by metering the hardware you control and estimating the services you don't, then turn both into carbon with local, time-matched grid data. Record the boundary, the method and the uncertainty with every figure, and report it per unit of useful work, not just as a total.

Key points

  • Meter where you can (hardware counters, GPU telemetry, plugs, scheduler logs) and estimate where you can't (hosted APIs, public cloud).
  • Every figure needs a stated system boundary: what is in, what is out, and why.
  • Report energy per unit of useful work, such as per request or per document, so efficiency is visible as usage grows.
  • Use location-based grid carbon intensity for the region and hour the work ran; certificates do not change it.
  • Repeat measurements and give a range. A single before-and-after pair can be noise.

Why measure at all?

You cannot reduce, compare or honestly claim what you have not measured. Measurement turns "AI uses a lot of energy" into numbers you can act on: which feature, model or job costs most, and whether a change helped.

At the 2026 UK green AI sandpit of mathematicians, computer scientists and industry, measurement came up again and again as the bottleneck. Many published figures are estimates, methods are often not documented, and the same work can produce very different numbers depending on what is counted. This guide sets out a method that holds up.

Should I meter or estimate?

Meter the hardware you control; estimate the services you don't. Both are valid, as long as every figure says which it is.

Situation Approach Typical tools
Your own laptop, workstation or server Meter with hardware counters perf or powercap (Intel/AMD RAPL counters), PowerJoular, Scaphandre
Your own GPUs Meter with vendor telemetry, sampled over time nvidia-smi or NVML, CodeCarbon
A research cluster Read the scheduler's energy accounting Slurm sacct (ConsumedEnergy field), where enabled
Public cloud virtual machines Estimate from utilisation and billing Cloud Carbon Footprint, provider carbon dashboards
Hosted AI APIs Estimate from tokens and model assumptions EcoLogits
Comparing open models before you choose Use a standard benchmark Hugging Face AI Energy Score

CodeCarbon's own guidance draws the same line: use CodeCarbon for code running on hardware you manage, and EcoLogits for calls to remote generative AI services, where you have no access to the hardware.

Know the limits of your meter

  • GPU telemetry samples, it does not watch continuously. Researchers found that on A100 and H100 GPUs, nvidia-smi samples only about 25% of the runtime for power, which can cause large under- or over-estimates for short jobs. Sample for the whole job, run long enough to cover many sampling intervals, and integrate over time.
  • Software meters are models. Check one against a wall-plug meter (a metering smart plug is enough) once per platform.
  • Cloud numbers are estimates. Hardware counters are usually unavailable on public-cloud virtual machines, so label cloud figures as estimated.

What should the system boundary include?

Draw the boundary wide enough to include everything significant, and write it down. A figure without a boundary cannot be compared with anything.

Google's 2025 paper on Gemini is a good illustration. It reports a median text prompt at 0.24 Wh when it counts the accelerators, host machines, idle capacity and data-centre overhead, and 0.10 Wh when it counts only active TPU and GPU use. Both are honest; they answer different questions. Google's boundary also leaves out the external network, users' devices and model training, and the figures have not been independently verified.

Work through these layers and decide, for each, whether it is in or out:

  1. The accelerator (GPU, TPU) doing the maths.
  2. The rest of the server: CPU, memory, storage.
  3. Idle and reserved capacity kept ready for your work.
  4. Data-centre overhead: cooling and power conversion, captured by PUE.
  5. The network between users and the service.
  6. Users' devices.
  7. Embodied emissions: the share of the hardware's manufacture attributable to your use.
  8. Training and fine-tuning, spread over the model's useful life.
  9. The surrounding process: retries, failed runs, human review time.

The sandpit made the same point about claims such as "AI costs thirty times more than doing it by hand": the answer depends on whether you count the person's working time, their equipment and everything around them. Define the boundary before you compare.

What should I record?

Record enough that someone else could repeat the measurement and get a similar number. For each figure, keep:

  • What was measured: model and version, task, dataset or prompt set, input and output token counts.
  • Where and when: hardware, region, date and time (needed for carbon intensity).
  • How: tool and version, sampling rate, measured or estimated, idle baseline included or excluded.
  • The boundary: which of the layers above are in.
  • The allocation rule: how shared or idle energy was split between users or services.
  • The spread: number of runs, mean and range.
  • Quality: the accuracy or task-success rate at the time. A cheap wrong answer is not a saving.

How do I turn energy into carbon?

Multiply energy by the carbon intensity of the grid where and when it ran, and add embodied emissions if they are in your boundary. The Green Software Foundation's Software Carbon Intensity (SCI) specification, published as ISO/IEC 21031:2024, puts it as a rate:

SCI = ((E × I) + M) per R

where E is energy, I is the location-based grid carbon intensity, M is embodied emissions and R is the functional unit. The specification uses location-based grid figures and does not let market-based measures such as renewable certificates, power purchase agreements or offsets reduce the score.

  • Use regional, time-matched data where you can. In Great Britain, the National Energy System Operator's Carbon Intensity API gives regional forecasts and estimates; Electricity Maps covers many other countries.
  • Fall back to annual national averages only when nothing finer exists, and say so.

Which per-task metrics work best?

Choose a functional unit that matches the value your AI delivers, and report against it. Totals rise with growth and hide efficiency; per-unit figures show it.

Good functional units for AI work:

  • per request, per answer or per conversation
  • per document summarised, ticket classified or image produced
  • per successful task (better still, because failed attempts count)
  • per training run or per fine-tuned model, for development work

Then track energy per unit, carbon per unit, quality per unit and total side by side. If energy per unit falls while the total rises, efficiency is improving but use is growing faster: that is the rebound effect, and your reporting should show it.

How do I handle uncertainty?

Repeat, report a range and only claim changes bigger than the noise. Readings vary with temperature, background processes and frequency scaling.

  • Measure the idle baseline first on the same machine.
  • Run each test several times; report mean and spread.
  • Change one thing at a time and re-measure on the same set-up and load.
  • For estimates, list every assumption (model size, hardware, utilisation, PUE) and give low and high cases.
  • Replace estimates with measured or supplier-reported figures as they become available.

Published per-prompt figures are useful context, not a substitute for your own numbers. Epoch AI's bottom-up estimate of about 0.3 Wh for a typical ChatGPT query rises to about 2.5 Wh with a 10,000-token input and almost 40 Wh with a 100,000-token input. Your usage pattern matters more than any headline average.

Checklist

  • [ ] Each figure is labelled measured or estimated, with tool and version
  • [ ] Boundary written down, including what was left out and why
  • [ ] Idle baseline recorded; allocation rule for shared energy stated
  • [ ] GPU power sampled over the whole job, not by snapshot
  • [ ] Energy reported per functional unit, alongside total and quality
  • [ ] Carbon calculated with location-based, time-matched grid data
  • [ ] Several runs; mean and range reported
  • [ ] Method stored with the results so others can repeat it

What to do next

  1. Pick one AI feature or workload and define its functional unit.
  2. Add CodeCarbon to a training or local inference script, or EcoLogits to your AI API client, and log energy with each request or run.
  3. Write a one-page measurement method: boundary, tools, allocation rule, carbon data source.
  4. Benchmark candidate models before you choose one; see choose the right model.
  5. Before you publish any figure, check it against honest green claims.
  6. Read why measurement is the bottleneck, and find measurement tools in the products directory or help from consultants.

Frequently asked questions

How much energy does one AI prompt use?

Published figures for a typical text prompt are a fraction of a watt-hour: Google reports a median Gemini text prompt at 0.24 Wh, and Epoch AI estimates about 0.3 Wh for a typical ChatGPT query. Long inputs, reasoning and image or video generation can cost far more.

Can I measure the energy of a hosted AI service like ChatGPT?

Not directly. You can estimate it from token counts and model assumptions with a tool such as EcoLogits, and ask the supplier for its own figures and method.

Is runtime a good stand-in for energy?

No. Speed and energy often move together but not always, so measure energy directly and report both.

What is a system boundary?

It is the list of what your figure includes: for example the GPU only, the whole server, the data centre overhead, the network, users' devices and the hardware's manufacture. Two honest figures can differ tenfold because their boundaries differ.

Sources

  1. Measuring the environmental impact of delivering AI at Google scale, Google (arXiv 2508.15734) (2025)
  2. How much energy does ChatGPT use?, Epoch AI (2025)
  3. Part-time power measurements: nvidia-smi's lack of attention, Yang et al. (arXiv 2312.02741) (2023)
  4. Software Carbon Intensity (SCI) specification, Green Software Foundation (2024)
  5. ISO/IEC 21031:2024 Software Carbon Intensity (SCI) specification, ISO (2024)
  6. When to use CodeCarbon, CodeCarbon (2025)
  7. EcoLogits documentation, EcoLogits (2025)
  8. AI Energy Score, Hugging Face (2025)
  9. Carbon Intensity API, National Energy System Operator (2025)
  10. Slurm sacct manual (ConsumedEnergy), SchedMD (2025)

Join us

Help cut AI's impact on the planet

Get new guides, funding calls and directory listings by email, about once a month. Or add your product, course, PhD or research to the directories.

No tracking. Unsubscribe in one click. Other ways to join · Privacy