greenai.guide

Home › How do I…? › How do I compare the energy use of different AI models?

How do I…?

How do I compare the energy use of different AI models?

Updated · 1 min read

In short

Use a standard benchmark where one covers your task, such as Hugging Face's AI Energy Score, and otherwise measure the candidates yourself on the same task, hardware and settings. Compare energy per successful task alongside quality, because a cheap model that fails costs more in the end.

Steps

  1. Check whether your candidate models already have an AI Energy Score rating for your task.
  2. For models without one, run the same benchmark on each, on the same GPU and settings.
  3. Use a fixed set of real prompts with realistic input and output lengths.
  4. Record energy per request and quality for each model, measured the same way.
  5. Compare cost per successful task, including retries.

Compare like with like: same task, same hardware, same settings, same measurement. Published figures from different providers use different methods and boundaries, so they rarely compare fairly.

Is there a ready-made comparison?

For many open models, yes. Hugging Face's AI Energy Score measures GPU energy per 1,000 queries for ten common tasks and gives one to five stars relative to other models on the same task. It also provides a way to benchmark private models with the same method. Start there.

How do I measure models myself?

  • Models you run: wrap each run in a tool such as CodeCarbon, which estimates energy from processor and memory power.
  • Hosted APIs: estimate from token counts with a tool such as EcoLogits, and label the result as an estimate.
  • Keep everything else fixed: prompts, batch size, input and output length, hardware and region.
  • Repeat runs and report a range, not one number.

The differences can be large. In Luccioni, Jernite and Strubell's comparison, multi-purpose generative models were orders of magnitude more expensive than task-specific systems for the same tasks.

Common mistakes

  • Comparing per call instead of per success. Count retries and failures.
  • Ignoring output length. A model that writes twice as much can cost twice as much, whatever its size.
  • Mixing estimated and metered figures in the same table without saying so.

For the method in full, see how to measure AI energy, and for reading ratings, see read an AI Energy Score rating.

Sources

  1. AI Energy Score, Hugging Face (2025)
  2. Power Hungry Processing: Watts Driving the Cost of AI Deployment?, Luccioni, Jernite and Strubell, ACM FAccT (2024)
  3. CodeCarbon, CodeCarbon (2026)
  4. EcoLogits documentation, EcoLogits (2025)

Join us

Help cut AI's impact on the planet

Get new guides, funding calls and directory listings by email, about once a month. Or add your product, course, PhD or research to the directories.

No tracking. Unsubscribe in one click. Other ways to join · Privacy