In short
Use a standard benchmark where one covers your task, such as Hugging Face's AI Energy Score, and otherwise measure the candidates yourself on the same task, hardware and settings. Compare energy per successful task alongside quality, because a cheap model that fails costs more in the end.
Steps
- Check whether your candidate models already have an AI Energy Score rating for your task.
- For models without one, run the same benchmark on each, on the same GPU and settings.
- Use a fixed set of real prompts with realistic input and output lengths.
- Record energy per request and quality for each model, measured the same way.
- Compare cost per successful task, including retries.
Compare like with like: same task, same hardware, same settings, same measurement. Published figures from different providers use different methods and boundaries, so they rarely compare fairly.
Is there a ready-made comparison?
For many open models, yes. Hugging Face's AI Energy Score measures GPU energy per 1,000 queries for ten common tasks and gives one to five stars relative to other models on the same task. It also provides a way to benchmark private models with the same method. Start there.
How do I measure models myself?
- Models you run: wrap each run in a tool such as CodeCarbon, which estimates energy from processor and memory power.
- Hosted APIs: estimate from token counts with a tool such as EcoLogits, and label the result as an estimate.
- Keep everything else fixed: prompts, batch size, input and output length, hardware and region.
- Repeat runs and report a range, not one number.
The differences can be large. In Luccioni, Jernite and Strubell's comparison, multi-purpose generative models were orders of magnitude more expensive than task-specific systems for the same tasks.
Common mistakes
- Comparing per call instead of per success. Count retries and failures.
- Ignoring output length. A model that writes twice as much can cost twice as much, whatever its size.
- Mixing estimated and metered figures in the same table without saying so.
For the method in full, see how to measure AI energy, and for reading ratings, see read an AI Energy Score rating.
Sources
- AI Energy Score, Hugging Face (2025)
- Power Hungry Processing: Watts Driving the Cost of AI Deployment?, Luccioni, Jernite and Strubell, ACM FAccT (2024)
- CodeCarbon, CodeCarbon (2026)
- EcoLogits documentation, EcoLogits (2025)
Related
How do I read an AI Energy Score rating?
What the AI Energy Score's one-to-five-star ratings mean, what they measure, how to use them to compare models and what they don't tell you.
How do I measure the energy AI uses on my own computer or GPU?
Measure local AI energy with hardware counters, GPU telemetry or a plug-in meter, sample over the whole job, and subtract the idle baseline.
How do I estimate the energy of my AI API calls?
You can't meter a hosted AI API, but you can estimate energy per call from token counts and model assumptions with EcoLogits, and log it next to cost.
How to measure the energy and carbon of AI
A practical guide to metering or estimating the energy and carbon of AI work: tools, system boundaries, per-task metrics and honest uncertainty.