In short
Define what "good enough" means on real examples, then test models from smallest to largest and stop at the first that passes. Small and task-specific models can use a fraction of the energy of large general ones: UNESCO and University College London report savings of up to 90% on suitable tasks.
Steps
- Write down the task and a pass mark, such as the share of answers that must be correct.
- Collect 20 to 50 real examples to test against.
- Try the smallest or "mini" model first, then move up only if it fails the pass mark.
- Check public energy ratings such as the AI Energy Score where they exist.
- Keep a larger model for the cases the small one can't handle.
- Re-test when models or your task change.
Use the smallest model that meets your quality bar, not the most capable one available. A large general model pays, on every request, for abilities your task may never use.
Why does model size matter so much?
It is usually the biggest single lever on energy. Luccioni, Jernite and Strubell found multi-purpose generative models to be orders of magnitude more expensive to run than task-specific systems for the same tasks. UNESCO and University College London found that small models tailored to specific tasks could cut energy use by up to 90%.
How do I test candidates?
- Define the bar on your own work, for example "extracts the invoice total correctly 98% of the time".
- Test small first. Many providers offer "mini", "lite" or "flash" versions; open-weight families come in several sizes.
- Stop at the first pass. Bigger is not automatically more accurate for narrow tasks.
- Compare energy, not just price. Hugging Face's AI Energy Score rates open models from one to five stars on energy per query for common tasks; see read an AI Energy Score rating.
For everyday users, the same idea applies: pick the fast or lighter option in your chatbot for simple questions.
Common mistakes
- Choosing on reputation rather than testing on your own examples.
- Measuring cost per call, not per success. A model that is too small and fails, forcing a rerun on a big one, can cost more overall.
- Never revisiting. Small models improve quickly; re-test every few months.
To send each request to the right size automatically, see route requests to smaller models. The full method is in choose the right model.
Sources
- Smarter, smaller, stronger: resource-efficient generative AI and the future of digital transformation, UNESCO and University College London (2025)
- Power Hungry Processing: Watts Driving the Cost of AI Deployment?, Luccioni, Jernite and Strubell, ACM FAccT (2024)
- AI Energy Score, Hugging Face (2025)
Related
Choose the right model for the job
How to right-size AI: when to use small or specialised models, when a frontier model earns its cost, how routing works, and when not to use AI.
How do I find a task-specific model instead of using a general chatbot?
How to find a small model built for one job, such as classifying, extracting or translating, and check it matches a general chatbot for less energy.
How do I route requests to smaller models?
How to route routine requests to a small model and only hard ones to a large model, and how to tune the router so retries don't eat the saving.
How do I compare the energy use of different AI models?
How to compare AI models on energy fairly: use benchmarks like the AI Energy Score, or measure them yourself on the same task, hardware and settings.