In short
If your inputs are structured data or short labelled text and the output is a number, a category or a ranking, try a classic model first: a linear model, gradient-boosted trees or a small classifier. They are far cheaper to run, faster, easier to explain and often as accurate. Reach for a large language model when the task genuinely needs open-ended language.
Steps
- Write down the input type, the output type and the quality bar.
- Build a simple baseline first, such as a rule, a linear model or gradient-boosted trees.
- Try a small fine-tuned classifier if the input is free text.
- Test an LLM only if the baselines miss the bar, and on the same evaluation set.
- Compare quality, latency and energy per prediction before choosing.
Use the simplest model that meets your quality bar. For structured data and narrow labelling tasks, that is usually a classic machine learning model, not a large language model.
Why does it matter?
Luccioni, Jernite and Strubell measured the energy of running models across many tasks. Classifying text was among the cheapest tasks, at about 0.002 kWh per 1,000 inferences, while general generative models cost orders of magnitude more than task-specific ones for the same task. The W3C Web Sustainability Guidelines also ask teams to evaluate the need for AI before implementing it.
A 2026 UK green AI sandpit of mathematicians, computer scientists and industry made the same point from another angle: people often reach for AI when a well-understood statistical or numerical method already does the job.
A quick test
| Your task | Try first |
|---|---|
| Predict a number from tabular data | Linear model or gradient-boosted trees |
| Assign one of a few labels | Small fine-tuned classifier |
| Find similar items | Embeddings plus a nearest-neighbour index |
| Forecast a time series | Established statistical forecasting methods |
| Write, explain or converse | This is where an LLM earns its cost |
Common mistakes
- Skipping the baseline. Without one, you cannot tell whether the LLM adds anything.
- Using an LLM to label data you then never retrain on. Use it once to create labels, then train a small model.
- Comparing on accuracy alone. Include latency, energy per prediction and how easy errors are to explain.
Read more in choose the right model and maths for efficient AI.
Sources
- Power Hungry Processing: Watts Driving the Cost of AI Deployment?, Luccioni, Jernite and Strubell, ACM FAccT (2024)
- Web Sustainability Guidelines, W3C (2025)
Related
How do I decide whether I need AI for a task at all?
A quick test for whether a task needs generative AI or whether search, a calculator, a template or a simple rule would do it faster and with less energy.
How do I find a task-specific model instead of using a general chatbot?
How to find a small model built for one job, such as classifying, extracting or translating, and check it matches a general chatbot for less energy.
Choose the right model for the job
How to right-size AI: when to use small or specialised models, when a frontier model earns its cost, how routing works, and when not to use AI.
The maths that makes AI more efficient
Pruning, sparsity, sketching, random matrix theory and physics-informed models: how mathematics cuts the computation, memory and energy AI needs.