# How do I decide whether classic machine learning will do instead of an LLM?

> A quick test for data scientists: when a regression, tree model or small classifier beats a large language model on cost, energy, speed and reliability.

Source: https://greenai.guide/how-do-i/choose-classical-ml-instead-of-an-llm/  
Updated: 2026-11-19

## In short

If your inputs are structured data or short labelled text and the output is a number, a category or a ranking, try a classic model first: a linear model, gradient-boosted trees or a small classifier. They are far cheaper to run, faster, easier to explain and often as accurate. Reach for a large language model when the task genuinely needs open-ended language.

## Steps

1. Write down the input type, the output type and the quality bar.
2. Build a simple baseline first, such as a rule, a linear model or gradient-boosted trees.
3. Try a small fine-tuned classifier if the input is free text.
4. Test an LLM only if the baselines miss the bar, and on the same evaluation set.
5. Compare quality, latency and energy per prediction before choosing.

**Use the simplest model that meets your quality bar.** For structured data and narrow labelling tasks, that is
usually a classic machine learning model, not a large language model.

## Why does it matter?

Luccioni, Jernite and Strubell measured the energy of running models across many tasks. Classifying text was among
the cheapest tasks, at about 0.002 kWh per 1,000 inferences, while general generative models cost orders of
magnitude more than task-specific ones for the same task. The W3C Web Sustainability Guidelines also ask teams to
evaluate the need for AI before implementing it.

A 2026 UK green AI sandpit of mathematicians, computer scientists and industry made the same point from another
angle: people often reach for AI when a well-understood statistical or numerical method already does the job.

## A quick test

| Your task | Try first |
|---|---|
| Predict a number from tabular data | Linear model or gradient-boosted trees |
| Assign one of a few labels | Small fine-tuned classifier |
| Find similar items | Embeddings plus a nearest-neighbour index |
| Forecast a time series | Established statistical forecasting methods |
| Write, explain or converse | This is where an LLM earns its cost |

## Common mistakes

- **Skipping the baseline.** Without one, you cannot tell whether the LLM adds anything.
- **Using an LLM to label data you then never retrain on.** Use it once to create labels, then train a small model.
- **Comparing on accuracy alone.** Include latency, energy per prediction and how easy errors are to explain.

Read more in [choose the right model](/practice/choose-the-right-model/) and
[maths for efficient AI](/articles/maths-for-efficient-ai/).

## Sources

- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://arxiv.org/abs/2311.16863), Luccioni, Jernite and Strubell, ACM FAccT (2024)
- [Web Sustainability Guidelines](https://www.w3.org/TR/web-sustainability-guidelines/), W3C (2025)
