greenai.guide

Home › How do I…? › How do I decide whether classic machine learning will do instead of an LLM?

How do I…?

How do I decide whether classic machine learning will do instead of an LLM?

Updated · 1 min read

In short

If your inputs are structured data or short labelled text and the output is a number, a category or a ranking, try a classic model first: a linear model, gradient-boosted trees or a small classifier. They are far cheaper to run, faster, easier to explain and often as accurate. Reach for a large language model when the task genuinely needs open-ended language.

Steps

  1. Write down the input type, the output type and the quality bar.
  2. Build a simple baseline first, such as a rule, a linear model or gradient-boosted trees.
  3. Try a small fine-tuned classifier if the input is free text.
  4. Test an LLM only if the baselines miss the bar, and on the same evaluation set.
  5. Compare quality, latency and energy per prediction before choosing.

Use the simplest model that meets your quality bar. For structured data and narrow labelling tasks, that is usually a classic machine learning model, not a large language model.

Why does it matter?

Luccioni, Jernite and Strubell measured the energy of running models across many tasks. Classifying text was among the cheapest tasks, at about 0.002 kWh per 1,000 inferences, while general generative models cost orders of magnitude more than task-specific ones for the same task. The W3C Web Sustainability Guidelines also ask teams to evaluate the need for AI before implementing it.

A 2026 UK green AI sandpit of mathematicians, computer scientists and industry made the same point from another angle: people often reach for AI when a well-understood statistical or numerical method already does the job.

A quick test

Your task Try first
Predict a number from tabular data Linear model or gradient-boosted trees
Assign one of a few labels Small fine-tuned classifier
Find similar items Embeddings plus a nearest-neighbour index
Forecast a time series Established statistical forecasting methods
Write, explain or converse This is where an LLM earns its cost

Common mistakes

  • Skipping the baseline. Without one, you cannot tell whether the LLM adds anything.
  • Using an LLM to label data you then never retrain on. Use it once to create labels, then train a small model.
  • Comparing on accuracy alone. Include latency, energy per prediction and how easy errors are to explain.

Read more in choose the right model and maths for efficient AI.

Sources

  1. Power Hungry Processing: Watts Driving the Cost of AI Deployment?, Luccioni, Jernite and Strubell, ACM FAccT (2024)
  2. Web Sustainability Guidelines, W3C (2025)

Join us

Help cut AI's impact on the planet

Get new guides, funding calls and directory listings by email, about once a month. Or add your product, course, PhD or research to the directories.

No tracking. Unsubscribe in one click. Other ways to join · Privacy