greenai.guide

Home › How do I…? › How do I decide between running AI on my device or in the cloud?

How do I…?

How do I decide between running AI on my device or in the cloud?

Updated · 1 min read

In short

Run small, frequent, private or latency-sensitive tasks on the device, and send large or occasional tasks that need big models to an efficient cloud service. Some studies find large energy savings on the device for suitable models, but well-run data centres are a hard benchmark, so measure for your own case where you can.

Steps

  1. List the task's needs: model size, how often it runs, privacy, and whether it must work offline.
  2. Favour the device for small models, frequent use, private data and offline needs.
  3. Favour the cloud for large models, occasional heavy tasks and work that can be batched.
  4. Compare energy per task on both options, including the network, where you can measure it.
  5. Consider battery life and hardware age as part of the device's cost.

Small and frequent suits the device; big and occasional suits the cloud. Neither is greener in every case. The answer depends on the model, how often it runs, and how efficient each side is.

What does the evidence say?

It is mixed. Li, Islam and Ren found a reduced footprint for edge deployment of several generative models, with energy savings of over 90% in tests on a recent phone. But their cloud baseline was not a fully optimised production service, and well-batched servers in an efficient data centre are harder to beat. A 2026 UK green AI sandpit of mathematicians, computer scientists and industry listed the net energy effect of AI spread across millions of devices as an open question.

How do I choose?

Favours the device Favours the cloud
Small model, such as Apple's roughly three-billion-parameter on-device model Large model that won't fit locally
Runs often, for example on every photo or keystroke Runs occasionally
Private or sensitive data Work that can be batched or scheduled
Must work offline or instantly Needs the latest, most capable model

The Green Software Foundation lists running AI models at the edge among its green software patterns, as an option for suitable models rather than a rule.

Common mistakes

  • Assuming local is free. It uses your battery and may push you towards new hardware.
  • Ignoring the network. Sending large files to the cloud has a cost too.
  • Running a model that's too big for the device, so it's slow and drains power.

For a fuller decision guide, see edge or cloud and AI at the edge.

Sources

  1. A case study of environmental footprints for generative AI inference: cloud versus edge, Li, Islam and Ren, ACM SIGMETRICS Performance Evaluation Review (2025)
  2. Apple Intelligence Foundation Language Models: Tech Report 2025, Apple (2025)
  3. Green Software Patterns: run AI models at the edge, Green Software Foundation (2025)

Join us

Help cut AI's impact on the planet

Get new guides, funding calls and directory listings by email, about once a month. Or add your product, course, PhD or research to the directories.

No tracking. Unsubscribe in one click. Other ways to join · Privacy