In short
Run small, frequent, private or latency-sensitive tasks on the device, and send large or occasional tasks that need big models to an efficient cloud service. Some studies find large energy savings on the device for suitable models, but well-run data centres are a hard benchmark, so measure for your own case where you can.
Steps
- List the task's needs: model size, how often it runs, privacy, and whether it must work offline.
- Favour the device for small models, frequent use, private data and offline needs.
- Favour the cloud for large models, occasional heavy tasks and work that can be batched.
- Compare energy per task on both options, including the network, where you can measure it.
- Consider battery life and hardware age as part of the device's cost.
Small and frequent suits the device; big and occasional suits the cloud. Neither is greener in every case. The answer depends on the model, how often it runs, and how efficient each side is.
What does the evidence say?
It is mixed. Li, Islam and Ren found a reduced footprint for edge deployment of several generative models, with energy savings of over 90% in tests on a recent phone. But their cloud baseline was not a fully optimised production service, and well-batched servers in an efficient data centre are harder to beat. A 2026 UK green AI sandpit of mathematicians, computer scientists and industry listed the net energy effect of AI spread across millions of devices as an open question.
How do I choose?
| Favours the device | Favours the cloud |
|---|---|
| Small model, such as Apple's roughly three-billion-parameter on-device model | Large model that won't fit locally |
| Runs often, for example on every photo or keystroke | Runs occasionally |
| Private or sensitive data | Work that can be batched or scheduled |
| Must work offline or instantly | Needs the latest, most capable model |
The Green Software Foundation lists running AI models at the edge among its green software patterns, as an option for suitable models rather than a rule.
Common mistakes
- Assuming local is free. It uses your battery and may push you towards new hardware.
- Ignoring the network. Sending large files to the cloud has a cost too.
- Running a model that's too big for the device, so it's slow and drains power.
For a fuller decision guide, see edge or cloud and AI at the edge.
Sources
- A case study of environmental footprints for generative AI inference: cloud versus edge, Li, Islam and Ren, ACM SIGMETRICS Performance Evaluation Review (2025)
- Apple Intelligence Foundation Language Models: Tech Report 2025, Apple (2025)
- Green Software Patterns: run AI models at the edge, Green Software Foundation (2025)
Related
Edge or cloud: where should AI run?
How to weigh on-device and edge AI against central data centres: energy, network, hardware, privacy and resilience, with a step-by-step way to decide.
AI at the edge: greener, or just less visible?
On-device and edge AI can save network and data-centre energy, but spreads computation across billions of devices nobody measures. What we know and don't.
How do I run an AI model locally?
How to run an open AI model on your own laptop or server with tools like Ollama or llama.cpp, choose a quantised model that fits, and measure its energy.
How do I make a model small enough for an edge device?
Shrink a model for phones, sensors and embedded boards: start small, distil, quantise and prune, then test accuracy and energy on the real device.