# How do I decide between running AI on my device or in the cloud?

> How to weigh on-device (edge) AI against cloud AI for energy, privacy and quality, and which kinds of task suit each, with honest caveats on the evidence.

Source: https://greenai.guide/how-do-i/choose-between-device-and-cloud-ai/  
Updated: 2026-11-19

## In short

Run small, frequent, private or latency-sensitive tasks on the device, and send large or occasional tasks that need big models to an efficient cloud service. Some studies find large energy savings on the device for suitable models, but well-run data centres are a hard benchmark, so measure for your own case where you can.

## Steps

1. List the task's needs: model size, how often it runs, privacy, and whether it must work offline.
2. Favour the device for small models, frequent use, private data and offline needs.
3. Favour the cloud for large models, occasional heavy tasks and work that can be batched.
4. Compare energy per task on both options, including the network, where you can measure it.
5. Consider battery life and hardware age as part of the device's cost.

**Small and frequent suits the device; big and occasional suits the cloud.** Neither is greener in every case. The
answer depends on the model, how often it runs, and how efficient each side is.

## What does the evidence say?

**It is mixed.** Li, Islam and Ren found a reduced footprint for edge deployment of several generative models, with
energy savings of over 90% in tests on a recent phone. But their cloud baseline was not a fully optimised production
service, and well-batched servers in an efficient data centre are harder to beat. A 2026 UK green AI sandpit of
mathematicians, computer scientists and industry listed the net energy effect of AI spread across millions of
devices as an open question.

## How do I choose?

| Favours the device | Favours the cloud |
|---|---|
| Small model, such as Apple's roughly three-billion-parameter on-device model | Large model that won't fit locally |
| Runs often, for example on every photo or keystroke | Runs occasionally |
| Private or sensitive data | Work that can be batched or scheduled |
| Must work offline or instantly | Needs the latest, most capable model |

The Green Software Foundation lists running AI models at the edge among its green software patterns, as an option
for suitable models rather than a rule.

## Common mistakes

- **Assuming local is free.** It uses your battery and may push you towards new hardware.
- **Ignoring the network.** Sending large files to the cloud has a cost too.
- **Running a model that's too big** for the device, so it's slow and drains power.

For a fuller decision guide, see [edge or cloud](/practice/edge-or-cloud/) and
[AI at the edge](/articles/ai-at-the-edge/).

## Sources

- [A case study of environmental footprints for generative AI inference: cloud versus edge](https://escholarship.org/uc/item/2kc978dg), Li, Islam and Ren, ACM SIGMETRICS Performance Evaluation Review (2025)
- [Apple Intelligence Foundation Language Models: Tech Report 2025](https://arxiv.org/abs/2507.13575), Apple (2025)
- [Green Software Patterns: run AI models at the edge](https://github.com/Green-Software-Foundation/patterns/blob/main/docs/architecture/system-topology/run-ai-models-edge.md), Green Software Foundation (2025)
