In short
Set a maximum output length in the API call, tell the model the format and length you want in the prompt, and use stop sequences or structured output so it ends when the job is done. Energy grows with the number of tokens a model generates, so capping output is one of the simplest savings in any application.
Steps
- Decide the longest useful answer for each feature and set the API's maximum output tokens to match.
- State the format and length in the prompt, such as "three bullet points".
- Use structured output, such as a JSON schema, when your code only needs fields.
- Add stop sequences where the answer has a natural end marker.
- Detect answers cut off by the limit and handle them rather than silently retrying.
Tell the model, and the API, how much you want. Models generate one token at a time, and each one costs computation.
Why does it matter?
Delavande, Pierrard and Luccioni found that energy grows linearly with generation length, and that larger models not only use more energy per token but also tend to write longer replies. UNESCO and University College London found that shorter prompts and responses could cut energy use by more than half.
Which controls should I use?
- Maximum output tokens. Every major API has a parameter for this. Set it per feature, not one generous global value.
- Instructions. "Answer in one sentence" or "return only the category name" work better than a hard limit alone, because the model plans a short answer instead of being cut off.
- Structured output. If your code parses fields, ask for JSON against a schema and skip the prose.
- Stop sequences. End generation at a marker such as a closing tag.
- Reasoning effort. Where a model offers a thinking or reasoning setting, use the lowest level that passes your tests: see when to use reasoning modes.
Common mistakes
- Limits so tight answers get cut off, followed by an automatic retry with a higher limit. That costs more than getting the length right.
- Asking for explanations nobody reads, such as "explain your reasoning" in a classification pipeline.
- Not logging output tokens per feature, so the wordiest features stay hidden.
For user-facing habits, read use AI efficiently.
Sources
- Small talk, big impact: the energy cost of thanking AI, Delavande, Pierrard and Luccioni (arXiv 2601.22357) (2026)
- Smarter, smaller, stronger: resource-efficient generative AI and the future of digital transformation, UNESCO and University College London (2025)
- What are tokens and how to count them?, OpenAI Help Center (2025)
Related
How do I write prompts that use less energy?
How to write AI prompts that need less computation: be specific, set a length, paste only what matters and get it right first time instead of retrying.
How do I stop my chatbot resending the whole conversation?
How to keep chatbot context small: cap past turns, summarise older messages, retrieve what's relevant and cache fixed prompts, so input stops growing.
How do I know when to use an AI "thinking" or reasoning mode?
When an AI reasoning or 'thinking' mode is worth it, when a standard model will do, and why reasoning can use many times more energy per answer.
Use AI efficiently: everyday habits
Everyday habits that cut the energy of using AI: clear prompts, fewer regeneration loops, care with images and video, caching and turning off defaults.