greenai.guide

Home › How do I…? › How do I limit how much an LLM writes?

How do I…?

How do I limit how much an LLM writes?

Updated · 1 min read

In short

Set a maximum output length in the API call, tell the model the format and length you want in the prompt, and use stop sequences or structured output so it ends when the job is done. Energy grows with the number of tokens a model generates, so capping output is one of the simplest savings in any application.

Steps

  1. Decide the longest useful answer for each feature and set the API's maximum output tokens to match.
  2. State the format and length in the prompt, such as "three bullet points".
  3. Use structured output, such as a JSON schema, when your code only needs fields.
  4. Add stop sequences where the answer has a natural end marker.
  5. Detect answers cut off by the limit and handle them rather than silently retrying.

Tell the model, and the API, how much you want. Models generate one token at a time, and each one costs computation.

Why does it matter?

Delavande, Pierrard and Luccioni found that energy grows linearly with generation length, and that larger models not only use more energy per token but also tend to write longer replies. UNESCO and University College London found that shorter prompts and responses could cut energy use by more than half.

Which controls should I use?

  • Maximum output tokens. Every major API has a parameter for this. Set it per feature, not one generous global value.
  • Instructions. "Answer in one sentence" or "return only the category name" work better than a hard limit alone, because the model plans a short answer instead of being cut off.
  • Structured output. If your code parses fields, ask for JSON against a schema and skip the prose.
  • Stop sequences. End generation at a marker such as a closing tag.
  • Reasoning effort. Where a model offers a thinking or reasoning setting, use the lowest level that passes your tests: see when to use reasoning modes.

Common mistakes

  • Limits so tight answers get cut off, followed by an automatic retry with a higher limit. That costs more than getting the length right.
  • Asking for explanations nobody reads, such as "explain your reasoning" in a classification pipeline.
  • Not logging output tokens per feature, so the wordiest features stay hidden.

For user-facing habits, read use AI efficiently.

Sources

  1. Small talk, big impact: the energy cost of thanking AI, Delavande, Pierrard and Luccioni (arXiv 2601.22357) (2026)
  2. Smarter, smaller, stronger: resource-efficient generative AI and the future of digital transformation, UNESCO and University College London (2025)
  3. What are tokens and how to count them?, OpenAI Help Center (2025)

Join us

Help cut AI's impact on the planet

Get new guides, funding calls and directory listings by email, about once a month. Or add your product, course, PhD or research to the directories.

No tracking. Unsubscribe in one click. Other ways to join · Privacy