Reasoning effort

Build agents
HoneyHive Engineering

Reasoning effort sets how much a model thinks before it answers. What OpenAI, Anthropic and Google offer, what it trades, and how to pick a level.

Reasoning effort is a request setting that tells a model how much to think before it answers. Low effort gets a faster, cheaper reply, and high effort lets the model spend more tokens on harder problems. OpenAI describes it the same way: lower effort favors speed and lower token usage, higher effort produces more complete thinking and higher-quality responses at the cost of more tokens.

Each provider names and scopes the setting differently, and the right level is something you measure on your own task.

How the providers expose it

Provider Setting Levels What it controls
OpenAI reasoning.effort (guide) Model-dependent: none, minimal, low, medium, high, xhigh, max Reasoning before the answer. Defaults vary by model.
Anthropic output_config.effort (docs) low, medium, high, xhigh, max All output tokens: text, tool calls and thinking. Most models default to high.
Google Gemini thinking_level (docs) minimal, low, medium, high, xhigh, depending on model Depth of thinking. Defaults vary by model.

Not every model accepts every level. OpenAI’s guide says supported values are model-dependent, Anthropic notes that not every model that supports max supports xhigh, and Google’s docs say minimal is not supported on Gemini 4 models. Check the page for your model before you hardcode a value.

Defaults differ too. Google’s docs list Gemini 4 Neon and Hydrogen defaulting to high and Carbon and Argon to medium. Anthropic’s docs say Opus 5.5 and Haiku 5.5 default to medium while the other supported models default to high. A request that omits the setting can therefore run at a different level after a model swap.

What it trades

Cost. Thinking is output you pay for. OpenAI says reasoning tokens are not visible through the API but still occupy context window space and are billed as output tokens. Google’s docs say thinking tokens are billed on the full thoughts even though only summaries are returned.

Latency. More thinking means more tokens generated before the answer is complete. Google warns that xhigh adds latency and tokens, can cause overthinking, and can exhaust max_output_tokens on simple prompts such as text or JSON extraction. See time to first token for how to measure the wait.

Quality. Gains flatten at the top. Anthropic’s docs say that on Opus 4.7, max adds significant cost for relatively small quality gains and on some structured-output or less intelligence-sensitive tasks can lead to overthinking. For Fable 5.2 they say max runs slower and costs more than xhigh, often for only a small gain.

Tool use. Anthropic’s effort setting reaches beyond thinking: lower levels tend to make fewer, combined tool calls and terser messages, and higher levels tend to make more calls and explain the plan first. For agents this changes behavior, not just depth of thought.

Truncation. Output caps include thinking. OpenAI says max_output_tokens covers reasoning tokens too, so a response can end as incomplete with no visible output while you still pay for input and reasoning tokens. It recommends reserving at least 25,000 tokens for reasoning and output when starting out. Anthropic makes the same point about max_tokens and says to set a large value at higher levels.

Effort is not a token budget

Anthropic calls effort a behavioral signal, not a strict token budget: at lower levels the model still thinks on difficult problems, just less than it would at higher levels. If you need a hard ceiling on spend, use the output cap and watch the token counts in your cost tracking.

Does a higher level always cost more per result?

Not necessarily. Artificial Analysis reported in November 2026 that at high effort, Gemini 4 Neon matches Carbon at high effort for less than half the cost per task, using Carbon’s standard pricing. The comparison is between two models, so it shows that a newer model at the same effort setting can be cheaper for the same score. It does not show that raising effort is cheaper. Cost per task depends on the model’s price and on how many tokens it spends, so compare both on your workload. The Artificial Analysis Intelligence Index explains the score they use.

How to choose a level

The providers give consistent advice:

  1. Start in the middle. OpenAI suggests evaluating both medium and high depending on the task’s complexity, and Google recommends high as the start for most complex tasks.
  2. Step up only with evidence. OpenAI says to reserve xhigh for cases where your evals show a clear benefit that justifies the extra latency and cost. Anthropic’s guidance for several models says the same about xhigh and max.
  3. Step down where quality holds. Anthropic’s docs recommend low for simpler tasks that need the best speed and lowest cost, such as subagents, and note that on Haiku 5.5, long agent prompts are more likely to skip a search or a check at low.
  4. Do not use effort to patch bad prompts. OpenAI calls it a tuning knob, not the primary way to recover quality. See context engineering and chain-of-thought prompting for what to try first.
  5. Match the level to the step. A cheap level for classification or routing and a higher one for hard planning is a pattern model routing covers.

How to test it

Run the same evaluation set at each level and compare quality, tokens and latency side by side.

  • Google’s guide says to evaluate at all four levels with a high max_output_tokens, then pick the lowest thinking_level that meets your quality bar. Setting a high cap keeps truncation from distorting the comparison.
  • Anthropic’s docs tell you to run a fresh effort sweep when you move models, because levels are calibrated differently, as they say of Sonnet 5.5 against Sonnet 5.
  • Score with a fixed evaluation and rerun it on model or setting changes as a regression test.
  • If you cache prompts, hold effort constant within a conversation. Anthropic warns that changing the top-level value between requests can invalidate the prompt cache, and on supported models a per-message change preserves it. See prompt caching.
  • On models that fix sampling settings, effort is one of the few knobs left. See LLM temperature.

Record the setting on every call

Log the effort level, the model and the thinking token counts with each request, so a jump in cost or latency can be traced to a settings change. HoneyHive’s configuration docs describe attaching model parameters to traces. See also LLM observability.

FAQ

What is reasoning effort?

A request setting that tells a model how much to think before answering. OpenAI's guide says lower effort favors speed and lower token usage, while higher effort gives more complete thinking at the cost of more tokens.

What reasoning effort should I use?

Start at your model's default or at high for complex tasks, then test one level down. Google's guide says to evaluate the levels and pick the lowest one that meets your quality bar.

Does higher reasoning effort cost more?

Usually, yes. OpenAI says reasoning tokens are billed as output tokens even though you cannot see them, and Google's guide says thinking tokens are billed too.

Is reasoning effort a token limit?

No. Anthropic describes effort as a behavioral signal, not a strict token budget. The hard cap is a separate parameter such as max_tokens.

Share