What is LLM temperature?
Temperature sets how random an LLM's next-token choice is. Lower is more predictable. Many current reasoning models no longer let you set it.
LLM temperature is a sampling setting that controls how random the model’s choice of each next token is. Lower values favor the most probable tokens, and higher values make less probable tokens more likely to be picked (IBM). It changes how the model picks from its predictions, not what it knows.
On a growing number of current models you can no longer set it at all. The provider sections below show which ones, as of November 2026.
What temperature does
At each step a model scores every possible next token. Temperature is applied during sampling, together with top-p and top-k (Google). Google describes lower temperatures as good for prompts that need a more deterministic, less open-ended response, and higher ones as giving more diverse or creative results. At a temperature of 0 the highest-probability token is always selected.
Temperature and top-p
Top-p, also called nucleus sampling, limits sampling to the smallest set of tokens whose cumulative probability passes a threshold (IBM). It comes from a 2019 paper by Holtzman and colleagues, which samples from the “dynamic nucleus” of the distribution to cut off the unreliable tail of unlikely words. Top-k instead keeps only the k most likely tokens (IBM, same page).
Both reshape the same choice, so they overlap. OpenAI’s reference gives temperature a range of 0 to 2 and top_p a range of 0 to 1, both defaulting to 1, and says: “We generally recommend altering this or top_p but not both.”
Does temperature control creativity?
Less than the folk advice says. A 2024 study by Peeperkorn and colleagues found temperature weakly correlated with novelty, with outputs only slightly more novel at higher temperatures, and no relationship to cohesion or typicality. IBM makes the same caution and notes that a higher temperature can raise incoherence.
Does it change accuracy?
On the tasks tested, not much. Renze and Guven (2024) report that changing temperature from 0.0 to 1.0 had no statistically significant impact on LLM performance across the models, prompting techniques and problem domains they tested. Treat that as a reason to test your own task before tuning, for instance with LLM evaluation.
Temperature 0 is not a guarantee
Setting temperature to 0 selects the top token (Google), yet Horace He at Thinking Machines Lab (10 September 2025) writes that LLM APIs are “still not deterministic in practice” at temperature 0. He argues the usual “concurrency + floating point” explanation does not tell the full story. The cause he names is a lack of batch invariance: kernel outputs can change with batch size, and batch size varies with server load.
Can you still set it? Provider status
This is changing quickly, so check your model’s docs.
| Provider | Status |
|---|---|
| OpenAI | Temperature accepts 0 to 2, default 1 (API reference). For the GPT-6 family, remove temperature and top_p when reasoning effort is not none (migration guide). |
| Anthropic | API version 2026-10-21 returns a 400 error for temperature, top_p or top_k, even at default values. Claude 4.7 and later accept only defaults. On models that still accept other values, such as Sonnet 4.6, a request that lowered temperature for consistency loses that effect unless it stays on API version 2023-06-01 (migration guide). Claude Sonnet 5.5 also returns an error (migration guide). |
Gemini 4 and Gemini 3.6+ models ignore custom temperature, top_p and top_k, with 400 errors planned for a future release. For earlier Gemini 3 models, keep them at defaults to avoid degraded reasoning (Gemini docs). |
The practical effect: for reasoning models, temperature is no longer your consistency lever. Anthropic’s migration guide says no other setting replaces it (source). For structured output, tighter prompts and constraints take over; see what is prompt engineering and few-shot prompting.
Log the settings you used
If you do set temperature, record it with each call so a change in output can be traced to a change in settings. HoneyHive’s configuration docs describe attaching the prompt version and model parameters, including temperature and top_p, to traces. See also LLM observability.
FAQ
What is a good temperature for an LLM?
There is no single value. OpenAI's API reference allows 0 to 2 with a default of 1, and IBM suggests low values for precision tasks and high values for creative ones. For several current models the answer is to leave it alone, because the provider fixes it.
Does temperature 0 make an LLM deterministic?
Not fully. Gemini's docs say a temperature of 0 always selects the highest-probability token, but Thinking Machines Lab reports that LLM APIs are still not deterministic in practice at temperature 0, because kernel results can change with batch size.
Should I change temperature or top_p?
One at most. OpenAI generally recommends altering temperature or top_p but not both.
Does a higher temperature make an LLM more creative?
Only slightly. A 2024 study found temperature weakly related to novelty and unrelated to cohesion or typicality.