Cost
Temperature is not its own charge. The bill is still the tokens the request reads and writes. You pay more only if the setting makes the reply longer or makes you send the request again.
Map · Foundations · 10 · Temperature
Concept card · Foundations
Temperature
Temperature is a sampling control that shifts how often less likely next tokens are picked.
A sampling control that shifts how often less likely next tokens are picked.
What changed
Oct 18: A note on top-p, from the OpenAI Chat reference. Oct 17: First card. The picture uses three fixed scores and is not a provider sampler. Sources are the OpenAI Chat reference, the Anthropic Messages API, and Gemini's prompting strategies.
1 What it is
Temperature is a number a request can send with the prompt. It changes how the next token is picked. Lower makes the leading candidate more dominant. Higher gives less likely candidates more of a chance. The allowed range is not the same on every model, and some current models ignore a custom value or reject it. Top-p is a different sampling control. The OpenAI Chat API names it top_p and calls it nucleus sampling: the model considers the tokens that make up the top_p probability mass. That page says 0.1 means only the tokens in the top 10% of that mass are considered, and that you should change temperature or top_p, but not both.
2 Why it matters
Quality
Lower keeps the reply closer to the leading candidate, which often fits a fixed shape. Higher lets less likely tokens through, which varies the wording. Gemini's prompting guide says earlier Gemini 3 models should keep the default, and that Gemini 4 and Gemini 3.6+ ignore a custom temperature.
Risk
A low temperature can still produce a confident wrong sentence. Anthropic's Messages API says that even 0.0 is not fully deterministic on models that accept it. OpenAI says to change temperature or top_p, not both.
3 How to apply · try today
Take a prompt you already send. If the model accepts temperature, run it once at the default and once lower, and change nothing else. See whether the reply stays in the shape you asked for.
Common mistake
Using a low temperature as a fact check, or expecting one number to mean the same thing on every model.
Self-check
Which of these is temperature?
Coming soon. Run one prompt at two temperatures and keep the reply that still matches the shape.
Coming soon. How a provider rescales token scores before a token is picked, and why a model can ignore the setting.
Questions
- What is temperature?
- Temperature is a sampling control that shifts how often less likely next tokens are picked.
- Does a lower temperature check that a reply is true?
- No. It changes which next token is more likely. It does not add a source, and a low setting is not a promise that the reply stays the same.
- What is top-p?
- OpenAI's Chat API calls top_p nucleus sampling. It is another sampling control: the model considers the tokens that make up the top_p probability mass. The same page says to change temperature or top_p, but not both.
- Do all current models accept a custom temperature?
- No. OpenAI documents a range from 0 to 2. Anthropic documents 0.0 to 1.0 on models that still accept it, and says models released after Claude Opus 4.6 reject any value other than 1.0. Gemini's prompting guide says Gemini 4 and Gemini 3.6+ ignore a custom temperature.