What is Temperature?
In generative AI, temperature is a numerical parameter (usually ranging from 0.0 to 1.0 or 2.0) that influences how a model samples probabilities when generating text. It determines how strictly the model adheres to the most statistically likely next word versus selecting less common options.
How does it work?
A language model assigns a probability to every possible next token. If the temperature is set very low (e.g., 0.0), the model is deterministic: it will almost always choose the token with the highest probability. If the temperature is raised (e.g., 0.8), the mathematical distribution is flattened, giving lower-probability tokens a higher chance of being selected.
What is it commonly confused with?
Temperature is frequently described merely as a "creativity slider." This is an inaccurate shortcut. High temperature does not magically make the AI think more creatively; it simply increases statistical variability. High temperature can make text sound more natural and diverse, but it also significantly increases the risk of hallucinations or logical derailment.
Why does it matter?
Setting the correct temperature is crucial for the task at hand. If you want an AI to write a marketing poem, a high temperature ensures diverse vocabulary. If you want an AI to extract JSON data from a medical document, a temperature of 0.0 is essential to ensure predictable, mathematically rigid outputs.