Scaling Laws
Also called Neural Scaling Laws
Observed patterns showing that an AI model's error falls predictably as you increase its size, training data and training compute.
What are scaling laws?
Scaling laws are empirical relationships that describe how a model's performance changes as you scale up its ingredients: the number of parameters, the amount of training data and the compute used for training. The best-known early study, by Kaplan and colleagues in 2020, found that a language model's loss follows smooth power-law trends with each of these.
How does it work?
Researchers train many models at different sizes and plot the results. The curves are regular enough to predict how a larger model will perform before it is built. That lets labs decide how to split a fixed budget between model size and data. Later work refined how that split should be made.
What is a simple example?
If doubling compute lowers a model's error by a predictable amount, a lab can estimate whether a much more expensive training run is worth it before starting it.
Why does it matter?
Scaling laws are a main reason labs have trained ever larger frontier models. They are observations, not guarantees: they describe past trends, and whether they continue is debated.