Emergent abilities, debated

A widely cited claim about large language models is that certain abilities are emergent: absent in smaller models, present in larger ones, and appearing abruptly rather than improving gradually. This lesson states the claim precisely, presents the main rebuttal, and settles on what the evidence actually supports.

1. The claim

Wei et al. (2022) defined an emergent ability as one that is "not present in smaller models but is present in larger models," such that its performance curve against scale is flat near chance until a threshold, then rises sharply[1]. Reported examples included multi-step arithmetic, word unscrambling, and certain multiple-choice benchmarks, measured by exact-match accuracy. On a log-scale x-axis of training compute, these look like sudden phase transitions.

If taken at face value, emergence is unsettling for safety: it implies new capabilities can appear without warning as models scale, so a benchmark that looks safe at one size gives no guarantee at the next.

2. The metric-artifact rebuttal

Schaeffer et al. (2023) argued that much apparent emergence is an artifact of the metric, not of the model[2]. The argument has two parts.

First, the underlying per-token quantity that scaling laws govern, cross-entropy loss, improves smoothly and predictably with scale. There is no discontinuity in the loss.

Second, the benchmark metrics that show sharp jumps are nonlinear and discontinuous functions of that smooth loss. Exact-match on a $k$-step problem requires all $k$ tokens correct, so accuracy scales roughly like $p^k$ where $p$ is per-token correctness; multiplying a smoothly rising $p$ through a steep power and a hard threshold manufactures a sharp-looking curve. Swap to a smooth metric (token edit distance, per-token log-likelihood, or a partial-credit score) and the same runs show continuous improvement.

The testable prediction follows: choosing a continuous metric should dissolve the discontinuity, and across many of the original tasks it does.

3. What this does and does not settle

The rebuttal does not say capabilities never change qualitatively. It says that abrupt benchmark jumps are weak evidence for a phase transition, because a discontinuous metric over a smooth loss produces them for free. Three honest takeaways:

• Pretraining loss scales smoothly; predict capability through metrics that track it rather than through brittle exact-match thresholds.
• Some phenomena do look like genuine internal phase changes. The formation of induction heads during training is a sharp, mechanistically identified transition in a smooth quantity, not a metric artifact.
• "Emergence" is therefore better treated as a statement about measurement and about in-distribution extrapolation than as a law of nature.

Exercises

Exercise 1. A task requires all $k = 10$ tokens correct under exact-match. Suppose per-token correctness improves smoothly with scale, taking values $p = 0.5$, $0.7$, and $0.9$ at three model sizes. Compute exact-match accuracy $p^{10}$ at each size, and use the numbers to explain why a smoothly rising $p$ produces an accuracy curve that looks flat near zero and then sharp.

Solution

Step 1. Exact-match needs every token right, so accuracy is $p^{10}$:

$$0.5^{10} \approx 0.001, \qquad 0.7^{10} \approx 0.028, \qquad 0.9^{10} \approx 0.349.$$

Step 2. Equal increments in $p$ ($0.5 \to 0.7 \to 0.9$) map to wildly unequal jumps in accuracy: roughly $0.001 \to 0.028$ (about a $29\times$ relative move off a near-zero base) then $0.028 \to 0.349$ (an absolute jump of $0.32$). The power $p^{10}$ keeps accuracy pinned near zero while $p$ is moderate, then lets it climb steeply once $p$ approaches $1$.

Step 3. Plotted against scale, that is a flat-then-sharp curve manufactured entirely by the metric. The underlying $p$ never jumps; the discontinuous-looking emergence is the power-and- threshold transform acting on a smooth quantity.

Exercise 2. Take the same task and same three values of $p$, but score it with a continuous metric: expected fraction of correct tokens, which equals $p$ itself. Tabulate this metric against the exact-match accuracy from Exercise 1, and state what the comparison shows about the emergence claim.

Solution

Step 1. The continuous metric is $p$, so its values are $0.5$, $0.7$, $0.9$, three evenly spaced points: a straight, smooth rise.

Step 2. Side by side:

metric	size 1	size 2	size 3
token fraction ($p$)	0.50	0.70	0.90
exact-match ($p^{10}$)	0.001	0.028	0.349

Step 3. The same runs look smooth under the linear metric and abrupt under exact-match. The discontinuity lives in the choice of metric, not in the model, which is the core of the metric-artifact rebuttal: swapping to a metric that tracks the smooth loss dissolves the apparent jump.

Exercise 3. The lesson cites the abrupt formation of induction heads during training as a phenomenon that is not a metric artifact. Explain what distinguishes that case from the exact-match curves above, and why this matters for whether "emergence" should be treated as a real phase transition or a measurement effect. (Conceptual; no computation.)

Solution

Step 1. The exact-match jumps arise from a discontinuous metric applied to a smooth internal quantity (the loss): change the metric and the jump disappears. The phenomenon is in the ruler, not the model.

Step 2. The induction-head transition is a change in the internal mechanism itself: a specific circuit forms over a narrow window of training, visible as a bump in the smooth loss and confirmed by direct mechanistic inspection, not by a brittle threshold metric. Smoothing the metric does not remove it.

Step 3. So "emergence" splits into two claims. A genuine internal phase change (a circuit appearing) is a real transition; an abrupt benchmark number is weak evidence on its own because a discontinuous metric over a smooth loss produces it for free. The right posture is to demand mechanistic or smooth-metric evidence before calling a benchmark jump a phase transition.

The practical posture is the same one scaling laws recommend: forecast with smooth predictors, and treat surprising benchmark jumps as a prompt to check the metric before concluding the model crossed a threshold. The final lesson surveys where the frontier is now, including the reasoning models that complicate this picture further.

---

References

1. [1]Wei, J., et al. (2022). "Emergent abilities of large language models". TMLR. arXiv:2206.07682. States the emergence claim and its examples.↩
2. [2]Schaeffer, R., Miranda, B., & Koyejo, S. (2023). "Are emergent abilities of large language models a mirage?". NeurIPS. arXiv:2304.15004. The metric-artifact rebuttal.↩