# GSM8K

> Grade school word problems, and the paper that introduced them topped out at 55%. In 2024 someone rebuilt the test from scratch, matched for difficulty, and watched whole model families drop up to 8 points — they had not learned arithmetic, they had read the answers. The frontier came through clean.

2021-10-27 — 2024-05-01

A benchmark gets no deprecation notice, so this date is this archive's editorial judgment rather than an announcement.

## The record

- Cause of death: Contaminated
- Frontier at launch: 55%
- Frontier at death: 91.1%
- Practical ceiling: 98% (breaking errors or ambiguities)

## Sources

- https://arxiv.org/abs/2110.14168
- https://arxiv.org/abs/2201.11903
- https://arxiv.org/abs/2405.00332

## Elsewhere

- [Stand at this grave](https://models.rip/gsm8k)
- [The benchmarks plot](https://models.rip/benchmarks)
- [The whole cemetery](https://models.rip/)