models.rip · benchmark

GSM8K

2021–2024 · 55% → 91.1% frontier score

Grade school word problems, and the paper that introduced them topped out at 55%. In 2024 someone rebuilt the test from scratch, matched for difficulty, and watched whole model families drop up to 8 points — they had not learned arithmetic, they had read the answers. The frontier came through clean.