# SuperGLUE

> Built to leave headroom, because GLUE had just run out of it: the BERT baseline opened at 69.0 against a human score of 89.8, a gap its authors called substantial. It lasted twenty months. DeBERTa crossed the human line in January 2021, and the board has barely moved since.

2019-05-02 — 2021-01-06

A benchmark gets no deprecation notice, so this date is this archive's editorial judgment rather than an announcement.

## The record

- Cause of death: Saturated
- Frontier at launch: 69%
- Frontier at death: 90.3%
- Practical ceiling: 91.3% (best posted — Vega)

## Sources

- https://arxiv.org/abs/1905.00537
- https://www.microsoft.com/en-us/research/blog/microsoft-deberta-surpasses-human-performance-on-the-superglue-benchmark/
- https://arxiv.org/abs/2212.01853

## Elsewhere

- [Stand at this grave](https://models.rip/superglue)
- [The benchmarks plot](https://models.rip/benchmarks)
- [The whole cemetery](https://models.rip/)