ARC-AGI

What it measures

Colored-grid abstraction tasks that test few-shot rule induction, not memorized facts, language fluency, or school math.

Primary metric

task success rate on the evaluation split (%)

Judge model

not judged by a model

Human baseline

98The official ARC Prize leaderboard lists a Human Panel score of 98% on the ARC-AGI-1 semi-private axis. Older public-page summaries also report lower average-human estimates; the sidebar uses the current official leaderboard value.

Languages

Language-neutral

Health

The public evaluation split has 400 tasks, but ARC Prize leaderboard scores use the semi-private axis and competitions may use private splits. Treat scores as comparable only when the split, attempt budget, model/system type, and cost budget are named.

Status evidence

ARC-AGI-1 remains useful as a historical abstraction benchmark, but the official ARC Prize leaderboard generated on 2026-07-23 lists multiple verified systems above 95% on the ARC-AGI-1 semi-private axis, including GPT-5.6 Sol at 97.5%. ARC Prize launched ARC-AGI-2 because ARC-AGI-1 had become too brute-forceable and less discriminative for frontier systems.

Saturated since

2024-12-20

Refresh cycle

static

Test set public

yes

The public train and evaluation tasks are fully available on GitHub. ARC Prize later introduced semi-private and private evaluation sets for leaderboard and competition testing, but ARC-AGI-1 still has a fixed, old private pool and a long history of submissions. Contamination risk is therefore high for public-set scores and medium-to-high even for public leaderboard comparisons unless the split and testing policy are clear.

History & lineage

Hand-maintained, not part of the benchwiki record above — each line cites the paper or leaderboard it comes from.

1. 2019-11ARC-AGI-1 introduces static colored-grid abstraction tasks.
2. 2025-03ARC-AGI-2 keeps the static grid format but raises difficulty and improves human calibration and private-set controls.
3. 2026-04ARC-AGI-3 appears as arXiv:2603.24621v2, introducing interactive turn-based environments and RHAE scoring.
4. 2026-07The official leaderboard generated on 2026-07-23 lists GPT-5.6 Sol (Max) at 7.78% on the ARC-AGI-3 semi-private axis, with substantially higher cost than static ARC runs.

Further reading

• ARC-AGI-3 overview
• ARC-AGI-3 paper
• ARC-AGI-3 HTML paper
• ARC Prize verified leaderboard
• ARC-AGI-2 paper

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

Trajectory

Best score measured on this site

As benchwiki records it

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• ARC Prize leaderboardofficial
• ARC Prize 2024 competitionofficial