AGIEval

What it measures

Standardized human exams — SATs, LSATs, China's Gaokao and civil-service tests — repurposed to grade models against real human test-takers.

Primary metric

accuracy, macro-averaged over the 20 tasks (%)

Judge model

not judged by a model

Human baseline

67Average human test-taker across the exam sections (~67%). The paper also reports a top-percentile ceiling of ~91% (top 1% of candidates, except the lawyer qualification test's top 10%). The gauge/timeline use the average-human figure as the reference line; the 91% top-human ceiling is the harder target the paper highlights. Both are MEASURED from real exam-taker distributions, not estimated — a key differentiator of AGIEval.

Languages

Chinese, English

Health

8,062 questions across 20 unequally-sized exam sections; because the aggregate is an unweighted task average, a small section (e.g. a few hundred cloze items) moves the headline as much as a large one. Per-section scores — not the 20-task mean — are where the remaining signal lives now that the aggregate is saturated.

Status evidence

GPT-4 already cleared the average-human line (67%) on several sections at launch; by 2024 frontier models pushed the 20-task aggregate past the average human and near the top-percentile 91% ceiling, and the human-exam framing lost its discriminative power at the top. saturated_date (2024-06) marks the GPT-4-class era where the average-human baseline stopped separating strong models.

Saturated since

2024-06-01

Refresh cycle

static

Test set public

yes

None built in. AGIEval is drawn from real, published exams (SAT, LSAT, Gaokao, Chinese civil-service and bar exams), so the exact questions and worked solutions circulate widely on test-prep sites and in web-scale training data. There is no held-out or refreshed variant, and exam questions are exactly the kind of high-value content trainers scrape. Treat post-2023 scores as memorization-inclusive.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• OpenCompass (AGIEval)third-party