CommonsenseQA

What it measures

Five-choice questions derived from ConceptNet relations, designed to require everyday knowledge beyond a supplied passage.

Primary metric

accuracy (%)

Judge model

not judged by a model

Human baseline

88.9Majority vote of five crowd workers on the official test set in the original paper; random choice among five answers is 20%.

Languages

English

Health

For 1,140 independent test questions, a score near 50% has an approximate 95% binomial margin of error of +/-2.9 points; near the 88.9% human reference it is about +/-1.8 points. Dev-set tuning and ensemble selection add uncertainty not captured by this interval.

Status evidence

KEAR reached 89.4% on the hidden test set versus the original 88.9% human reference in 2021; the benchmark's authors then introduced the adversarially collected CommonsenseQA 2.0.

Saturated since

2021-12-06

Refresh cycle

static

Test set public

no

Official test answers are hidden, but training and validation questions are public and static. Use the hidden evaluation server when available, disclose retrieval and fine-tuning data, and audit overlap with public mirrors.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official CommonsenseQA leaderboardofficial