TabMWP

What it measures

Grade-level math word problems that require combining a question with structured, textual, or image-rendered tables.

Primary metric

answer accuracy (%)

Judge model

not judged by a model

Human baseline

90.22Official test-set estimate reported by the benchmark authors; subgroup accuracy varies by free-text, multiple-choice, and answer type.

Languages

English

Health

The test split is large, but errors are clustered by source table and answer format; close results must use the same table representation, prompt selection, tools, and parsing rules.

Status evidence

The official leaderboard reports Chameleon with GPT-4 at 98.78% on text-ground-truth tables, above the published 90.22% human estimate; the fixed public test set no longer separates frontier tool systems.

Saturated since

2023-04-19

Refresh cycle

static

Test set public

yes

None beyond fixed train/dev/test splits; all tables, solutions, and test answers are public in the repository.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• Official PromptPG leaderboardofficial