AgentBench

What it measures

A multi-environment benchmark for LLM agents across OS, database, knowledge-graph, game, shopping, household, and web tasks.

Primary metric

overall AgentBench score (%) — weighted average of 8 environments

Judge model

not judged by a model

Human baseline

—not reportedThe benchmark is built from human-solvable interactive tasks, but no single blind human aggregate score is protocol-comparable across all eight original environments.

Languages

English, Shell, SQL

Health

AgentBench scores are system scores, not model-only scores. Task server implementation, context management, retries, tool wrappers, and split choice can change the result.

Status evidence

Superseded by AgentBench FC (2025-10-10, 5-env function-calling containerized protocol via AgentRL). Original 8-env remains discriminative at ~31% OA but main branch now serves FC; v0.1/v0.2 are archived and new evaluations should use FC (see github.com/THUDM/AgentBench — 'revert to v0.1/v0.2' for original).

Saturated since

not saturated

Refresh cycle

static

Test set public

no

The original dev split is public while the official test split is not fully exposed. AgentBench FC improves deployment reproducibility, but public code and environment descriptions still create scaffold overfitting risk.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

No model on this site has a score on this benchmark yet. That usually means the models that were measured on it are not in the registry, rather than that nobody has run it.

Trajectory

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

• AgentBench leaderboardofficial