Skip to content
LLM Atlas

Benchmarks

FrontierCode

activeCodingcontamination risk: lowlaunched 2026-06-08

Benchmark metadata from benchwiki — this page mirrors their record so scores can be joined to it. The original is canonical.

What it measures

Private maintainer-authored repository tasks graded for mergeability: correctness, tests, scope discipline, style, and code quality.

Primary metric
pass rate on FrontierCode 1.1 Main (%)
Judge model
not judged by a model
Human baseline
not reportedNo controlled human aggregate is published. Repository maintainers author the tasks and define mergeability criteria, but that provenance is not a measured human score.
Languages
Multilingual

Health

Main contains 100 tasks and Extended 150; results average five trials and select the best reasoning effort. Timeline dates are model launch dates; source_url points to Cognition's score table. Never mix versions, subsets, or pass rate with weighted score.

Status evidence
FrontierCode 1.1 remains difficult and newly launched; Cognition's official results show substantial headroom on the current Main and Extended sets.
Saturated since
not saturated
Refresh cycle
static
Test set public
no

Tasks and solutions remain private. FrontierCode 1.1 allows ordinary documentation search but prompts agents to avoid solution-bearing sources and programmatically zeroes runs that access them.

Models scored on this benchmark

The reverse lookup, which is this site’s addition: every model here with a score on this benchmark. Independent measurements and vendor-reported numbers are kept apart rather than interleaved, because they are different claims.

Independently measured

ModelScoreHarnessMeasuredProvenanceHealth
Claude Fable 5Anthropic53.5%claude-code2026-06-09ind.clear
Grok 4.6xAI48.0%grok-build2026-08-12ind.clear
Grok 4.7xAI47.6%grok-build2026-09-21ind.clear
GPT-5.6 SolOpenAI47.5%codex2026-07-09ind.clear
Claude Opus 4.8Anthropic46.5%claude-code2026-05-28ind.clear
Kimi K3Moonshot44.2%mini-swe-agent2026-07-16ind.clear
Gemini 3.7 FlashGoogle DeepMind43.6%chisel2026-08-13ind.clear
GPT-5.5OpenAI43.0%codex2026-04-23ind.clear
Claude Sonnet 5Anthropic42.7%claude-code2026-06-30ind.clear
Grok 4.5xAI42.4%grok-build2026-07-08ind.clear
GPT-5.6 TerraOpenAI41.3%codex2026-07-09ind.clear
GPT-5.6 LunaOpenAI39.8%codex2026-07-09ind.clear
Claude Opus 4.7Anthropic38.5%claude-code2026-04-16ind.clear
Gemini 3.6 FlashGoogle DeepMind34.4%chisel2026-07-21ind.clear
Kimi K2.7Moonshot30.1%mini-swe-agent2026-06-12ind.clear
GPT-5.4 MiniOpenAI27.0%codex2026-03-17ind.clear
Claude Opus 4.6Anthropic26.6%claude-code2026-02-05ind.clear
Composer 2.5Cursor25.6%cursor-cli2026-05-18ind.clear
GLM-5.2Z.ai (Zhipu AI)24.5%mini-swe-agent2026-06-16ind.clear
Claude Sonnet 4.6Anthropic24.3%claude-code2026-02-17ind.clear
DeepSeek V4 FlashDeepSeek18.8%chisel2026-07-31ind.clear
DeepSeek V4 ProDeepSeek17.6%mini-swe-agent2026-04-24ind.clear
MiniMax M3MiniMax14.7%mini-swe-agent2026-06-01ind.clear
InklingThinking Machines14.0%mini-swe-agent2026-07-15ind.clear
Nemotron 3 UltraNVIDIA13.6%chisel2026-06-04ind.clear
Qwen3.7-PlusAlibaba10.2%mini-swe-agent2026-06-02ind.clear
Mistral Medium 3.5Mistral AI8.0%chisel2026-04-28ind.clear

Vendor-reported

Reported by the model's own maker. No independent measurement of these numbers is on this site.

ModelScoreHarnessMeasuredProvenanceHealth
SWE-1.7Cognition42.0%chisel2026-07-08vendorvendor only
SWE-1.6Cognition9.4%chisel2026-04-07vendorvendor only

Trajectory

Best score measured on this site

0255075100Claude Opus 4.6: 26.6%Claude Sonnet 4.6: 24.3%GPT-5.4 Mini: 27.0%Claude Opus 4.7: 38.5%GPT-5.5: 43.0%DeepSeek V4 Pro: 17.6%Mistral Medium 3.5: 8.0%Composer 2.5: 25.6%Claude Opus 4.8: 46.5%MiniMax M3: 14.7%Qwen3.7-Plus: 10.2%Nemotron 3 Ultra: 13.6%Claude Fable 5: 53.5%Kimi K2.7: 30.1%GLM-5.2: 24.5%Claude Sonnet 5: 42.7%Grok 4.5: 42.4%GPT-5.6 Sol: 47.5%GPT-5.6 Terra: 41.3%GPT-5.6 Luna: 39.8%Inkling: 14.0%Kimi K3: 44.2%Gemini 3.6 Flash: 34.4%DeepSeek V4 Flash: 18.8%Grok 4.6: 48.0%Gemini 3.7 Flash: 43.6%Grok 4.7: 47.6%20262026
Each dot is a score measured on this site. No line is drawn: these scores come from different agent harnesses, so a new best may be a better harness rather than a better model.

As benchwiki records it

SWE-1.6 (FrontierCode 1.1 Main pass rate) — 9.4 on 2026-04-07 (vendor-reported)Kimi K2.7 Code (FrontierCode 1.1 Main pass rate) — 30.1 on 2026-06-12 (independent)SWE-1.7 (FrontierCode 1.1 Main pass rate) — 42.3 on 2026-07-08 (vendor-reported)GPT-5.5 (FrontierCode 1.1 Main pass rate) — 43 on 2026-04-23 (independent)Claude Opus 4.8 (FrontierCode 1.1 Main pass rate) — 46.5 on 2026-05-28 (independent)20262026470
independent · vendor-reported. Points are not joined: the scores come from different models, harnesses and protocol versions, and a line between them would claim a continuity the data does not have.

Where results are tracked

Leaderboards benchwiki lists for this benchmark. These are links out, not sources this site ingests — no score below comes from any of them.

Source

This record is maintained by benchwiki and mirrored here. Read the original. Upstream last updated 2026-08-03; fetched 2026-10-19 by this site.