Claude 3.5 Sonnet (20241022)
anthropic/claude-3-5-sonnet-20241022
Creator
AnthropicReleased
2024-10-22Weights
not reportedContext
not reportedPrice in / out per Mtok
not reported / not reportedState
rankedHealth summary
2 of 3 scores sit on benchmarks that still discriminate. Every number here has been measured independently. 1 sits on benchmarks with a high risk of test-set contamination.
Scores
| Benchmark | Config | Score | Harness | Provenance | Measured | Health |
|---|---|---|---|---|---|---|
| Knowledge | ||||||
| GPQAnearing saturation | base | 55.3% | not reported | ind. | 2024-10-22 | superseded |
| Humanity's Last Examactive | base | 4.1% | not reported | ind. | 2024-10-22 | clear |
| On benchmarks that no longer discriminateKept visible on purpose. These are often the numbers a launch announcement leads with. | ||||||
| MMLUsaturated | base | 87.3% | not reported | ind. | 2024-10-22 | saturatedcontaminatedsuperseded |