Gemini 1.5 Pro (Feb 2024)
google-deepmind/gemini-1.5-pro-001-feb24
Creator
Google DeepMindReleased
2024-02-15Weights
not reportedContext
not reportedPrice in / out per Mtok
not reported / not reportedState
rankedHealth summary
0 of 2 scores sit on benchmarks that still discriminate. Every number here has been measured independently. 2 sit on benchmarks with a high risk of test-set contamination.
Scores
| Benchmark | Config | Score | Harness | Provenance | Measured | Health |
|---|---|---|---|---|---|---|
| On benchmarks that no longer discriminateKept visible on purpose. These are often the numbers a launch announcement leads with. | ||||||
| BIG-Bench Hardsaturated | base | 84.0% | not reported | ind. | 2024-02-15 | saturatedcontaminatedsuperseded |
| MMLUsaturated | base | 81.9% | not reported | ind. | 2024-02-15 | saturatedcontaminatedsuperseded |