AI Reasoning Benchmarks: GPQA, MMLU, and Math, Explained

An AI reasoning benchmark is a fixed set of hard questions that scores how well a model thinks through knowledge, logic, and math. The main ones are GPQA Diamond, MMLU, MMLU-Pro, Humanity's Last Exam, and ARC-AGI-2, plus the math tests MATH and AIME.

This family is the most saturated and most contaminated of all AI benchmarks. Top models now score above 90% on several of them, so the room to improve is small. Many older tests also leaked into training data, which inflates scores.

This guide explains what each reasoning and math benchmark measures, its main limitation, and how leading models score. It also shows when these numbers predict good work for your business and when they do not.

The Reasoning and Knowledge Benchmarks

Reasoning and knowledge benchmarks ask multiple-choice or short-answer questions across science, law, and general topics. They test what a model knows and how it reasons, not whether it can write code or run tools.

Each test has a real limitation. Knowing the limitation matters as much as knowing the score. Here are the five that lead this family.

• GPQA Diamond — about 198 PhD-level physics, chemistry, and biology questions, hand-written to be "Google-proof" (Rein et al., 2023). It is the hardest subset of GPQA. Limitation: the set is small and now near saturation at the frontier.
• MMLU — a 57-subject multiple-choice knowledge test spanning STEM, humanities, and law (Hendrycks et al., 2020). Limitation: heavily saturated above 90%, and training-data contamination is likely.
• MMLU-Pro — a harder MMLU redesign with 10 answer options and more reasoning-heavy questions (TIGER-Lab, 2024). Limitation: newer and less saturated, but still multiple-choice.
• Humanity's Last Exam (HLE) — about 2,500 extremely hard expert questions built to resist saturation (Center for AI Safety and Scale AI, 2025). Limitation: very new, scores are low, and some reported figures use tools or search.
• ARC-AGI-2 — abstraction-and-reasoning grid puzzles that test novel problem-solving, not knowledge (Chollet, 2019; ARC-AGI-2 in 2025). Limitation: costly high-compute runs can inflate scores, and it is not a coding or business proxy.

The Math Benchmarks: MATH and AIME

Math benchmarks score a model on competition-style problems with a single correct answer. That makes them easy to grade, but easy to over-read.

Two tests lead here, and both carry a warning. MATH is largely solved, and AIME is tiny enough to swing on luck.

• MATH — 12,500 competition math problems with step-by-step solutions (Hendrycks et al., 2021). It is largely saturated now, and the MATH-500 subset is the version most often used. We do not publish an on-site MATH score, because the test no longer separates top models.
• AIME — the 15-question American Invitational Mathematics Examination, used as a yearly LLM math benchmark. Limitation: 15 items means high variance, one year's questions can leak once public, and a reported 100% is easy to over-read.

How Leading Models Score

Here are the reasoning and math scores our site already publishes, grouped by model tier. We list only verified, sourced numbers, and we flag every gap honestly.

These figures mix model generations, so treat them as a snapshot, not a clean head-to-head. Always confirm the model version before you compare two numbers.

• Frontier tier (Claude Fable 5, GPT-5.6 Sol) — no published numeric scores. Fable 5 is gated to qualitative claims, and OpenAI has not published numeric scores for the GPT-5.6 family. We do not invent numbers to fill this gap.
• MMLU-Pro (see the chart tab) sits below MMLU because the test is harder, not because the models are weaker.
• ARC-AGI-2 — Claude Opus 4.5 scores 37.6%, more than doubling GPT-5.1. These scores stay low, which is the point of the test.
• Missing on purpose — we publish no on-site MATH score for any model, and no real-model HLE score. The only HLE figure we hold is Fugu Ultra's vendor-reported 50.0, which lacks independent verification.

Why Reasoning Benchmarks Saturate and Get Contaminated

Saturation means the top models cluster near the ceiling, so a benchmark stops telling them apart. Contamination means the test questions leaked into training data, so a model may recall answers instead of reasoning them out.

Both problems hit this family hard. A 94% or a 100% can look like a solved task when it is not. The gap between the leaders is often smaller than the noise in the test.

Treat any single number with care. Many frontier models are considered partly contaminated on older benchmarks, so testing on your own tasks beats a headline score. A benchmark is a starting point, not a verdict.

What Reasoning Benchmarks Mean for Your Business

Reasoning benchmarks predict good work when your task looks like the test. A high GPQA Diamond or MMLU-Pro score signals strength on dense technical reading, research synthesis, and multi-step analysis.

They predict poorly when your task does not match. A top MMLU number says little about tone in a client email, reliability in an agent loop, or accuracy on your private documents. Math scores say little about business judgment.

Pick based on the work you do most, not on raw benchmark scores. Run a short pilot on your own hardest cases, and weigh reliability over a two-point benchmark edge.

The Bottom Line

Reasoning and math benchmarks are useful signals, not final answers. Use GPQA Diamond and MMLU-Pro to gauge research and analysis strength, and read MATH and AIME with their saturation and variance in mind.

The frontier tier stays numerically quiet on purpose, so do not assume a missing score means a weak model. When a number is missing, test the model yourself.

The best choice for your team is the one that performs on your real tasks. We help you design that test and read the results without the benchmark theatre.