Grok 4.5 Benchmarks: Published Scores and How to Read Them

xAI published a narrow, coding-focused benchmark set for Grok 4.5 rather than a full sweep of MMLU-style reasoning evals. The launch numbers cover DeepSWE 1.0, DeepSWE 1.1, SWE Marathon, Terminal Bench 2.1, and SWE Bench Pro. There is no official MMLU-Pro or GPQA Diamond score in the launch materials.

This page lists every score xAI has stated in its own announcement, defines what each benchmark measures in plain English, and flags where third-party numbers exist versus where readers should treat a figure as unconfirmed. Where a number is not published by xAI, we say so and point you to the vendor page.

Benchmarks are a starting point, not a purchase decision. The last two sections cover why a top score on SWE Bench Pro does not automatically translate to shipped tickets, and what a small team should actually test on their own workload before switching stacks.

The scores xAI published for Grok 4.5

xAI's Grok 4.5 announcement leans on software-engineering and agentic evals. The stated launch scores are: DeepSWE 1.0 62.0%, DeepSWE 1.1 53%, SWE Marathon pass@1 29.0%, Terminal Bench 2.1 83.3%, and SWE Bench Pro resolve rate 64.7%. These are the numbers xAI publishes as its own frontier claim.

Notably absent from the launch page are the classic reasoning benchmarks. There is no official MMLU-Pro, GPQA Diamond, MATH, or HumanEval score in xAI's Grok 4.5 announcement. If you see one of those figures quoted elsewhere, assume it is a third-party test, not a vendor claim, until you find it on x.ai.

xAI publishes current pricing and the full model card at x.ai and docs.x.ai — verify before you budget or commit to a benchmark comparison. The published input and output token rates are $2.00 and $6.00 per million tokens respectively, but rate limits and context window size point back to the model detail page.

What each benchmark actually measures

Every benchmark tests a narrow slice of behavior, and the slice matters more than the score. DeepSWE 1.0 and 1.1 measure whether a model can resolve real GitHub-style issues end to end, given a repository and a bug report. A higher score means more issues closed by patches that pass tests.

SWE Marathon pass@1 measures single-shot fixes on long-horizon software tasks. Pass@1 means the model gets one attempt — no retries, no self-correction loops. That is the honest number for a first-response coding assistant.

Terminal Bench 2.1 measures shell-level task completion — installing packages, running scripts, handling command output. It is closer to what an agent does in a sandbox than what a chat window does in a browser.

SWE Bench Pro measures resolve rate on a harder, curated set of software engineering problems. A 64.7% resolve rate means the model produced a patch that passed the hidden test for that fraction of tasks.

What is missing from the launch materials

xAI's launch page does not include MMLU-Pro, GPQA Diamond, MATH, HumanEval, or agentic evals like GAIA. That silence is a signal — vendors typically publish the benchmarks where they win. Buyers who care about general reasoning or graduate-level science QA should not assume Grok 4.5 leads there just because it leads on SWE Bench Pro.

Third-party trackers and blogs are already circulating GPQA Diamond scores for Grok 4.5 in the low-90s range, but those are not vendor-published numbers and testing methodology varies. Treat them as directional until xAI or an independent benchmark authority confirms.

If you need a specific eval score to make a decision — say, MATH for a tutoring product or MMLU-Pro for a research assistant — run the eval yourself against your own prompts. Public leaderboards rarely match your production distribution.

Third-party replication and where to look

Independent benchmark tracking is thin so soon after launch. Aggregators like Artificial Analysis, LMSYS Arena, and vendor-neutral eval hubs typically publish community-run comparisons within the first few weeks of a frontier release. Check those before quoting any figure not on x.ai.

Community coding benchmarks tend to show the widest gap between marketing scores and lived experience. A model that resolves 64.7% of SWE Bench Pro tasks in a controlled harness might resolve a very different fraction inside a real IDE with a real user hitting cancel.

In our own work running the /keyword-gap and /mindmap-pass routines across dozens of AI-content sites in our portfolio, the pattern we see with a new model launch is that the first month's third-party numbers swing 5-10 points as prompt templates and harnesses get standardized. Anchor your evaluation on your own workload, not on the first-week leaderboard.

How to read a coding benchmark score

A benchmark score is a percentile on a specific test, not a promise about your codebase. Coding evals like SWE Bench Pro pull from public GitHub issues that skew toward well-scoped bugs in mature open-source projects. Your production repo probably has private dependencies, sparse tests, and undocumented conventions.

Look at three things before you trust a score. First, the harness — how many attempts, what tooling the model can call, whether it sees test output. Second, the sample size — a 20-task benchmark has huge variance. Third, the pass criterion — pass@1 is much stricter than pass@5.

Compare like-for-like or don't compare. A Grok 4.5 SWE Bench Pro resolve rate cannot be directly stacked against a competitor's HumanEval score. They test different things at different difficulties with different rules.

Why benchmarks under-predict real usefulness

Benchmarks measure model output in a vacuum. Real usefulness is model output plus a prompt, plus context, plus the tolerance a user has for the wrong answer. A model that scores 5 points lower on SWE Bench Pro but responds twice as fast can ship more tickets on a real team.

Latency, cost per task, and the shape of failure modes rarely appear on a benchmark card. A model that fails silently is worse than one that fails loudly, even at the same accuracy. That property does not show up in a resolve-rate number.

Context handling under load is the other gap. A 64.7% resolve rate on curated SWE Bench Pro tasks says little about how a model behaves at 100k tokens of messy repo context with three tool calls in the loop. Test the shape of the workload you actually run.

What to test yourself before switching stacks

Pick 20 real tasks from your last two sprints. Include the messy ones — the ones with vague acceptance criteria, missing tests, and legacy code. Run them through Grok 4.5 and your current model side by side. Score them yourself; do not outsource the judgment to a rubric you did not write.

Time each response and log each failure. A 10-second response that needs one revision usually beats a 40-second response that lands first-try. Cost matters too — at $2 input and $6 output per million tokens, long agentic loops add up fast.

Do not migrate a stack on a benchmark headline. Run the test, look at where each model fails on your work, and decide from there. That is the test that predicts whether Grok 4.5 will earn its slot.