88% on Benchmarks, 12% on Real Work

The best AI agent SemiAnalysis tested scored 12%¹ on its GDPx benchmark. That is the top score on GDPx, a benchmark SemiAnalysis published on November 2, 2026. Anthropic's Claude Opus 5.5 posted it across 206 long tasks in 11 high-paying fields², from chip design to legal work.

The same week, Google's Gemini 4 Carbon announcement listed 88.2%³ on SEC-bench Pro, a test of finding vulnerabilities in critical software. On GDPx, SemiAnalysis scored Gemini 4 Carbon at 10.9%⁴. Leader to leader, that is a 76.2-point gap. Model to itself, it is 77.3 points.

We will own what we don't know. Neither number tells you what an agent is worth to your customer. The distance between them does show where agent roadmaps break, and it belongs in every quote a builder sends.

Long tasks multiply small failures

Reliability compounds across steps, and it compounds downward. Take an agent that gets each step right 95% of the time. Over 20 steps, the odds it gets every one right are 0.95 multiplied by itself 20 times, which comes to about 35.8%.

Push the same agent to 41 steps and the odds fall to about 12%¹. Raise it to 99% per step and 20 steps still land at about 81.8%. SemiAnalysis says GDPx tasks run for up to eight hours inside simulated workplaces. Eight hours of real work is a lot of steps.

That arithmetic is the last-mile problem, and the week's numbers sort into three layers. Narrow tests like SEC-bench Pro grade step accuracy: one bug, one patch, one clear grader. GDPx grades chain completion, which means taking an underspecified goal all the way to a deliverable an expert would accept.

The third layer is accepted value, and it's what a customer pays for. Roughly, it is the value of completed tasks minus the cost of human review. A score of 88% lives on the first layer. A score of 12%¹ lives on the second. Customers live on the third, and no public leaderboard measures it.

Launch coverage from Magica makes the same point about the headline. GDPx measures how often a frontier agent finishes hard, expensive assignments. For anyone selling autonomy, that question is narrower and more useful.

Inside the 206 tasks and harnesses

Picture a contractor who lays perfect tile and then forgets to turn the water back on. Every grout line passes inspection, and the house still floods. GDPx is built to catch the flood.

SemiAnalysis ran each model in its native harness, AICoder reports. A harness is the software wrapper that lets a model read files and call tools. Opus 5.5 ran in Claude Code and GPT-6 Astra ran in Codex. Gemini 4 Carbon ran in Antigravity. Models without a native harness got OpenCode.

The headline is pass@1 over three attempts, with all 11 domains weighted equally¹. Pass@1 counts whether a single run finishes the job, averaged across three tries. Equal weighting makes each domain worth about 9.1% of the total. Fourteen specialist data partners contributed to the tasks, and SemiAnalysis did the final quality check and iteration.

The weighting hides a sharp detail. AICoder reports that almost every model scored zero on the finance and legal tasks. For nearly everyone, those two dead domains take 2/11 of the board off the table. If Opus 5.5 also scored zero there, its 12.0%¹ implies an average of about 14.7% across the other nine.

Safety is the second hidden variable. SemiAnalysis says safety classifiers held back Opus 5.5 and GPT-6 Astra, and it is unclear whether that cost them a point or ten. The scoreboard counts a refusal as a miss. A customer running biology workloads may prefer the miss.

The third variable is who did the reporting. BenchLM.ai, which tracks SEC-bench Pro, compiles its rows from provider self-reports. It lists GPT-6 Astra at 85.4%⁵ there, against 9.6%¹ on GDPx, a ratio of about 8.9x. Gemini 4 Carbon's ratio is about 8.1x.

Before any agent score goes into a sales deck, run it through three checks in order. Step one: name the harness, because your product probably won't run in the lab's exact setup. Step two: find out whether the grading is binary or gives partial credit. Step three: find out whether the lab reported the score itself.

The grading rule is where the money hides. Say a reviewer bills $120 an hour and checks every output for 30 minutes. That is $60 of review per run before any inference cost. If half the runs pass, review alone costs $120 per accepted deliverable.

Agent scorecards may multiply by 2028

The yardsticks are piling up fast. OpenAI shipped GDPval in September 2025, sampling work by occupation. SemiAnalysis shipped GDPx 14 months⁶ later and picked domains by expected AI revenue. That same week, OpenAI unveiled an internal benchmark for everyday tasks like shopping and meal planning, and reported strong results for GPT-6 Astra.

AICoder reports that SemiAnalysis plans a new GDPx version every three to six months. From November 2026 to November 2028, that works out to four to eight new versions of one test. Add vendor-built task sets like OpenAI's, and a buyer may keep seeing more scoreboards. Each lab has a reason to publish the one it wins.

Money is moving faster than completion rates. Grand View Research estimates the AI agents market at $10.9 billion⁷ in 2026 and $182.9 billion by 2033, roughly 16.8x in seven years. A summary of McKinsey's 2026 State of AI survey puts about 20%⁸ of organizations at the scaling phase for agents.

GDPx has only one release, so it has no trend line yet. Even a generous guess leaves a wide gap. If the top score doubled every year, it would reach 24% in late 2027 and 48% in late 2028. The best agent would still fail more than half of these assignments.

That leaves builders with an asymmetric bet. Pricing on narrow benchmark scores exposes a builder to every scorecard change and every vendor-defined task set. Pricing on accepted output inside a customer's own workflow produces numbers that hold up whichever scoreboard wins. We think that second position compounds, because every logged task makes the next quote more accurate.

The counter-case deserves a hearing. SemiAnalysis chose fields where it expected the greatest AI impact over the next one to two years and skipped sectors like government and insurance. Repetitive, easy-to-check work may already pay off far below the GDPx bar. A bounded security tool that reliably finds one class of bug can earn its keep at 88% and never touch an eight-hour assignment.

Score your agent on accepted deliverables

The build is a scorecard for one workflow, made from your own tasks and rerun every time you swap a model. It needs a spreadsheet and 20 real tasks. You can set it up this weekend and fill it over the following week.

Step one: pick one bounded workflow your customers already pay for. Bounded means clear inputs and a clear pass test, such as invoice matching or vulnerability triage. Write down what "accepted" means in one sentence, for example "a reviewer signs off with no edits."

Step two: pull 20 real tasks from last month. Keep them private and out of any public repo, so no model has trained on them. Run each task three times, the same way GDPx does. That gives you 60 runs.

Step three: log five columns per run. Record whether it finished without a human taking over, the review minutes, whether rework was needed, the inference cost and whether a severe error escaped. A severe error is one that would have reached the customer if the reviewer had missed it.

Step four: compute cost per accepted deliverable. Add inference cost and review cost across all runs, then divide by the number of accepted runs. With placeholder numbers of $3 inference and $60 review, 60 runs cost $3,780. If 30 runs are accepted, each one costs $126.

Step five: sort the workflow into a tier. Assisted work means the agent speeds up a human. Supervised execution means the agent does most steps behind approval gates. Autonomous completion means accepted work with almost no review, and it earns the highest price and the strictest reliability bar.

Step six: compare that $126 against what the same deliverable costs a person today. If the agent only wins in the assisted tier, sell it as assistance and price it that way. Swap the placeholders for your real reviewer rate and token bill before you quote anyone.

Expect the first pass to look rough. If 3 of 20 tasks pass on day one, that is 15%, and it is your real baseline. Find the step that breaks most often, patch it and rerun the same 20 tasks. When the next GDPx version lands in three to six months, rerun the sheet with whichever model leads.

Practise it step by step: Design an AI-Assisted Workflow