OpenAI's GPT-6 Astra reportedly scored 99.9%01 on ARC-AGI-3 under the Provider Adapter harness on September 3, 2026. The same model, on the same Semi-Private environments, in the same week, reportedly scored 62.7%02. The weights did not change. Only the harness did.
That is a 37.2-point gap. ARC Prize verified both runs and says both numbers are correct. Here is my damaging admission: I do not know what Astra's "real" score is. Neither does anyone outside OpenAI.
Four new models landed in the first week of September: GPT-6 Astra, Claude Fable 5.105, Gemini 3.8 Flash, and Muse Spark 1.3. Each shipped with a headline number. Each number came out of a different harness. You are not comparing six models. You are comparing six pieces of plumbing.
The Scaffold Delta
Name the thing. The Scaffold Delta is the vendor-harness score minus the neutral-harness score for the same model on the same benchmark. Astra's Scaffold Delta on ARC-AGI-3 is 37.2 points. That one number tells you how much of the product is the wrapper.
Two verified runs, one set of weights, four numbers that decide what you are buying.
Sort models into three buckets by their delta. Under 5 points: the model carries the number, and you can mostly trust the weights. Between 5 and 20: the harness matters, so ask which one you will actually get. Over 20: the harness is the product, and you are buying orchestration software with a model attached.
The Standard harness lets a model carry forward only the notes it chooses to keep across turns. The Provider Adapter harness preserves opaque reasoning state between requests and uses compaction for long conversations, so the model reuses prior work. Same brain, different memory.
Now the part that should bother every buyer. The Standard run cost $26,09819. The Provider Adapter run cost $18,81720. The better plumbing scored 37.2 points higher and cost about 28% less. Whether the gain comes mostly from hidden state or mostly from compaction is anyone's guess, because ARC Prize did not publish an ablation between the two.
Where the 37.2 Points Actually Live
Think of a frontier model as a 500 IQ intern. Brilliant, fast, and completely dependent on the desk you give it. The harness is the desk: the filing cabinet, the to-do list, the manager who says "try that again," and the clerk who grades the work. Four parts. Tool wiring, retry logic, context management, and the scoring adapter.
The Standard harness hands the intern a stack of sticky notes and wipes their short-term memory every turn. Whatever they did not write down is gone. The Provider Adapter lets the intern keep their whole train of thought and hands them a summarizer when the desk gets cluttered. Same intern, and ARC Prize reports the second setup used fewer actions than the median human on 96% of levels.
Use the website test here. A model in a bare harness is a Tractor: it works, it is ugly, and it grinds through tasks at $26K a run. Astra inside OpenAI's adapter is a Unicorn on ARC-AGI-3, beautiful and converting at 99.9%01. Astra inside your hastily built Python loop with no state and no compaction is a Ferrari with the engine removed.
An ounce in pre is worth a pound in post. The teams that win the agentic era spend their effort on context management and retry paths before they spend a dollar on more tokens. Astra bills at $1021 per million input tokens and $5022 per million output tokens on a 1.0508-million-token context window. Every time your harness forgets and re-reads, you pay that bill again.
Look at the demo claims through this lens. OpenAI showed a 5-hour human job search done in 2 minutes 51 seconds and a 3010-minute pet care task done in 5 minutes 27 seconds. Those are agentic runs, which means they are harness runs. Nobody has shown you the same tasks through a neutral scaffold, and I would bet the times look different.
Then Sam Altman apologized within 48 hours23 for a staggered rollout. Most paid ChatGPT tiers, the API, and Bedrock customers gained access over the coming days. Read that carefully: the harness that produced the 99.9%01 is OpenAI's internal orchestration. The harness you get is the API plus whatever you wire yourself.
Cut the noise. Of the four harness parts, 80% of your agent's reliability comes from two: state preservation and retry logic. The scoring adapter only matters for leaderboards. Tool wiring matters, but a model with good memory and a second chance will usually route around a clumsy tool.
Translate it to dollars one more time. The gap between the two ARC runs was $7,281 on a single benchmark pass. Multiply that by every agentic workflow your company runs per month. The harness is not an engineering detail. It is a line item.
What The Wrapper Owns That The Weights Do Not
The cheaper run is the one you cannot port.
The Provider Adapter pass cost $18,81720 against $26,09819 for the Standard harness, about 28% less, while scoring 37.2 points higher. The saving comes from opaque reasoning state you only get inside one provider's orchestration, so the discount and the lock-in arrive together.
Nobody has split hidden state from compaction.
ARC Prize verified both runs but did not publish an ablation between preserved reasoning state and context compaction. Until someone does, buyers cannot tell which half of the gap they could rebuild themselves and which half is proprietary.
The benchmark went from floor to ceiling in half a year.
ARC-AGI-3 launched on March 25, 202603 with a best frontier score of 0.37%, held by Gemini 3.1 Pro Preview, with Opus 4.6 Max at 0.25%. By September 3 the top figure was 99.9%01 under a vendor harness and 62.7%02 under the standard one. Weights depreciate on that clock; wiring does not.
2031: Nobody Buys a Bare Model
Pull back five years. ARC-AGI-3 launched on March 25, 202603, and the best frontier score was 0.37%, held by Google's Gemini 3.1 Pro Preview, with Anthropic's Opus 4.6 Max at 0.25%. By September 3 the same benchmark hit 99.9%01 under a vendor harness, though 62.7%02 on its standard harness. Six months from floor to ceiling. That is impermanence in its purest form, and it will happen to ARC-AGI-4 too.
Here is the contrast pair to carry. Weights depreciate. Wiring compounds. A model is state of the art for roughly one quarter now. A harness that manages state, retries cleanly, and compacts context keeps paying off across every model you swap in behind it.
Costco sells a hot dog and a soda for $1.50, a price unchanged since 1985. The hot dog is not the business. The membership is. The 99.9%01 headline is the hot dog: a loss leader that gets you in the door. The harness, the part that locks your workflows to one provider's hidden state, is the membership.
That is counterpositioning, and it cuts both ways. Vendors will keep publishing adapter scores because a neutral harness cannot use their proprietary state features, so it makes their model look worse. Buyers face asymmetric risk. Build on a portable harness and you lose a few points today. Build on a proprietary one and you lose the option to leave.
Amateurs benchmark the model. Operators benchmark the system. I think by 2031 procurement teams will treat vendor-harness scores the way they treat vendor-run speed tests today: interesting, unverified, and not in the contract. ARC Prize already lists both harness types on its leaderboard. The market will follow.
The data is mixed on one thing. Some analysts argue the 62.7%02 understates Astra because real deployments do have vendor tooling available. Others argue the 99.9%01 overstates it because it hides weaknesses a neutral setup exposes. Both are true, which is why you need your own number.
Rerun One Vendor Demo on Your Stack
Show, don't tell. You do not need a CS degree to measure your own Scaffold Delta. You need one task, one afternoon, and a spreadsheet.
First, pick one demo claim you actually care about. The 3-minute job search is a good one, because most teams can define what "done" means for a search. Write the success criteria down before you run anything. If you cannot describe done in two sentences, the task is not ready to automate.
Second, write down your harness. A harness is just the code that wraps the model call: how you pass tools, how many times you retry a failed step, and how you handle a full context. Compaction means summarizing old turns so the model can keep working without re-reading everything. If you have none of these, write "none." That is your Standard harness.
Third, run the task ten times. Log three columns: did it finish, how many model calls did it take, and what did it cost. Use the $1021 input and $5022 output rates for Astra, or whatever your provider charges. Ten runs is enough to see the shape.
Fourth, add one thing. Give the agent a notes file it can write to and read from between steps. Run ten more times. The difference between your two averages is your own Scaffold Delta. It will not be 37.2 points, but it will not be zero either.
If your team is evaluating Jason AI for B2B outreach, do this before you turn it on. Jason AI runs the sequence, replies to prospects, and books the meeting. The handoff to a human is a harness decision, so define it first: which reply triggers escalation, who gets pinged, and what the agent does if nobody answers.
Use Sidenote while you read the ARC Prize results page with your team. It adds a comment layer to any URL, so you can flag the two harness footnotes in place instead of pasting links into chat. Then build a two-slide summary in Beautiful.ai: slide one shows 62.7%02 and 99.9%01, slide two shows your own two numbers. The layout rules stop you from fiddling with text boxes, so you spend the time on the argument.
Things will break. The notes file will get corrupted on run six. The retry loop will loop forever once. That is the whole point of running it on your own stack before you re-architect around a headline. Get your reps in, and stop reading benchmark scores as if they measure the model. They measure the desk.
Measure your own Scaffold Delta before you trust anyone else's number.
- Pick one demo claim and define done. Take a task you actually care about, such as the 5-hour job search OpenAI ran in 2 minutes 51 seconds, and write the success criteria in two sentences before you run anything. If you cannot describe done that briefly, the task is not ready to automate.
- Run it ten times with no harness. Log three columns per run: did it finish, how many model calls it took, and what it cost at $1021 per million input and $5022 per million output tokens. That average is your Standard baseline.
- Add a notes file and rerun ten times. Give the agent one file it can write to and read from between steps, then compare averages. The difference is your own Scaffold Delta, and it will not be 37.2 points but it will not be zero.
Stop reading benchmark scores as if they measure the model. They measure the desk.
Four models shipped in the first week of September - GPT-6 Astra, Claude Fable 5.105, Gemini 3.8 Flash and Muse Spark 1.3 - and each headline number came out of a different harness. Astra's 37.2-point spread between 99.9%01 and 62.7%02 is the cleanest proof available that the wrapper, not the weights, is what you are actually procuring. Sort every vendor claim by its Scaffold Delta: under 5 points the model carries the number, over 20 points you are buying orchestration software with a model attached. The $7,281 cost gap on a single pass is a line item, not an engineering detail. Build the portable harness, lose a few points today, and keep the option to leave.