A stealth model on Vercel's AI Gateway solved 28 01 of 31 Next.js tasks. Then it got a text file in the project root. It solved 30 02.
Same model. Same test. The file was AGENTS.md, and it pointed the agent at Next.js documentation.
According to an AGI Hunt report on September 25, 2026, that file pushed Pixel Canary from 90.3% to 96.8% 03. That reportedly tied GPT-6 Astra on Vercel's Next.js leaderboard. Vercel is offering the model free for a limited time.
I will admit the weak spot first. Two tasks on a 31 01-task benchmark is a small sample, and each task is worth about 3.2 points. The scores also use pass@4, which means the agent gets up to four tries per task. Even so, the full leaderboard says something builders should hear: the context you hand an agent can matter as much as the model you pay for.
The Flat Ceiling Rule for Agent Docs
Vercel's Next.js evals page, last run September 25, 2026, lists 14 04 current model and agent pairs. Without docs, their success rates spread from 74% to 97%. With docs, the spread shrinks to a reported range of 90% 11 to 97%.
Three models hit 97% with docs, but they do not charge the same to get there.
I call it the Flat Ceiling Rule. Good framework docs pull weaker models toward the top, and the top barely moves. The short version: once the right docs are in the room, model choice stops being your biggest lever.
Sort the leaderboard and three groups fall out.
Already knows it: Claude Opus 5.5 07 (high) and GPT 6 Sol score 97% 04 with and without AGENTS.md, and Claude Fable 5.1 09 matches them. Grok 4.7 10, running in OpenCode, sits at 94% either way. These models already carry current Next.js knowledge, so the file adds nothing.
Docs close the gap: Pixel Canary jumps from 90% 11 to 97% 04, and Gemini 3.8 12 Flash and GPT 6 Astra make the same jump. Kimi K3 climbs 13 14 points. GLM 5.2 15 and Claude Sonnet 5 each climb 16 points to hit the ceiling.
Docs help, but not enough: Cursor Composer 2.5 17 rises from 81% to 94% 10. MiniMax M3 and Kimi K2.7 Code reach 90% 11. GPT 6 Luna tops out at 87% 06.
Pixel Canary sits in the middle group. Its raw baseline is nothing special on this board, since GPT 6 Astra and Gemini 3.8 12 Flash post the same 90% 11. What stands out is how hard it responds to context. Three failures became one, which cuts its misses by two-thirds.
Inside the 8 KB File That Beat Skills
AGENTS.md is a plain markdown file at your repo root that tells coding agents how to work in your project. According to the Next.js docs, tools like Claude Code, Codex, Cursor and GitHub Copilot commonly read it at session start. Next.js now bundles version-matched docs inside the package at node_modules/next/dist/docs/, and the AGENTS.md file points the agent there instead of at its frozen training data.
Picture your model as a brilliant contractor who memorized the building code two years ago. Next.js has changed the code since then, including caching defaults and the boundaries between server and client. A React answer can compile fine and still be wrong for your Next.js version. AGENTS.md staples this year's code to the clipboard he checks before every job.
The surprising part is why a passive file wins. Vercel ran a separate test comparing AGENTS.md against skills, which are add-on modules an agent can choose to load. With no docs, agents passed 53% 21 of tasks. A skill left on default behavior also scored 53%, because the agent often never opened it.
Explicit instructions to use the skill pushed that to 79% 23. An 8 KB compressed docs index pasted straight into AGENTS.md hit 100% 24 on build and lint, and passed every test as well, according to Vercel. A skill is a manual sitting in the break room. AGENTS.md is a sticky note on the monitor, read on every turn.
Simple beats complex here. A file the agent cannot skip is about as simple as agent plumbing gets. Nobody has to remember to call it, and the model does not have to guess when it needs help.
Cost is where the Flat Ceiling Rule pays rent. GLM 5.2 15 runs at an average list cost of $0.152 per task and hits 97% 04 with AGENTS.md. Claude Opus 5.5 07 costs $0.234 and also hits 97%. That works out to about 35% less per run for the same score on this benchmark.
GPT 6 Astra costs $0.891 per task for that same 97% 04 with docs. Across 1,000 agent runs, that comes to about $891 for Astra and about $152 for GLM 5.2 15. Once the docs were in place, the $739 gap bought zero extra passes on this test.
Pixel Canary comes with its own fine print. The evals page lists its cost as N/A because it is free right now. Its average run took 1015.80 seconds, roughly 17 minutes per task, compared with about 4.5 minutes for Opus 5.5 07. The AGI Hunt report also notes no ZDR (zero data retention), so your prompts and responses may be used for training.
The fine print under a 97% ceiling
Budget models reach the top with docs.
GLM 5.2 15 and Claude Sonnet 5 each climbed 16 points to hit 97% 04 with AGENTS.md. GLM 5.2 gets there at $0.152 per task, about 35% less than Claude Opus 5.5 07 for the same score.
Near-perfect still compounds into failure.
The scores use pass@4, so agents get up to four tries per task. If 20 independent changes each succeed 96.8% 03 of the time, all 20 land cleanly only about 52% of the time.
Vercel owns every layer of this result.
Vercel controls the framework, the benchmark, the docs and the gateway, and nobody has independently replicated the numbers. Some descriptions cite 24 tasks where the Pixel Canary result cites 31, and each task is worth about 3.2 points.
2031: Every Framework Ships Its Own Agent Brain
Look at how fast this board moves. Claude Opus 4.8, measured May 28, 2026, scored 68% base and 74% 04 with AGENTS.md. Four months later, the top rows sit at 97%. Models and framework APIs both change every quarter, while training data freezes the day it is cut.
That gap between a moving framework and a frozen model is permanent. Every agent session starts with a kind of forced beginner's mind, and the only question is whether you hand it a current map.
The bet is lopsided in your favor. Writing and trimming an 8 KB context file costs an afternoon. It works with whatever model you swap in next quarter, and the payoff grows each time a cheaper model reaches the ceiling. Most teams are leaving that advantage on the table.
The frameworks see it too. Next.js 16.3 and later automatically generate AGENTS.md and CLAUDE.md when they detect a coding agent, according to the Next.js docs. The repository was still updating its AGENTS.md on September 24, 2026. I think that by 2031, shipping agent-facing docs will be as expected as shipping TypeScript types, and frameworks that skip it will lose share of agent-written code.
Vercel controls the framework, the benchmark, the docs and the model gateway, and nobody has independently replicated the results yet. Some descriptions of the suite also cite 24 tasks where the Pixel Canary result cites 31 01, which suggests the task set has shifted over time.
It is unclear whether the effect holds for Python backends or infrastructure code. Large monorepos are another open question. And the ceiling is not perfect. If 20 independent changes each succeed 96.8% 03 of the time, all 20 land cleanly only about 52% of the time. Tests and human review still earn their keep.
Run Your Own Docs-On, Docs-Off Test
You do not need a CS degree for any of this. It is a markdown file, a handful of tasks and a spreadsheet. Budget about two hours.
First, check your Next.js version. The docs list 16.3.6 as the latest, and anything at 16.3 or later can generate AGENTS.md for you. Confirm the folder node_modules/next/dist/docs/ exists in your project.
Then open AGENTS.md and make sure it points at those bundled docs. Add your own project rules underneath: folder structure, naming, which data layer you use. Keep the whole file small, since Vercel's winning index was about 8 KB. If you have never written a conventions doc, the Document Kit templates from today's digest give you a starting outline.
Next, pick 10 real tasks from your backlog. Run each one with AGENTS.md and again with the file renamed. Count passes, count dollars, and count first-try passes separately, because pass@4 hides retries.
For a cheap model to test with, route stealth/pixel-canary through Vercel's AI Gateway while it is free. Use throwaway or open-source code only, since there is no zero data retention. If you want this test to rerun every week on its own, Tines 3B lets you describe that workflow in plain language.
Expect your first run to be messy. Maybe the docs show no lift because your model already knows the APIs. That is still a win, because now you know the lift is not there, and you can stop paying for a bigger model you do not need.
Build the small test, log the numbers and share what breaks. One markdown file might save you more than your next model upgrade.
Run your own docs-on, docs-off test.
- Check your Next.js version. Anything at 16.3 or later can generate AGENTS.md for you. Confirm the folder node_modules/next/dist/docs/ exists in your project.
- Tighten your AGENTS.md. Point it at the bundled docs, then add project rules like folder structure, naming and your data layer. Keep the whole file near the 8 KB size of Vercel's winning index.
- Score 10 real backlog tasks. Run each with AGENTS.md and again with the file renamed, logging passes, dollars and first-try passes separately. Use throwaway code if you route through Pixel Canary, since it has no zero data retention.
Before you upgrade the model, upgrade the context.
Pixel Canary's jump from 90.3% to 96.8% 03 came from a markdown file, not a new model. Across Vercel's board, docs compressed a 74% 04 to 97% spread into roughly 90% 11 to 97%, letting cheaper models match premium ones. The sample is small, the retries are hidden and the vendor grades its own work, so test it on your own tasks. An afternoon spent on an 8 KB file may save more than your next model upgrade.