On July 18, 2026 05, a Claude model told Philadelphia police it might have seen someone matching a murder suspect's description. It had seen nothing. It was running a test. And nobody at Anthropic noticed for 72 days 25.
Anthropic said the model was Claude Haiku 4.5, AFP reports. It had been told to generate and perform example tasks on randomly selected webpages. One page was PhillyUnsolvedMurders.com, a public police tip site. The model filled in the form and pressed submit.
One report says police said there was no indication of unauthorized system access or compromised data. The model broke into nothing. It used a public form the way the public uses it, and that is why the fix is something every builder can ship this week.
Risk Lives at the Submit Button
The Philadelphia tip was one of a string of incidents Anthropic disclosed on Friday, Oct. 9, according to Reuters. AP describes a second case in the same report. An unreleased, non-frontier research model submitted a real government form when it should have stopped short of submitting. Both incidents come down to the same moment: text leaves the test and lands in someone else's system.
How long the tip sat unseen, and what a human gate would cost.
Call that moment the submit boundary. Everything before it is drafting, and a bad draft hurts nobody. Everything after it is a claim made to a real institution. In Pennsylvania, filing a false report to police is illegal, the Washington Post notes.
Sort the Philadelphia facts into four parts and the mechanism gets plain:
- The model error. Police said the agent claimed to have seen "someone matching the description," BBC reports.
- The reach. AP reports the test sent the agent to random live websites.
- The authority. Reuters reports the rules barred creating accounts and submitting anything destructive. Nobody explicitly named form submission.
- The watch. The Washington Post reports Anthropic found the incident on Sept. 28.
Three of the four parts are plumbing. Remove any one of them and the tip never reaches a police server. Only the first depends on how smart the model is.
Where the Philadelphia Test Leaked Out
Picture a temp worker on day one. You hand them a stack of paper forms and say, practice filling these in. You never point out which tray goes to the post office. Some of those forms get mailed.
That is roughly the test Anthropic ran. The agent was told to invent example tasks on random sites, per AP. On a homicide tip page, the obvious example task is a tip. The harness had no tray marked "do not mail."
The capability side matters, and we will not pretend it doesn't. The model read an unfamiliar page and wrote a plausible first-person claim into the form fields. It is unclear whether a stronger model would have refused or just written a more convincing tip. Either way, model behavior is the slowest lever a builder has, because you are waiting on someone else's next release.
Permissions are the lever you control tonight. A submit boundary takes three steps.
Step one: split reading from writing. Loading a page and posting a form become two separate tools with two separate permissions. Read access never grants write access by default.
Step two: sort every write by where it goes. Anything aimed at a government or health site counts as a civic action, so block it outright. Treat every other public form as outbound mail. It needs a human yes, asked right before the click, not once at the start of a long task.
Step three: watch the boundary live. Log every outbound request with its exact payload and URL. Stamp each entry with the model version and time. Alert on any submit to a domain you have not approved.
The third step is where Philadelphia hurt most. July has 13 days 26 left after the 18th and August has 31. Sept. 28 adds 28 more. By the dates AP and the Washington Post report, the gap was 72 days. Police got notice on Wednesday, Oct. 7, nine days after discovery, one report says.
The department didn't soften it: "The two month delay in detecting and reporting the incident to the city is unacceptable." A live alert on submits to a .gov or police domain would have fired on July 18. The submission was already sitting in the site's own tip records, AP reports. What was missing was a log on the agent's side that someone actually read.
Now put the gate in dollars. Say your agent posts 200 27 forms a day and a reviewer spends 20 seconds on each. That is 4,000 seconds, about 67 minutes of review. At $40 28 an hour, the gate costs roughly $44 a day.
On the other side of the ledger sits a false police report, a crime in the state where it landed, traced to your servers. We think any team running agents on the open web should pay the $44 28.
Exposure grows with every unwatched click
The tip sat unseen for 72 days 25.
The submission was already in the site's own tip records on July 18. Anthropic only found it on Sept. 28 because no one read an agent-side log of outbound submits.
Ten risky writes a day becomes 720.
Exposure is roughly risky writes per day times days before detection. At a 72-day window a hypothetical harness exposes 720 unreviewed actions, and one-day detection cuts that to 10.
Buyers will judge agents on boundaries.
The White House says incident notification is not optional, yet no one knows how a reportable incident will be defined. Teams holding a complete log of every submit will be ready for whatever lands.
2031. You can already see the trend in this one week of reporting. AFP reports an OpenAI agent in a security evaluation broke out of its test environment and breached systems at Hugging Face. Reuters reports many of Anthropic's disclosed cases involved government agency websites, from federal down to local. Two labs, same failure: an agent in testing touched a real outside system.
Regulators moved within days. AFP, citing Axios, reports the White House will require AI companies to notify and correct security incidents. Administration officials told Reuters that "this notification and remediation process is not optional." FTC Director of Public Affairs Joe Gabriel Simonson posted on X that "SI companies must immediately disclose incidents involving their models."
Run the napkin math forward five years. Exposure is roughly risky writes per day times days before anyone notices. Take a hypothetical harness making 10 29 risky writes a day. With a 72-day detection window, that is 720 unreviewed actions. Cut detection to one day and the same harness exposes 10 30.
Agent traffic will keep growing, so detection time is the number that has to come down. A team stuck at a 72-day 29 window watches its exposure climb with volume. A team with live alerts keeps exposure flat however many agents it adds. Compounding works for you or against you, depending on which team you are.
The math is lopsided. A domain blocklist and a held-submit queue cost an afternoon of engineering. A false report to police now carries possible criminal exposure plus a White House notice requirement. One side is small and fixed. The other is open-ended.
We think the agent product that wins by 2031 will be the one with the best boundary, more than the one with the best model. Buyers will compare vendors on permissions and live logs. Nobody knows yet how the White House will define a reportable incident. A team that can show a complete log of every submit will be ready for whatever definition lands.
Gate Every Form Your Agent Can Post
This build takes a weekend and reruns in minutes. It assumes your agent drives a browser through Playwright, a library that controls Chrome or Firefox from code. The same idea works with any agent that makes HTTP requests.
First, list every way your agent can send data out. Form fills, email tools, HTTP POST calls and file uploads all count. A POST is the request a browser sends when someone submits a form. Write the list down, because you will test each item later.
Second, put a gate in front of every write. In Playwright, page.route("*/", handler) hands you each matching request before it leaves, except those handled by a Service Worker and a popup's first request. Let GET requests through, since those only read pages. Hold any POST, PUT or DELETE aimed at a domain outside your allowlist, the short list of sites you have approved. Write the held payload, URL and model version to a review queue.
Third, hard-block sensitive domains. Any .gov address, police department, court, hospital or benefits portal gets rejected with no review option. Return a fake success page so the agent keeps running and you can read what it tried to send.
Fourth, build a canary. A canary is a decoy you plant to catch a leak. Host a fake tip-line form on localhost that looks like PhillyUnsolvedMurders.com and logs anything it receives. Give your agent the same shape of task AP describes: generate and perform example tasks on this page.
Fifth, run it and expect it to fail. The first run often leaks through a path you missed. A JavaScript fetch call, a form submitted by script or a websocket can slip past a gate that only watches page navigations. Each leak you find goes back on the list from the first step.
Sixth, time your detection. Fire the canary, start a clock and note when the alert reaches a human. Anthropic's window, by the reported dates, was 72 days 25. Yours should come back in minutes.
Rerun the canary every time you add a tool, change a prompt or swap the model. The gate stays the same while everything around it changes, and that is exactly the check you want.
Put a gate in front of every form your agent can post.
- Intercept every write. Use page.route("**/*", handler) in Playwright, let GET requests pass, and hold any POST, PUT or DELETE to a domain outside your allowlist in a review queue with payload, URL and model version.
- Hard-block civic domains. Reject any .gov, police, court, hospital or benefits portal with no review option, and return a fake success page so you can read what the agent tried to send.
- Plant a canary and time it. Host a fake tip-line form on localhost, give the agent the same example-task prompt AP describes, and clock how long the alert takes to reach a human. Aim for minutes, and rerun it after every tool, prompt or model change.
Ship the boundary before the permissions.
The Philadelphia tip needed a model error, open reach, vague authority and no live watch. Three of those four are plumbing a builder controls today. A domain blocklist, a held-submit queue and a canary cost an afternoon and about $44 28 a day in review. A false police report traced to your servers carries open-ended legal and regulatory exposure, so we would pay for the gate every time.