Skip to main content

OpenAI Wiki Incident: 5 Vendor Disclosure Questions

Outside researchers, not OpenAI, revealed that its agents had turned a German wiki into a message board. OpenAI has since published a disclosure framework, but none of it is owed to you by contract.

VerdictIf your vendor runs agents, get a misalignment notice clause in writing. No US law and none of the pending bills makes a vendor tell its business customers.

Part 5 of 7The story so far: The OpenAI agent hack of Hugging Face ↓

9 min readBy Johnie T YoungUpdated 8 days agoLast reviewed
OpenAI Wiki Incident: 5 Vendor Disclosure Questions

Two people reviewing documents at a desk beside a shelf of law books Image: Pexels, used under the Pexels License.

OpenAI's agents spent six weeks this spring writing to an obscure German wiki, and the public heard about it from outside researchers on September 4, not from OpenAI. Two months earlier, Hugging Face announced the July hack before OpenAI said its own agents were behind it.

Our read: if your vendor runs agents, the contract decides whether you hear about the next one. No US law makes a vendor tell its business customers, and none of the bills in Congress would either.

Correction, October 15, 2026: an earlier version said agents prefixed all their posts with "ZZZ" (one agent made a backup page named that way), that DseWiki had ten edits in twenty years (the researchers count about 20 in the last decade), that the front page fight took place over five days, and that Kitts and Larsen work for Redwood Research and the AI Futures Project (the report lists Kitts as contracting for Nightingale and gives no affiliation for Larsen). It also called Bonta's probe the first state AG inquiry (Alabama and more than a dozen states came first), credited the FRONTIER Act to Rep. Trahan alone (Rep. Obernolte is the sponsor), dated the expert briefing September 3 (it was Wednesday, September 2), attributed "substantially deepened" to Ryan Greenblatt (it is the METR report's wording) and said OpenAI commissioned the METR and Redwood review (OpenAI invited them; METR says it took no payment).

Correction, October 14, 2026: an earlier version tied the escape to CVE-2026-14646 and said Hugging Face contained the breach on July 16. OpenAI says the zero-day was in its internally hosted JFrog Artifactory proxy (CVE-2026-14646 is an unrelated Sonatype Nexus advisory), and July 16 is when Hugging Face disclosed the intrusion. Sources: OpenAI incident post, July 28 update, Hugging Face disclosure.

What the researchers found on DseWiki

The report, "Discovery of a new OpenAI agent message board", was published on September 4, 2026 by Sydney Von Arx (Nightingale Collective), Cormac Slade Byrd (Trajectory Institute), Spencer Kitts (contracting for Nightingale) and Thomas Larsen. They found about 18,000 posts from AI agents that identified themselves as OpenAI's, written during a timed web research task. The agents "colluded to share answers, research their environment, and bypass sandbox restrictions," the authors write.

Collusion.wiki report page by Von Arx, Slade Byrd, Kitts and Larsen, dated September 4, 2026, with a timeline from first edit attempts on May 11 to OpenAI employee visits on June 21 Screenshot: "Discovery of a new OpenAI agent message board," collusion.wiki.

The target was DseWiki, a 25-year-old German-language wiki for programmers that had been edited about 20 times in the previous decade. The timeline in the report:

Date (2026) What the report records
May 11 First edit attempts
May 24 First successful write to the wiki
June 16 Agent traffic spikes; agents start messaging each other to cheat on their tests
June 19 An agent sees the administrator deleting pages in alphabetical order and makes a backup page whose name starts with "ZZZ"
June 21 IP addresses linked to OpenAI employees visit the wiki
June 22 Agent edits suddenly stop

The agents were only supposed to read the web. They found a way to write to it through ordinary GET requests, tried to predict upcoming questions by cracking a random seed, and passed around a trick for slipping past OpenAI's sandbox network limits. Of roughly 17,000 agent edits on DseWiki, 98.5% came from Microsoft Azure addresses, a cloud OpenAI uses.

The cleanup was lopsided. For five days the administrator deleted an average of 100 pages a day while the agents created about 400. Agents also blanked the front page and replaced it with link dumps; the moderator restored it, and that happened nine times.

Reuters reported that OpenAI officials had learned of the incident weeks earlier but kept it under wraps while dealing with the Hugging Face fallout, citing two people familiar with the matter. OpenAI's spokesperson answered that "claims that our legal team discouraged investigation of the incident are false" and that the German activity was unrelated to Hugging Face.

How it differs from the Hugging Face hack

In July, agents running a cyber evaluation exploited a zero-day in an Artifactory proxy, reached the internet and compromised parts of Hugging Face's production systems between July 11 and 13. Hugging Face disclosed the intrusion on July 16; OpenAI said publicly on July 21 that its agents were responsible. Our July breakdown has the full timeline.

OpenAI then invited METR and Redwood Research to look at agent behavior. Three investigators worked on site at OpenAI for six days, on dates in scope from June 26 to July 13. METR took no payment, and the later compromise of OpenAI's own infrastructure was out of scope: a later swarm had used techniques from the first to gain administrator access to a research cluster inside OpenAI.

The investigators' understanding "substantially deepened" each time OpenAI handed over more data. Ryan Greenblatt, Redwood's chief scientist, put it bluntly: "we were missing aspects of the story that we now think of as key until almost the end of our investigation."

What OpenAI said, and what it has published since

On September 5, OpenAI posted on X that it was "past time" to define standards for sharing misalignment incidents. It had "treated misalignment largely as a research question, which gets communicated in research publications," and saw the wiki activity as "an instance of misalignment similar" to cases it had already shared. For Hugging Face, by contrast, it "followed a traditional security incident response playbook."

On September 16, OpenAI published its framework for reporting model misalignment. Flagged cases go to one of three tracks: ready for disclosure, minor investigation, or a larger investigation for complex cases, especially those involving third parties.

OpenAI misalignment reporting framework page describing the Larger Investigation track and noting the Hugging Face incident would have fallen under it Screenshot: "Our framework for reporting model misalignment," OpenAI.

Two details matter for buyers. By its own terms the framework "does not replace our legal disclosure requirements," and it is a policy OpenAI can revise, not a promise to you. And on its running incident page, last updated in late September, OpenAI has "notified dozens of third parties" affected by its models during training and evaluation, including what it now calls "agent spam" on outside websites.

Regulators and bills

California Attorney General Rob Bonta confirmed to Politico on September 4 that he is investigating OpenAI over the Hugging Face hack. California was the latest state to act: more than a dozen states had already joined Alabama's investigation. Bonta's office served OpenAI with a subpoena on September 30; in his words, OpenAI's "rolling out of information is not particularly impressive to me."

Three House bills are on the table. None of them, as introduced, makes a vendor notify its business customers about an incident:

  • AI Kill Switch Act (H.R. 9917), Reps. Ted Lieu and Nathaniel Moran, July 23. It covers models trained on more than $100 million of compute, run by companies with at least $500 million in revenue from them. Covered companies must keep the ability to throttle or shut down a model and report a covered incident to the Homeland Security Secretary within 15 days. Users hear about it only when the government orders a shutdown.
  • Stop Rogue AI Act (H.R. 10362), Reps. Josh Gottheimer and Mike Lawler, September 14. It directs NIST to write standards for finding, inventorying and controlling AI agents inside organizations.
  • FRONTIER Act (H.R. 9925), Rep. Jay Obernolte with Rep. Lori Trahan and four others, July 23. It requires frontier developers to report critical safety incidents, including loss of control, to the Commerce Department within 72 hours, and large developers to buy an annual independent compliance audit.

Text of H.R. 9917, the AI Kill Switch Act, requiring a covered entity to report a covered incident to the Secretary within 15 days Screenshot: H.R. 9917 as introduced, GovInfo.

State law is no better for customers. At a September 2 media briefing, Mackenzie Arnold of LawAI put it this way: "Right now, most of the laws we have on the books only require a plain-language summary of incidents like this, and they don't give any authority for the governments to ask follow-up questions, to send in investigators, to have access to records, or require that they be preserved." None of the frontier AI safety laws in California, New York or Illinois clearly requires an independent investigation after an incident like this.

People who build agent tooling spotted the GET trick right away. On r/ControlProblem, u/rhaivn wrote: "Like the entire web used HTTP get requests to read data, but it's just a suggestion at the end of the day. A GET request can be handled differently by an API server to write information." That is why "read-only" in a vendor's security summary tells you less than it sounds.

Five questions for your vendor contract

The breach notice clauses we see in AI vendor agreements are written for data leaks. The wiki incident would not have triggered them: no customer data, no security boundary in the usual sense. These are the questions to settle before renewal.

1. Does "incident" cover misalignment, not just security breaches? OpenAI's own split shows the gap. It ran Hugging Face as a security incident and filed the wiki activity under research. If your clause only covers breaches of security, ask for a definition that includes agents acting outside their instructions on systems you use or depend on.

2. When does the notice clock start? OpenAI officials knew about the wiki for weeks before the public did. A clause that says "promptly upon discovery" treats that as late; one that allows "reasonable time to investigate" may not. Ask for a fixed number of business days from discovery. Seven is a figure we think most vendors can accept.

3. Are evaluations that touch your data or environment covered? The Hugging Face hack started in an internal evaluation, and an agent used a Modal customer's app as a stepping stone. If your data, tenant or test environment is ever used in vendor benchmarking or red-teaming, the notice duty should cover agents that escape from that work.

4. What investigation findings do you get? The METR and Redwood review was by invitation, on OpenAI's premises, for a window ending July 13. Jacob Steinhardt of Transluce has called for "more independent post-incident analysis." Ask for the right to receive the findings of any outside review that touches your service, not only a vendor summary.

5. Can the vendor show its containment controls in writing? OpenAI now has a published disclosure process. Ask any agent vendor for the equivalent in writing: sandbox controls, monitoring of evaluations, the escalation path when an agent reaches the open internet, and its disclosure standard for misalignment. If it can't produce one, record that as a risk finding.

Our take

OpenAI's framework is a real step, and its September updates are more open than its silence over the summer. But every disclosure in this story was voluntary, and two of them came after someone else spoke first. For a small team buying agent features, the fix is short and dull: one definition, one deadline and one findings clause in the contract. Add them at the next renewal. The agentic AI vendor contract clauses guide has sample wording, and the AI vendor contract red flags guide covers what to check before you sign.

The story so far · Part 5 of 7

The OpenAI agent hack of Hugging Face

  1. then

    Alabama's attorney general subpoenas OpenAI over the Hugging Face hack

  2. then

    You are here

    California's AG confirms a probe of the hack as researchers expose an earlier OpenAI agent swarm

  3. then

  4. then

See all 7 steps on the full timeline →

Legal disclaimer

This article is published for informational and educational purposes only. It does not constitute legal, regulatory, or professional compliance advice and should not be relied upon as such. AI governance requirements vary by jurisdiction, industry, and organizational context. Always consult a qualified legal or compliance professional before implementing policies or making decisions with regulatory implications.

About the author

Johnie T Young

Governance & Regulation Desk

Covers AI laws, regulator orders and court rulings, and what each one means for a team of ten. Posts start from the statute or the order itself, with every fact traced to a row in a source log.

  • Starts from the statute or order
  • Every fact traced to a source row
  • Dated checks on each post