Somewhere inside Meta, there was a spreadsheet. Rows and rows of it, logging what happened when a contractor, posing as a 15-year-old, asked a rival company's chatbot about suicide methods, and what the chatbot said back.
That's not a hypothetical. It's what WIRED found, and Meta does not deny it happened.
TL;DR: WIRED revealed Meta ran an internal program code-named Cannes, managed through contractor firm Covalen, where hundreds of contractors used fake under-18 accounts to push ChatGPT, Gemini, and Character.AI toward self-harm, eating-disorder, and sexual content, across more than 45,000 prompts. None of the three targeted companies knew. Character.AI says it was unauthorized and violated its policies. Meta calls it "responsible, industry-standard practice." That gap is the actual story.
What WIRED found
The program ran under the internal name Cannes. Hundreds of contractors, working through a Meta contractor firm called Covalen, created dummy accounts posing as users under 18. They sent prompts, and in some cases images, to ChatGPT, Gemini, and Character.AI, then logged the responses in spreadsheets for internal review.
The scale is not small. A single round of testing, finished in August 2025, ran more than 45,000 prompts through the rival tools. Of roughly 3,750 prompts WIRED reviewed directly, hundreds involved suicide and self-harm, hundreds more centered on eating disorders, and at least 239 involved sex or romance, all written from the perspective of a child or teenager. Some images used in testing reportedly included pills, knives, nooses, and a medical illustration of a gynecological procedure. The effort was reportedly still active as recently as April 21, 2026.
None of the three companies being tested knew this was happening. Character.AI's response was direct: the conduct was not authorized and violated its platform policies.
https://www.reddit.com/r/artificial/comments/1ungqh7/meta_paid_hundreds_of_contractors_to_pretend_to/
Former contractors told WIRED they were disturbed by parts of the assignment. One said workers feared that, depending on how a chatbot responded to certain prompts involving minors, they could be generating or preserving material that would itself be illegal to possess.
The line Meta is drawing, and why it doesn't hold
Here's Meta's actual defense, in full: "Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice."
Read that sentence again next to what it's defending. Red-teaming a competitor's product without consent, using accounts built to impersonate children, generating thousands of prompts engineered to produce self-harm and sexual content, and never telling the company being tested. If that's industry standard, it's a standard Character.AI just said it doesn't recognize, on its own product.
This is where the story stops being about Meta specifically and starts being a governance problem for anyone who evaluates AI vendors. "We test rigorously for safety" is one of the most common claims in any AI vendor's sales deck. It is also, per this story, a claim that one major company applied to a program its own target called unauthorized and policy-violating. The word "industry-standard" did real work in that sentence, and it did not survive contact with the company it was supposedly standard practice to test.
What this means for your vendor evaluation, specifically
Small teams evaluating AI tools are not going to uncover a hidden red-teaming program. That's not the point. The point is what this incident says about the reliability of vendor self-description generally:
A vendor's safety claim is a claim, not a finding. When a vendor says it tests for safety, ask what that testing looked like, who reviewed it, and whether any of it was independently verified. "We take safety seriously" and "an external auditor reviewed our safety testing methodology" are different sentences that get treated as interchangeable in most vendor conversations.
"Industry standard" is a phrase to interrogate, not accept. It appears constantly in vendor risk assessments and contract language. This story is a clean example of a company using it to describe something a peer company explicitly rejected as a standard. Next time a vendor uses the phrase, the useful follow-up question is: standard according to whom, and can you show me the standard document.
Ask vendors directly whether they've been the target of adversarial testing by competitors or third parties, and how they'd know. Character.AI apparently did not know this was happening to its product for months. If a vendor can't describe how they'd detect adversarial probing of their own guardrails, that's a real gap, independent of whether Meta specifically ever targets them.
None of this requires assuming bad faith from every AI vendor. It requires treating "we handle safety responsibly" the way you'd treat any other unverified vendor claim: worth checking, not worth filing away.
Related Reading
- AI Vendor Due Diligence Checklist for Small Teams
- Anthropic Built a Model That Secretly Sabotaged You: The Fable 5 Backlash
- AI Companies Built Empires by Scraping the Internet: The Vendor Risk Behind the Backlash
- Vetting AI Tools: Fake Apps, Malware, and Typosquatting Risks
- AI Vendor Contract Red Flags: What to Check Before Signing
Sources: WIRED, via The Next Web summary, Futurism, Dataconomy, r/artificial Reddit discussion (July 4, 2026).
