Skip to main content

Meta Commercial Terms: Two Versions of the AI Crawl Clause

Meta says it narrowed the crawling clause after the backlash, but its own preview pages don't agree with each other, which is reason enough to set your robots.txt before the terms bind.

VerdictIf your business runs a Facebook Page, an Instagram business account or Meta ads, the new terms bind you on October 30 just because you keep using them. Save a dated copy of the edition you're shown, and if you don't want your site feeding Meta's AI, block Meta-ExternalAgent in robots.txt this week.

6 min readBy Johnie T YoungLast reviewed
Meta Commercial Terms: Two Versions of the AI Crawl Clause

Image: Pexels, used under the Pexels License.

Meta told reporters on October 17 that it had taken the word "train" out of the crawling clause in its new Commercial Terms. When we opened Meta's own preview of those terms on October 19, that was true of one edition and not the other. The US English page has the narrower wording. The UK English page, the one Facebook served us by default, still has the original sentence word for word.

If you run a business on Facebook, Instagram or Meta ads, you don't get to pick: the terms say continued business use after notice means you agree. Here's what each edition says, who it covers and what you can do before the effective date.

Event date: October 17, 2026 (Meta revised the crawling clause after criticism). First reports: October 17, 2026, by The Verge and Nieman Lab; TechCrunch followed the same evening.

What Meta changed, and where it didn't

The story started with Nieman Lab on October 15, where legal experts read the new clause as letting Meta scrape any website "associated with" a business account and use the content to train its AI. Andy Stone, Meta's vice president of communications, called that reading false the next day.

On October 17 Meta revised the text. According to Nieman Lab's follow-up, the company removed "train" from the list of allowed uses and limited those uses to "advertising and business products." A Meta spokesperson told The Verge: "We've heard you and want to clarify: the crawling terms we consolidated only applied to our ads and business products, and we've clarified that in the terms themselves."

A day later, PPC Land reported that the UK English preview it captured on October 18 still held the old sentence. We checked both language editions of Meta's preview page on the morning of October 19 (UTC). They still disagree.

Edition of the preview What Meta may do with crawled content Says "train"? Checked
English (US) "provide, develop, improve, research, and maintain advertising and business products" and "maintain the integrity and security of Meta Company Products" No Oct 19, 2026
English (UK), shown by default "Meta may use any content from such domains, web pages or URLs to provide, develop, improve, research, train and maintain the integrity of, the Meta Company Products" Yes Oct 19, 2026
Both Crawl any domain "associated with your products or services," using "test credentials" if needed; neither mentions robots.txt n/a Oct 19, 2026

Meta Commercial Terms preview, English (UK) edition, still saying Meta may use crawled website content to train Meta Company Products, captured October 19, 2026

Screenshot: Meta Commercial Terms preview, English (UK) edition, Meta. Captured October 19, 2026.

Meta Commercial Terms preview, English (US) edition, limiting crawled content to advertising and business products, captured October 19, 2026

Screenshot: Meta Commercial Terms preview, English (US) edition, Meta. Captured October 19, 2026.

Who the terms cover

More businesses than the headlines suggest. The preview defines business or commercial purposes to include "using ads or marketing messages, selling or listing products, developing apps," "managing a Page," and "using any business product, such as Ads Manager or Meta Business Suite." A two-person shop with a Facebook Page counts.

The update clause does the rest. Continuing "any access or use of any Meta Company Products for business or commercial purposes after any notice of an update" means you're bound, and the only exit the text offers is this: "If you do not agree to the updated terms, you may stop all access or use of our Products for business or commercial purposes." If you or your business is in the US, both editions also send most disputes to binding arbitration on an individual basis, with no class actions or jury trials.

Even the narrower US wording worries lawyers. "I think 'develop' is broad enough to include 'train,'" Matt Topic, a media and IP attorney at Loevy & Loevy, told Nieman Lab. James Grimmelmann of Cornell Law School said the "develop...advertising and business products" language is "still broad enough to drive a truck through."

What robots.txt does and doesn't do

Meta's spokesperson told Nieman Lab that content owners "still have the ability to disable the crawler via robots.txt flags." That promise sits in a press statement, not in the contract: the word robots.txt appears in neither edition.

Meta's web crawlers page is where the controls live. It names a crawler for each job, and two of them can ignore your file.

Meta's web crawler documentation listing the Meta-ExternalAds, Meta-ExternalAgent and Meta-ExternalFetcher user agents and what each crawls for

Screenshot: Meta web crawlers documentation, Meta for Developers. Captured October 19, 2026.

  • Meta-ExternalAgent crawls "for use cases such as training foundation AI models or improving products by indexing content directly." This is the one to block if training is your concern.
  • Meta-ExternalAds crawls "for use cases such as improving advertising and other business-related products and services," which matches the wording of the US edition.
  • Meta-ExternalFetcher "may bypass robots.txt rules" because it fetches links a user asked for, and FacebookExternalHit "might bypass robots.txt when performing security or integrity checks."

Meta says crawlers may cache robots.txt for up to 24 hours, so a change you make today takes about a day to bite. And a robots.txt rule is a request, not a right. Benjamin Sobel, a University of Wisconsin law professor, told Nieman Lab: "You can still express the preference, and unless it's a legally enforceable preference, it can still be ignored all the same."

What to do before the terms take effect

  1. Save both editions. Print the preview page to PDF in English (US) and English (UK) with today's date (the language switcher is in the page footer). If the text changes again, you'll have what you agreed to.
  2. Block the training crawler. Add this to robots.txt on every domain linked from your Page, catalogue or ads:
User-agent: meta-externalagent
Disallow: /
  1. Think before blocking Meta-ExternalAds. Meta ties it to advertising products. If you rely on Meta ads or a product catalogue, test the change on one domain first; Meta's page doesn't say what blocking it does to ad delivery.
  2. Check your server logs for meta-externalagent, meta-externalads and meta-externalfetcher so you know what's already visiting.
  3. Flag the "test credentials" wording to whoever handles your paywall or member area. Grimmelmann told Nieman Lab it could technically let Meta sign up as a user to get past a registration wall, and Meta wouldn't say whether it would.
  4. Log it as a vendor change. Meta is now an AI vendor that sets its own terms by notice. Our AI vendor contract red flags list covers the same "continued use means consent" pattern.

What we could not confirm

Neither edition says which language version controls, and Meta hasn't published a changelog for the October 17 revision. We couldn't tell whether the UK English page is lagging or meant to stay as it is, or which edition applies to a business outside the US and UK.

Our take

The US revision is a real narrowing on paper, and Meta deserves credit for moving within two days. But a business reading the page it's served can still see the old sentence, and the promise that matters most (robots.txt is respected) lives only in a press statement. We'd save both editions, block Meta-ExternalAgent today, and treat the new terms the way we treat any AI vendor clause that changes by notice: assume the broadest reading until the text says otherwise.

Community reaction

On r/technology, the highest-voted reply to the Nieman Lab story was short: "Forced consent doesn't seem like consent," wrote u/Flashy_Worry7792. Not everyone agreed. "ok that has always been allowed this is such a non-story how do people think google search works??" replied u/Avimox, a comment the thread voted down.

Legal disclaimer

This article is published for informational and educational purposes only. It does not constitute legal, regulatory, or professional compliance advice and should not be relied upon as such. AI governance requirements vary by jurisdiction, industry, and organizational context. Always consult a qualified legal or compliance professional before implementing policies or making decisions with regulatory implications.

About the author

Johnie T Young

Governance & Regulation Desk

Covers AI laws, regulator orders and court rulings, and what each one means for a team of ten. Posts start from the statute or the order itself, with every fact traced to a row in a source log.

  • Starts from the statute or order
  • Every fact traced to a source row
  • Dated checks on each post