Google, OpenAI, Microsoft, Anthropic Back Shared AI Training Opt-Out
OpenAI, Google and Microsoft each said on Friday, October 30, that they will honor a single robots.txt signal letting website operators refuse the use of their content for generative AI training across participating providers. The signal is defined in an Internet-Draft, draft-kelley-aipref-training-preference-signal, submitted to the Internet Engineering Task Force's AI Preferences working group, known as AIPREF, by Kevin Kelley of Anthropic.
For publishers, it replaces a growing list of company-specific crawler rules with one instruction. Microsoft describes it as a simpler, common way for web publishers to state training preferences in robots.txt.
Inits announcement, OpenAI said the signal will cover the use of website content to train general-purpose generative AI models, that it plans to implement it in the coming months, and that its existing controls will stay in place.
Google said in a post on its Search Central blog that the signal "will help websites express their AI training preferences across multiple AI providers in a single step," and that it will support it in early 2027. Google said site owners can still give it individual instructions through its existing Google-Extended control.
Microsoft went further. According to Microsoft's Bing blog, it is extending the training preference to individual pages, where a page-level setting overrides the site-wide robots.txt rule, and adding a separate AI Grounding preference that can only be set page by page. Microsoft says the new preferences won't affect eligibility for Bing search results, and it is asking publishers to set them before March 1, 2027. Once it detects the new preferences on a site, Bing will ignore any NoArchive and NoCache settings left on that site.
The signal is a request, not a technical block. An earlier AIPREF vocabulary draft states that it covers only expressing preferences, without enforcement mechanisms or client authentication. None of the companies said how compliance would be verified, and the announcements do not address content already used to train existing models. Google and OpenAI did not say whether they will add a separate grounding control like Microsoft's.
The draft is still under discussion at the IETF and has not been approved as a standard.
Publishers are already pressing AI companies over how their content is used, including USA Today Co.'s copyright suit against OpenAI this month, as AI Briefing News reported.
