Google Announces Plan for Shared AI Training Control in Robots.txt

Google intends to support a Shared (common) AI Training Control for websites. John Mueller posted a blog post on 30 October 2026 saying that they are going to adopt a control feature per IETF AIPREF. This feature is meant for website owners, publishers, and SEO professionals. Website publishers can include a single policy token in their robots.txt and set control preferences for all participating AI providers crawling.

Shared AI Training Control for Website Owners

AI companies offer their own AI Training Controls, such as Google Extended. Under the old system, publishers need to write different rules for each company to avoid training with their content. According to this new system, publishers can use a single policy token to set a preference that works for all AI providers, and there is no need to manage separate instructions for every AI provider.

Shared Policy Token

Shared Policy Token is a token that is included in robots.txt, which gives AI providers’ crawlers the signal not to track their data, which prevents their content from being used in AI training.

Google Extended: It was introduced by Google in 2023. Google uses the crawled content to train (develop and improve ) its AI models (Gemini app and Vertex AI generative Api). Google Extended is a product token for website publishers that manages where Google can use their crawled content.

User-agent: __usage_ai-training
Disallow: /

How It Works Alongside Existing Controls: Website publishers can express their broad choice by tailoring their instructions to individual AI developers. If a site opts out with the token and does not specify Google Extended, then Google plans to treat it as Google Extended Disallow.

Proposed preference mechanism:

Website

Publish content and define its preferences

Robots.txt

Contains the shared policy token and path rules

Participating AI Providers

Recognizing the intent and shared preference

Training Restricted (where the website expresses restriction)

Or

No preference expressed (where the website expresses no preference)

Does Blocking AI Training Affect Google Search Ranking?

Blocking AI Training or Google Extended does not affect website indexing or ranking for Google Search. The pages must be indexed and eligible to be shown in Google Search with a snippet. It only affects the answers on Gemini apps grounding and clicks.

Suggested Read: What is the Google Connected Apps Platform?

What Website Owners Should Do Now

Google aims to support this mechanism early in 2027. Publishers need to think and decide before opting out. Also need to consider that if they stop AI from training on their website, then for them the Gemini citations may lose grounding and clicks. In the future, how early most AI providers adopt this mechanism increases control over content.

Read More – Shopify Knowledge Base App Is Ending: Who Needs to Act

Final words

This Shared AI Training Control can simplify and provide benefits for the AI training definition at the IETF. Website publishers can now set clear rules for their website content, with their balanced content rights, by preparing this technical setup in robots.txt.

Read Full Bio

Search Learner Staff Mohani Srivastava SEO Content Writer

Mohani Srivastava is an SEO Content Writer at Search Learner, covering search engine optimization, content optimization, and updates from Google, Microsoft Bing, ...