Those are two different decisions with two different switches, and the one everybody is talking about is not the one that affects your traffic. Here is what actually changes, when it takes effect, and which build fits your business.
There Are Two Switches, and Almost Everyone Is Looking at the Wrong One
Something arrived this week that gives you a real decision about your own website, and the coverage of it has mostly been wrong.
Here is the direct answer. There are two separate controls, and conflating them is the expensive mistake.
The first is AI training: whether a company may use your content to adjust a model's parameters. Blocking this is close to free. It does not affect your search rankings and, by itself, does not remove you from AI answers.
The second is AI grounding: whether your content may be pulled in and cited when an AI answers a question right now. That is the one connected to your visibility, and Microsoft said so in plain language in its own announcement: "If you select DisallowExtended or Disallow for grounding, you will see reduced brand visibility in AI experiences."
Most of the posts you have read this week treated these as one thing. They are not. You can block training and still be cited. You can allow training and still be invisible.
And a detail nobody is reporting: the shared signal that Google, Microsoft, OpenAI and Anthropic all endorsed is not actually a standard yet. It is a ten-page individual Internet-Draft, filed October 30, 2026, and its own page on the IETF Datatracker says: "This I-D is not endorsed by the IETF and has no formal standing in the IETF standards process."
None of this takes effect until 2027. Which means you have roughly three months to make this decision on purpose rather than by default.
Key Takeaways
- AI training and AI grounding are separate controls. Blocking training costs you little; blocking grounding reduces your visibility in AI answers, and Microsoft states this explicitly.
- Pew found users clicked a search result on 8% of visits where an AI summary appeared versus 15% where none did, and clicked a link inside the summary on just 1% of visits.
- A controlled study of 252,000 trials across six models found that formatting-only edits have little impact on whether content gets cited, while relevance, recency and explicit pricing do.
- Microsoft begins honoring the shared robots.txt preference February 1, 2027, and both training and grounding default to Allow if you publish nothing.
- Silence is not an opt-out. If the policy token appears nowhere in your robots.txt, you have expressed no preference at all.
The Problem: Your Traffic Plan Was Built on a Deal That Is Being Renegotiated Without You
For twenty years the arrangement was simple and unspoken. You published useful content. Search engines crawled it, sent you visitors, and you converted some of them. Everybody understood the trade.
That deal is being rewritten right now, in public, by the parties with the most to lose, and small business owners are not in the room.
Look at what happened in a single month. Reddit's CEO reported the platform lost an average of more than one million weekly Google search referrals during October, and announced a move to a predominantly logged-in experience specifically to reduce scraping. Reporting indicates Google is struggling to make AI Mode searches pay despite enormous infrastructure spending. And OpenAI, Google and Microsoft all committed to a shared preference signal that did not exist two months ago.
When the largest community site on the internet starts closing its doors and the company that invented AI search cannot make the ad math work, that is not a story about them. That is a story about the ground your traffic strategy is standing on.
Here is where owners get stuck, and I understand why. The advice available right now is either panic or denial. One camp says block everything before the robots eat your content. The other says ignore it and keep publishing. Both are telling you to avoid a decision rather than make one.
The reason you cannot avoid it is that the two builds are genuinely different. Content written to be cited by an AI looks different from content written to rank, which looks different again from content written for an audience that already trusts you. Different structure, different length, different measurement. You can do all three, but you cannot do all three in the same piece, and trying is how you end up with content that is mediocre at every job.
Candidly, the muddled middle is where most business content already lives. This decision is just going to make that visible.
The Evidence: What the Numbers Actually Show
The click-through decline is real and independently measured twice. Pew Research Center tracked browsing data from 900 US adults who installed a monitoring app, covering 68,879 unique Google searches in March 2025, of which 12,593 produced an AI summary. Users clicked a traditional search result on 8% of visits where an AI summary appeared, versus 15% where none appeared. They clicked a link inside the AI summary on just 1% of visits. And they ended the browsing session entirely on 26% of pages with a summary, versus 16% without.
Ahrefs approached it differently, comparing Google Search Console data on 300,000 keywords between December 2023 and December 2025. They measured a 58% lower click-through rate for the top-ranking page on AI Overview keywords, worsening from 34.5% nine months earlier. By position: 58.0% at position one, 50.8% at two, 46.4% at three, down to 19.4% at position ten.
An honest caveat that Ahrefs themselves raise: their control group of keywords with no AI Overview also fell by roughly half over the same period, so AI Overviews are one cause among several. Google publicly disputed Pew's methodology as using a skewed query set. I am giving you both the numbers and the objections.
Here is the finding that should change what you do, and it contradicts most advice you have read. A paper submitted to arXiv in May 2026 built a controlled testbed: inject exactly two candidate sources into a model's context, measure which gets cited first, vary exactly one content factor between paired trials. 252,000 trials across six models, 18 content factors, with brand anonymization and counterbalanced ordering to separate content effects from position bias.
The results, in order of effect: topical relevance and list position dominate. Explicit price information and a recent timestamp help consistently. Completeness and trust cues add smaller gains.
And then this: "formatting-only edits have little impact."
I want to be careful here because it matters. I could not find any independent study establishing that schema markup increases AI citation rates. The widely repeated claim that structured data earns 42% more citations traces to vendor guides with no primary study behind it. The best-controlled evidence available points the other way on formatting. The authors of that study are affiliated with a software company, which you should factor in, though the paper carries an ACM DOI.
So if someone is selling you AI search optimization that consists mostly of adding markup, the research does not support the pitch.
What does correlate is being talked about elsewhere. Ahrefs analyzed 75,000 brands against AI Overview visibility using Spearman correlation. Branded web mentions correlated at 0.664. Branded anchors 0.527. Domain Rating 0.326. Backlinks 0.218. Brands in the top quartile for web mentions averaged 169 AI Overview mentions, versus 14 in the next quartile down. And 26% of the 75,000 brands had zero AI Overview mentions at all.
Correlation, not causation, as Ahrefs notes. But the direction is consistent: AI answers favor entities that other people discuss, not entities with the most links.
Ranking well and being quotable overlap less than you would expect. Ahrefs pulled the top 1,000 pages ChatGPT cited in September 2025. Wikipedia was 29.7%. Homepages and landing pages 23.8%. Educational pages 19.4%. Blog and article pages were 1.9%. Only 32.3% fell into categories a business can realistically influence.
Two details worth your attention. First, 28.3% of the most-cited pages had zero organic keywords, meaning no traditional search visibility whatsoever. Second, among cited pages that do rank, 65.3% sat on domains rated 81 or higher with a median of 90, while 67.3% had a page-level URL Rating between zero and ten. AI cites authoritative domains, not the most-linked pages on them.
And freshness carries real weight: of the cited URLs with detectable update dates, 76.4% had been updated within the previous month.
AI referral traffic is small and converts far better. Microsoft Clarity analyzed 1,277 publisher and news domains. AI-driven referral traffic grew 155.6% over eight months, against search at 24.0%. But AI referrals still account for less than 1% of overall traffic. On conversion in a single month: sign-up click-through was 1.66% for large language model traffic versus 0.15% for search, and subscription click-through 1.34% versus 0.55%. Fifty-two percent of those domains had already converted AI-referred traffic that month.
Microsoft operates Copilot, so treat their figures as first-party analytics rather than an independent study. Also worth knowing: Semrush notes that AI platforms do not always pass referrer information, so much AI-driven traffic lands in analytics as direct, which means your reported AI numbers are probably undercounts.
The Build: Pick One Job Per Piece, and Know Which Switch You Are Flipping
Here is how I think about this as a system rather than a tactic, because it is a system.
Start with the technical decision, since it has a deadline attached.
The shared signal is a robots.txt group keyed on the policy token User-Agent: __usage_ai-training, with Disallow: /path expressing a preference against training and Allow: expressing a preference in favor. Three mechanics will trip people up.
A wildcard group does not cover it. The draft states that User-Agent: * "MUST NOT be interpreted as matching the policy token." Your existing robots.txt rules do nothing here.
Silence is not an opt-out. The draft is explicit: if no group contains the token, "then no preference has been expressed." Both Microsoft preferences default to Allow if you publish nothing.
And the signal is not a crawl directive. The draft states the token "is not a directive for crawling behavior," and that content-usage preferences and crawl rules are independent. Allowing usage does not override a crawl block, and blocking usage does not remove you from search results.
Dates, so you can plan. Microsoft begins honoring the shared preference February 1, 2027, as it crawls, and adds page-level control via a meta tag or header, with page-level overriding robots.txt. Microsoft's grounding preference is page-level only and has a third value, DisallowExtended, which permits grounding inside Microsoft's own experiences while excluding the grounding services Microsoft sells to third parties. Google says it aims to support the control in 2027.
Two Microsoft warnings worth copying into your notes. "Publishing a setting is not the same as activating it. Microsoft must crawl the page to discover the setting." And the one that should temper anyone's expectations: "Neither choice guarantees inclusion, traffic, or payment."
Now the content decision, which is the one that actually changes your revenue.
Three jobs, three builds. A piece built to rank optimizes for a search query and a click. A piece built to be cited answers a question directly in the first 150 words, makes claims that stand alone when extracted, identifies its author clearly, carries a recent date, and includes specifics like actual prices. A piece built for an owned audience does not need either, because the person already knows who you are.
The research says pick one per piece. Topical relevance and position drove citation. Formatting did not. So a piece crammed with markup and hedged prose to satisfy three masters satisfies none.
My own read, and this is an opinion rather than a finding: most small businesses should go open on their marketing content, protected on their paid curriculum, and build the owned audience regardless. The citation traffic is tiny but converts at roughly eleven times search on sign-ups in Microsoft's data. The owned audience is the only asset in this entire article that cannot be deprecated by somebody else's policy change.
On that last point, the numbers are worth seeing. Klaviyo's benchmarks, drawn from over 183,000 customers, show automated email flows at a 5.58% click rate versus 1.69% for campaigns. Flows are 5.3% of send volume and generate nearly 41% of total email revenue. Nearly 48% of flow-driven revenue comes from new buyers, which makes owned email an acquisition channel rather than only a retention one. Klaviyo sells email software and the data is ecommerce-skewed, so read it accordingly.
Practical Steps You Can Take This Week
1. Read what AI assistants currently say about your business. Before you change anything, get a baseline. Ask three or four assistants what your company does, what it charges, and who it serves. Write down the answers. Most owners have never done this and are surprised, usually unpleasantly.
2. Sort your content by type and make an open-or-protected call on each. Blog and marketing content, free resources, paid curriculum, client work, templates. One call per type with one sentence of reasoning and the trade-off you are accepting. Do not do this all-or-nothing.
3. Decide training and grounding separately, and write the decision down. Blocking training is low cost. Blocking grounding costs visibility, by Microsoft's own statement. If you want the protection without the invisibility, that is a specific combination and you should name it before you hand instructions to anyone.
4. Rewrite your single most important page to be quotable. Direct answer in the first 150 words. Question-shaped headings. Claims that stand alone when lifted out. Clear author identification. A real date. Actual prices where you have them, since explicit price information was one of the factors that consistently helped in the controlled study.
Here is a Perfect Prompt for it:
[The Job]
Rewrite my page on [TOPIC] so an AI answering a question could cite it accurately and attribute it to me.
This is for: being the source an AI pulls from rather than a page that ranks and gets skipped.
It matters because: being summarized away and being cited are different outcomes with different structures.
[The Background]
Here is what you need to know: here is my current page, [PASTE IT]. The question a reader is really asking is [STATE IT]. My credentials on this topic are [DESCRIBE]. My business name is [NAME].
Do not use: keyword stuffing, and do not add any claim or statistic I did not give you.
[The Deliverable]
Return: a rewritten page with a direct answer in the first 150 words, clear question-shaped headings, and specific attributable claims.
Must include: a short section of plain question-and-answer pairs, each answer able to stand alone.
Optimize for: accurate citation.
[The Questions]
Ask me any questions you have.
5. Update your most important pages and keep updating them. Of the ChatGPT-cited pages with detectable dates, 76.4% had been updated within the previous month. Freshness is not a vanity signal here. Pick your five most important pages and put a recurring date on your calendar to revise them.
6. Fix what is wrong about you in AI answers, at the source. If an assistant quoted an old price or a service you no longer offer, find where that lives. Old pages, directory listings, press mentions. Correct them at the source and point everything at one canonical page. You cannot contact the AI companies about it, so this is the lever you have.
7. Build the owned audience this month, regardless of which switch you flip. One free resource that genuinely works on its own, one place for people to opt in, and a real cadence of something useful. The research consistently shows this converts better than anything else in this article, and it is the only part nobody else can turn off.
Frequently Asked Questions
If I block AI companies from training on my content, do I disappear from search?
No, and both companies say so. Microsoft states these usage preferences do not change eligibility for traditional Bing search results. The IETF draft is explicit that the token is not a directive for crawling behavior and that usage preferences and crawl rules are independent and orthogonal.
Is there one line I can add to robots.txt today that opts me out everywhere?
Not yet in force. The syntax is a group keyed on User-Agent: __usage_ai-training. Microsoft begins honoring it February 1, 2027, and Google aims to support it in 2027. The underlying document is an individual Internet-Draft with no formal standing in the IETF standards process.
How much traffic am I actually losing to AI summaries?
Pew found users clicked a result on 8% of visits with an AI summary versus 15% without, and clicked inside the summary on 1% of visits. Ahrefs measured a 58% lower click-through rate for the top-ranking page on AI Overview keywords, though their control group also fell by roughly half.
Will schema markup and FAQ sections get me cited by AI?
There is no independent evidence that they will. The best-controlled study available, 252,000 paired trials across six models testing 18 factors, found that formatting-only edits have little impact, while topical relevance, position, explicit pricing and recency drove citation. Claims of a 42% citation lift from structured data have no primary study behind them.
My blog ranks well. Is that the same as being quotable?
No. In Ahrefs' analysis of the 1,000 pages ChatGPT cited most, 28.3% had zero organic keywords, blog and article pages were only 1.9% of citations, and Wikipedia was 29.7%. Among cited pages that ranked, authority sat at the domain level rather than the page level.
The Close
The open web ran on an unspoken handshake for twenty years, and this month the parties to it started negotiating out loud.
I do not think that is a catastrophe. I think it is the first time in a long while that a business owner has had an actual switch to flip rather than an algorithm to guess at. For two decades the deal was take it or leave it, and most of us took it and complained about updates. Now there is a token you can put in a file, a page-level tag, a grounding preference with three values, and a date it takes effect. That is more control than search ever gave you.
But control only helps if you use it, and the default is already set. Both preferences default to Allow. Silence expresses no preference. If you do nothing between now and February, you have not stayed neutral. You have chosen, and somebody else wrote the choice down for you.
So here is what I would do with the three months. Read what the assistants say about you today. Decide, content type by content type, what is open and what is held back. Rewrite the one page you most want to be the source on. Then build the email list, because that is the one thing in this article that no working group, no crawler policy, and no quarterly earnings pressure can take away from you.
The handshake is being renegotiated. You have a seat at the table this time. Sit in it.
About the Author
Jonathan Mast is the founder and CEO of White Beard Strategies, where he teaches non-technical entrepreneurs how to use AI to amplify the skill and experience they already have. He runs a 500,000-member community of business owners learning AI in public, built the Perfect Prompt Framework 2.0 that this article's prompts are built on, and has spent the last several years watching owners get whipsawed by platform changes they had no say in, which is why this one matters to him.
