AI search could kill the web without new quality signals and revenue models

AI answers are killing content publishers, as they cause readers to stay on Google or on the chatbot of their choice, rather than navigating to dedicated websites. The end result could be catastrophic for the open web.

Alex Chan, assistant professor at Harvard Business School, floats that prediction based on an economic model he describes in a paper titled "AI and the collapse of the www."

A paper from Saharsh Agarwal, assistant professor at the Indian School of Business, and Ananya Sen, associate professor at Carnegie Mellon University entitled "The Impact of Google AI Overviews on Publisher Traffic and User Experience: Evidence from a Field Experiment," shows that Google AI Overviews "reduce outbound organic clicks by 39.8 percent and increase zero-click searches by 34.5 percent, without affecting sponsored clicks or overall search frequency."

A zero-click search is what happens when a search user's query gets answered on the search results page, so there's no need to click through to a source website.

"Overall, the results suggest that [AI Overviews] divert traffic away from publishers without improving the user experience or quality of engagement for websites," conclude Agarwal and Sen.

Google CEO Sundar Pichai has argued otherwise, to some skepticism and despite Pew research to the contrary.

Chan, from Harvard Business School, undertook his research to explore what has to be replaced to reimagine the market that AI is destroying.

The open web, he begins, is based on a bargain: "Publishers produce content. Search and social discovery send users to that content. Visits generate revenue."

In doing so, web visitors also generate information about informational quality through their clicks, subscriptions, and other interactions. These signals, he observes, help future visitors and search systems find quality sources.

"Generative AI changes this bargain," Chan continues. "An AI answer can use publisher content while keeping the user in the AI interface. Users may be better served in the short run. But the source may lose the visit, the revenue from the visit, and the source-level signal that the visit would have produced."

Chan's exploration of the topic is noteworthy because it looks beyond the revenue impact of diminishing visitor traffic – something industry players like Cloudflare are already trying to address. It also identifies other economically valuable signals lost to AI answer systems, what he refers to as "durable attention capital." This includes: subscribers, repeat readers, backlinks, bookmarks, reputation, and search authority.

Chan characterizes his argument as a more disciplined version of the AI-will-make-the-web-collapse scenario. He's not saying every website will vanish, that all AI reduces the diversity of information, or that search is doomed. Rather, he contends, "when an AI platform diverts the revenue and measurement events without replacing them, costly human information may fall below replacement."

Really, though, that is the web collapse scenario. AI answers make it uneconomical for people to produce content and their disincentive in doing so prevents the formation of quality and trust signals that add value to their work.

To restore a functioning market, Chan says, the answer shouldn't be a "visitor replacement royalty" that AI search services pay to websites or a ban on AI answers. That just props up the traditional model. Instead, he argues that the focus should move toward the new point of attention – which for some queries may be AI answers – while finding a way to distinguish between costly human information and cheap AI imitation. 

He concludes, "Provenance, diversity prices, exploration credits, and informative audits are needed to restore the broader ecosystem: quantity, quality, diversity, source-level signals, and the conventional-search discovery channel that prevents self-reinforcing migration into AI answers."

That may be more difficult than it sounds. Distinguishing between AI- and human-authored content can be quite challenging. And to the extent that there's potential profit in selling low-cost AI output as premium human-authored content, expect active resistance to labeling or other mechanisms that threaten slop arbitrage. ®