← All transmissions

8 min

The traffic did not drop. It stopped being sent.

Publisher referrals are down sharply, most searches never produce a click, and half of all AI crawler traffic is training that will never refer anyone. What that changes about how you build.

I run the infrastructure for a news portal, so I get to watch this from both ends: the analytics that say fewer people arrived, and the access logs that say plenty of things are still asking for the content. Those two facts used to move together. They do not any more.

The numbers, with a caveat

Treat every figure in this space as directional. Most of it comes from vendor studies and industry analyses with different panels and different definitions of a session, and the spread between them is wide. That said, the direction is consistent enough across sources to plan against.

  • Publisher referral traffic from Google is reported down roughly 38% year over year, with AI-driven search features cited as a 30–40% erosion across referrals generally.
  • On queries where an AI summary appears, organic click-through is reported to fall by around 61%. The ranking did not change; the click did.
  • Roughly 60% of searches globally now end without a click, and about 77% on mobile.
  • AI crawlers are now around 22% of bot traffic, second only to traditional search crawlers.
  • Of that AI crawler volume, close to half is training rather than retrieval — fetches that will never produce a referral to anybody.
  • AI-referred traffic is still tiny in absolute terms, around 1% of traffic against roughly 25% from traditional organic, but it has grown nearly tenfold in under two years.

The part that is an infrastructure problem

That fifth point is the one I have not seen discussed enough by the people who have to pay for the servers. If half of AI crawler traffic is fetching your archive to train a model, you are serving real bandwidth, real database queries and real cache pressure to a client that structurally cannot send you a reader. On a large archive that is not a rounding error. It is a line item.

Which turns a content strategy question into an operations one. You need to know which bots are hitting you, at what rate, against which paths, and what each category is actually for. Retrieval crawlers that surface you in an answer with attribution are worth serving. Training crawlers are a decision, and it should be a deliberate one rather than the default of whoever forgot to look at the logs.

  • Separate crawler traffic from human traffic in your analysis before you conclude anything about a traffic drop.
  • Distinguish retrieval from training in your bot classification. They are not the same customer.
  • Rate-limit at the edge rather than the application. A crawler that pins your database is a capacity incident regardless of its intentions.
  • Decide your robots policy on purpose, and revisit it. Blanket-blocking everything is a real option with real costs on both sides.

What still works

The reflex is to conclude that technical SEO stopped mattering. It did not — it changed jobs. Being included in an AI answer depends on being parseable, fast, and unambiguous about what you are claiming, which is the same work as before with the payoff moved.

Structured data does more now, not less: it is how a machine reads your page without guessing. Speed still decides whether you get fetched cleanly. Clear entity and section structure decides whether you can be quoted precisely rather than paraphrased into anonymity.

The strategic change is the honest one, and it is uncomfortable: a search result is no longer reliably a visit. If your entire model assumed that a ranking converts into a session that converts into revenue, the middle link of that chain has weakened and no amount of on-page work restores it. The response is not better incantations. It is owning more of the relationship — the channels where you reach someone directly, and the reasons for them to come to you rather than be shown a summary of you.

For anyone who does what I do, there is one practical upside. When traffic falls, the useful question stopped being which keyword slipped. It is now whether the click was ever offered at all — and answering that takes access to the logs, not just the dashboard.

Sources