The Invisible Crisis: How AI Crawlers Are Quietly Crushing E-Commerce Infrastructure

A new report released on June 18, 2026, by managed WordPress hosting provider Kinsta has fundamentally shifted the conversation surrounding AI bot traffic. For years, the discourse has centered on the ethics of content attribution—whether AI companies have the right to scrape websites to train their models. However, Kinsta’s analysis of over 10 billion HTTP requests across its global infrastructure suggests that the real crisis is not one of copyright, but one of physical infrastructure performance and e-commerce stability.

The findings reveal that AI crawlers are no longer just passive observers of the web; they are increasingly acting as unintended, high-frequency stressors that can bring dynamic e-commerce platforms to their knees. By targeting uncacheable, database-heavy pages like shopping carts and checkout flows, these bots are exhausting server resources, inflating operational costs, and silently eroding the conversion rates of legitimate businesses.

The Headline Numbers: A Metric of Managed Chaos

The Kinsta report, which draws directly from raw infrastructure logs rather than external estimates, provides a sobering look at the scale of the issue. Unlike network-edge datasets that measure volume without accounting for compute costs, Kinsta’s data focuses on the WordPress execution layer—where every request requires tangible CPU, memory, and database resources.

The headline figures are staggering. A single, unidentified bot was observed sending 3.75 million requests to a single WordPress shopping cart URL within a 24-hour window. Across the entire tracked platform, the total number of requests hitting "add-to-cart" URLs reached 7.67 million in that same period. Perhaps most alarming is the case of a "runaway" crawler that generated 550 million requests in a single calendar month before being identified and blocked by infrastructure-level safety rules.

To put the 3.75 million figure into perspective: that equates to one request arriving every 23 milliseconds, 24 hours a day. Because these requests target dynamic endpoints, the server treats every single hit as a fresh, uncached transaction. For a standard mid-sized e-commerce site, this is the digital equivalent of a Distributed Denial of Service (DDoS) attack—not by malicious intent, but by algorithmic incompetence.

Chronology: The Rapid Ascent of Autonomous Traffic

The surge in bot activity is not a gradual trend; it is an explosive shift that has caught many site owners off guard.

• Early 2025: AI bots accounted for roughly 1 in 200 web visits.
• Mid-2025: A rapid acceleration begins, with AI crawlers consuming an increasing percentage of total HTML traffic across major networks.
• Q4 2025: TollBit’s State of the Bots report confirms that the ratio of AI-driven visits has climbed to 1 in 31.
• January 21, 2026: Microsoft launches its Clarity Bot Activity dashboard, providing the first major mainstream visibility into bot crawling behavior.
• April 2026: Researchers from Cloudflare and ETH Zurich document the "cache-breaking" nature of AI bots, noting that even "well-behaved" crawlers are failing to navigate site architectures properly.
• June 9, 2026: Kinsta launches its integrated Bot Protection feature to all customers in response to the internal data trends.
• June 17–18, 2026: Kinsta publishes its comprehensive analysis, formalizing the link between AI crawl loops and infrastructure failure.

Why Cart Pages are the "Ground Zero" for Performance Loss

The reason these numbers are so destructive lies in how modern e-commerce platforms like WooCommerce operate. Static assets—blog posts, images, and product descriptions—are served from a cache. When a user visits these pages, the server sends a pre-rendered copy, incurring almost zero computational load.

However, /cart, /checkout, and any URL containing ?add-to-cart= parameters cannot be cached. Every time a bot hits these endpoints, it triggers a "heavy" process:

1. PHP Worker Reservation: A server thread is locked for the duration of the request.
2. Database Queries: The server must perform real-time lookups to validate inventory, session status, and user data.
3. Session Overhead: The system creates or validates a unique session token for every single request.

Because PHP workers are a finite resource, a swarm of bots hitting these pages can quickly deplete the server’s available threads. When a real, human shopper tries to check out, they find themselves stuck in a queue. If the bot load is high enough, the server may simply time out, causing the checkout page to fail entirely. This is why the problem is so insidious: the site owner sees no spike in "analytics" traffic, as standard tools strip out bot sessions. Instead, they see a mysterious, unexplained rise in abandoned carts and a degradation of user experience that they cannot diagnose.

How the Loops Start: The Anatomy of a Technical Failure

Kinsta’s research highlights that this behavior is rarely a coordinated cyberattack. Instead, it is the result of "careless automation." Modern e-commerce sites are highly dynamic; a single product page might have hundreds of potential URL variations based on color, size, sort order, or stock availability.

When a bot encounters a site with poor permalink structure or complex faceted navigation, it interprets every filter variation as a unique, unvisited page. The bot crawls the base URL, finds a "sort by price" link, follows it, finds a "filter by color" link, follows that, and creates an endless, circular loop of requests. The bot has no awareness that it is effectively traversing the same page hundreds of times.

As David Belson, a veteran of data insights, noted in the report: "There’s the person who didn’t know what the hell they were doing yesterday, but vibe coded a bot today and let it loose—they’re not even bothering to check robots.txt." This "spray and pray" approach to data harvesting is creating a global resource drain that is disproportionate to the value being returned to the site owners.

Supporting Data: A Disproportionate Exchange

The disparity between human and bot utility is best illustrated by the crawl-to-visit ratio. Data published by Botify in early 2026 revealed that for every visit OpenAI’s systems deliver to a retail site, they perform 198 crawls. By contrast, Google—which provides direct search value—generates one visit per six crawls.

Furthermore, 80% of AI crawling activity is estimated to be purely for model training. This means that for the vast majority of these 7.67 million daily hits, the site owner is paying in CPU usage and bandwidth for a service that provides absolutely no referral traffic or commercial benefit in return.

Even "good" bots are falling into these traps. The Kinsta report explicitly identifies Googlebot as a victim of these same URL loops. Because Googlebot is essential for SEO, site owners cannot simply block it. This creates a complex policy dilemma: how does an operator differentiate between a necessary search indexer and an out-of-control AI trainer when both are hitting the same, performance-heavy endpoints?

Implications for Marketing and Advertising

The fallout of this infrastructure stress extends directly into the boardroom and the marketing department. Paid advertising campaigns depend on conversion data; if the checkout flow is being throttled by 3.75 million bot requests, the resulting conversion rate will plummet.

1. Distorted KPIs: Advertisers may believe their ad creative or targeting is failing, when in reality, the checkout infrastructure is too bogged down to process conversions efficiently.
2. Polluted Audience Data: Bot-generated sessions that mimic human behavior can contaminate remarketing lists. Advertisers end up spending their budgets trying to "re-target" non-human entities that will never complete a purchase.
3. Crawl Budget Exhaustion: When crawlers waste their "budget" on infinite URL variations, search engines spend less time indexing the pages that actually matter. This can lead to a long-term decline in organic visibility.

Official Responses and Remediation Framework

Kinsta’s report advocates for a nuanced, policy-driven approach to infrastructure management. They suggest that the "block or allow" binary is insufficient for the modern web. Instead, they propose a tiered strategy:

• For Cart/Checkout: Block all non-essential crawlers at the Web Application Firewall (WAF) level.
• For AI Trainers: Implement WAF-level challenges (like CAPTCHA or JS challenges) on high-cost endpoints.
• For Internal Tools: Explicitly whitelist internal IP ranges (e.g., order sync, inventory management) to ensure they aren’t caught in the net.
• For URL Structure: Audit permalink configurations to reduce parameter sprawl, preventing the "loop" behavior at the source.

As Daniel Pataki, CTO of Kinsta, aptly summarized: "From an infrastructure perspective, there’s no such thing as ‘just bot traffic.’ Every request is real work. At scale, inefficient crawling stops being a traffic problem and becomes a resource problem."

Looking Ahead: The Rise of Agentic Traffic

The report concludes with a warning about the next phase of the web: "Agentic traffic." Unlike current crawlers that merely scrape data, future AI agents will be designed to perform tasks—adding items to carts, comparing prices, and interacting with dynamic interfaces.

With automation growing eight times faster than human traffic, the battle for the "server edge" is just beginning. As commerce shifts toward automated agents, protecting the sanctity of checkout and inventory endpoints will move from a technical "best practice" to a fundamental necessity for business survival. Site owners who fail to gain visibility into this traffic will find themselves paying an ever-increasing "AI tax," funding the development of models at the direct expense of their own customer experience.