AI bots are inflating your analytics - here's what to do about it

If your website traffic spiked in the last year and you're not sure why, there's a good chance AI bots are the reason.

Since 2024, AI companies have dramatically increased the volume of web crawling to train and update their large language models. GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended, and dozens of others are visiting websites at scale - and most analytics tools count them as real visitors.

The scale of the problem

Industry estimates suggest that AI bot traffic now accounts for anywhere from 15% to 40% of total web traffic, depending on the type of site. Content-heavy sites, blogs, documentation, and news outlets are hit hardest. If your analytics don't distinguish between bots and humans, your data is fundamentally misleading.

This matters because you're making decisions based on that data. Inflated pageview counts distort your understanding of real user engagement. Bounce rates become unreliable. Time-on-page metrics lose meaning. Campaign performance looks different than it actually is.

How AI bots affect specific metrics

Pageviews and unique visitors

AI crawlers hit many pages in quick succession. A single bot session can generate hundreds of pageviews, inflating your numbers significantly. Because most bots don't execute JavaScript, some client-side analytics tools miss them entirely - but server-side and hybrid tools will count every hit.

Bounce rate

Bots typically request a single page and move on, which looks identical to a bounce. If bot traffic represents a significant portion of your visits, your bounce rate will appear much higher than your actual human bounce rate.

Geographic data

Many AI bots operate from data centres concentrated in the US (Virginia, Oregon) and parts of Europe. If you see unexpected spikes from these regions, bots may be the cause.

Referral data

Bot traffic rarely carries referral information, so it typically shows up as "Direct" traffic. A growing proportion of direct traffic with no corresponding marketing activity is a common indicator of bot inflation.

What you can do

1. Use analytics that separate bot traffic

This is the most effective solution. Cabin has a dedicated AI Crawlers section in the dashboard that shows you exactly what percentage of your traffic comes from AI bots versus real humans. You can see which bots are visiting, how much traffic they generate, and view your human-only metrics separately.

2. Monitor your server logs

If you have access to server logs, look for user-agent strings containing identifiers like GPTBot, ClaudeBot, Googlebot, Bytespider, CCBot, and Applebot-Extended. The volume may surprise you.

3. Use robots.txt wisely

You can block specific AI crawlers using your robots.txt file:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

Be aware that not all bots respect robots.txt, and blocking them means your content won't appear in AI-generated responses. That's a trade-off worth considering carefully.

4. Review your data critically

If your analytics don't offer bot filtering, apply scepticism to any metric that changed significantly without a clear cause. Cross-reference traffic spikes with server logs before drawing conclusions.

The bigger picture

AI web crawling isn't going away. If anything, the volume will increase as more companies build and maintain AI products. The question isn't whether your site is being crawled - it's whether your analytics are honest about it.

Most analytics platforms were built before the AI crawling explosion and haven't adapted. They either count bot traffic as real visits or ignore it entirely, giving you no visibility into the problem.

At Cabin, we built AI bot detection directly into the analytics dashboard because we believe your data should reflect reality. You deserve to know how many real humans are visiting your site, and you deserve to understand the AI traffic separately.

Getting started

Cabin's AI Crawlers feature is available on all plans, including the free tier. Sign up and add a single line of code to your site to start seeing the real picture of your traffic.