Plenty is claimed about AI crawlers and very little is shown. Anyone running a site can see in their own logs which systems actually arrive, what they take and how often — and almost nobody publishes it. Here is one day of it, with the query that produced it.
AI systems made more requests than search engines did — 299 against 282. The search engines pulled more data (4.0 MB against 2.1 MB), because Googlebot fetches images and stylesheets that a language model has no use for.
This domain was registered on 1 August 2026. It is two days old, ranks for nothing, and has no inbound links worth counting — and seven distinct AI systems have already read it.
| System | Requests | Data | Share |
|---|---|---|---|
| ClaudeBot (Anthropic) | 135 | 795 kB | 45 % |
| GPTBot (OpenAI) | 96 | 798 kB | 32 % |
| PerplexityBot | 22 | 213 kB | 7 % |
| CCBot (Common Crawl) | 16 | 69 kB | 5 % |
| ChatGPT-User (OpenAI, im Auftrag eines Menschen) | 15 | 152 kB | 5 % |
| OAI-SearchBot (OpenAI) | 9 | 43 kB | 3 % |
| Amazonbot | 6 | 45 kB | 2 % |
| Crawler | Requests | Data | Share |
|---|---|---|---|
| Googlebot | 134 | 2658 kB | 48 % |
| YandexBot | 130 | 1306 kB | 46 % |
| Bingbot | 10 | 78 kB | 4 % |
| Applebot | 8 | 53 kB | 3 % |
YandexBot at 130 requests is within reach of Googlebot, which is not the ratio most sites see. Bingbot at 10 is the surprise in the other direction.
Separated out rather than counted: requests carrying an AI user agent that went straight for files nobody links to.
| User agent claimed | Requests | Paths sought |
|---|---|---|
| Bingbot | 5 | /.env.prod |
| PerplexityBot | 5 | /terraform.tfvars |
| OAI-SearchBot (OpenAI) | 5 | /.git/HEAD |
| ChatGPT-User (OpenAI, im Auftrag eines Menschen) | 5 | /keys.json |
A user agent is not an identity. It is a string the client chooses. Whether these came from the named operators or from someone borrowing their name cannot be determined from this side, and this page does not claim to know. What can be said: they were not reading, all of them got 404, and nothing they sought exists here.
The most-requested paths across all of them are the machine-readable ones —
/sitemap.xml, /llms.txt, /mcp — before
the articles. That is the whole argument for maintaining those files: they are
not decoration, they are what gets read first.
These numbers cannot be checked from outside. They come from
our own provider's analytics, retrieved with a token only the operator holds.
Nobody can recompute them. What is published instead is the method: the query
runs in
tools/crawler-bericht.py,
in plain text, and anyone with a Cloudflare zone can run the same one against
their own.
So this is a self-report with a disclosed method, not a measurement someone could repeat. Everywhere else on this site that distinction is the point, and it would be dishonest to blur it here because the numbers happen to be flattering.
Two further limits. One day is one day — a crawler that visits weekly is invisible in it. And requests are counted at the edge, so a system reading a cached copy through an intermediary never appears at all.