There is no crawler token, and that is a documented absence
Every other assistant on this section gives publishers something to address in robots.txt. OpenAI has GPTBot and its siblings, Anthropic has ClaudeBot, Perplexity has PerplexityBot, and Google has Google-Extended. xAI publishes nothing equivalent.
We checked directly. x.ai/robots.txt, grok.com/robots.txt and grokipedia.com/robots.txt contain no xAI, Grok or GrokBot token. Cloudflare's AI Crawl Control bot reference, which lists documented crawlers by operator for a dozen companies, has no xAI entry.
One thing that looks like a control and is not: x.ai's own robots.txt does contain groups for GPTBot, ClaudeBot, PerplexityBot, Google-Extended and Applebot-Extended, plus a Content-Signal line. That is xAI stating how it wants its OWN site treated by other companies' bots. It is not a lever you can pull.
Search Engine Journal's crawler reference lists Grok under a separate heading for agent crawlers with an unidentifiable user agent, noting "there is no way to track this crawler from accessing webpages other than by identifying the explicit IP". Search Engine Land's guide says some crawlers "don't identify themselves at all in user-agent strings" and names Grok, which "seems to mimic an iPhone, according to some tests".
Treat that last mechanism as thin. It is soft sourcing with no named study or methodology, and the more elaborate versions circulating in SEO coverage, rotating residential IPs and specific spoofed strings, have no primary source at all. The core negative is solid and the specific technique is not. If you want to restrict Grok, the honest answer today is that it has to happen at the server or WAF level rather than in robots.txt.
Grok does cite, and it is documented
xAI's developer documentation describes a real citation system with two formats: a full citations list of every source URL the model touched, and inline markdown links embedded in the response text as [[N]](url), numbered from one. Each carries structured metadata including the URL, a title, and character offsets marking exactly which span of text it supports.
Defaults differ by surface, which matters if you are testing what users see. Inline citations are on by default in the Responses API, and off by default in xAI's Python SDK unless explicitly requested.
Citing and citing correctly are different things, though, and the only independent accuracy study found Grok-3 failing 94% of citation tests, worst of eight tools. That study is from March 2025 and predates the current model line.
Measuring Grok traffic: one door open, one closed
Statcounter, which runs the largest public cross-site AI referral panel, excludes Grok outright. Its wording has been identical across releases a year apart, in June 2025 and again in May 2026: "Grok cannot be included in the data, as unlike the other chatbots, it does not provide referral data in its header." Every Statcounter AI referral chart you see, including the ones on our Gemini hub, therefore has no Grok line at all.
Your own analytics is the better route. GA4's AI Assistants channel names Grok among the referrers it recognises and buckets those sessions automatically under the medium "ai-assistant", with no setup.
The two facts do not cancel out, and neither gives you a number. GA4 catches Grok visits that arrive with a usable referrer. Statcounter is describing a consistency problem across an entire panel of sites. Nothing we could find quantifies what share of Grok-driven visits carry a referrer at all, so treat your own GA4 channel as the measurement and treat any published "Grok sends X% of AI traffic" figure with suspicion.