See your site the way AI crawlers see it: a ten-minute self-check

When the two views differ, and on many sites they differ a great deal, the crawler's version is the one that decides how you are described and cited. Our research on what AI crawlers actually see (internal study, n=10, mid 2026) found that none of the major brand sites tested exposed their structured data to a non-JavaScript fetch, even though it was present once a browser rendered the page.

The three checks below take about ten minutes and show you, on your own domain, exactly what an engine reads. You can run every one of them with tools you already have, and act on what you find without anyone's help. Run them against your home page first, then any page that matters for how you are described.

You do not need to be a developer to get value here. The first check is a single command you paste into a terminal, the second is a page you can read in your browser, and the third is a comparison of the two. If you can copy and paste, you can see what an engine sees, and that alone puts you ahead of most sites, which have never looked at their raw view at all.

This fetches your home page the way an AI crawler would, as GPTBot and without running JavaScript, and checks whether any structured data is present in the raw response. Zero means none was found; any other number means at least some is present.

One caution before trusting a zero. Some bot-protection layers verify crawler identity and serve a challenge page to requests that claim to be GPTBot but come from an ordinary connection. If the zero comes back alongside an unusually short response, rerun the command without the -A "GPTBot" flag. If structured data appears in the plain request but not the bot-flagged one, the finding is a bot wall, not missing schema, and it needs a closer look rather than a conclusion.

Open yoursite.com/robots.txt and look for GPTBot, ClaudeBot, Google-Extended, OAI-SearchBot, Claude-SearchBot and PerplexityBot, so you can see which AI crawlers your site allows and which it blocks.

These fall into two groups. GPTBot, ClaudeBot, and Google-Extended are training crawlers; OAI-SearchBot, Claude-SearchBot, and PerplexityBot fetch pages to answer live questions. A site can block the training group and still allow the retrieval group, so check each line rather than assuming one verdict covers them all.

Compare the raw fetch from check one with the fully rendered page your browser shows. If the structured data appears only in the rendered view, it is client-injected, and AI crawlers do not see it. A page can look complete on screen while the raw response an engine reads is empty.

The simplest way to run the diff is to open the raw fetch from check one, search it for the structured data you expect, then compare that with what the rendered page contains. If the schema is in the browser but not the fetch, your markup is being added after the page loads, which is the pattern that hides it from the systems that read it. Many teams are surprised at this step, because the rendered page they are proud to ship looks nothing like the sparse response a crawler actually collects.

These three checks are worth running, but it helps to be clear about their limits. They find access and carrier problems: whether a crawler can reach your pages, and whether the structured data you rely on actually travels in the raw response. The crawlers behind ChatGPT, Claude, and Perplexity retrieved the raw response and did not execute JavaScript (Vercel crawler analysis, 2025; confirmed across independent tests through mid 2026); Google's surfaces, Gemini, Google AI Overviews, and Google AI Mode, draw on Google's rendered index, though rendering there is queued rather than guaranteed, so server-side structured data is the only version every engine can read. Those are real issues, and fixing them removes the most common reasons a brand is read incompletely.

What they do not tell you is whether the entity graph behind your markup resolves correctly. A site can serve valid schema in the raw view and still describe its organisation, its divisions and its products in a way that leaves an engine unsure who is who. That is structural diagnosis, a deeper question than access, and it is not something a single command can answer. If a check shows that your schema or llms.txt is missing from the raw view, Zicy's Action Center can generate both for you, though the fixes above are within reach of any developer who wants to do them by hand.

Treat this as triage rather than a full audit. Clearing the three checks means a crawler can reach you and your structured data is in the response it reads. That is the floor, the point below which nothing else you do about AI visibility can work. It is not the ceiling, but it is the right place to start, and most sites have not started.