Insights

Why most AI crawlers cannot read your JavaScript content.

Your product page looks complete in the browser. But GPTBot, ClaudeBot and PerplexityBot, the crawler programs that read websites for ChatGPT, Claude and Perplexity, never open a browser. They read the raw HTML your server sends, and whatever JavaScript builds on top of it simply does not exist for them. Here is how to see your site the way AI sees it, in about two minutes.

7 min read·
Abstrakte Darstellung von AI-Bots, die eine Website umkreisen und lesen

Two versions of your website exist. AI only sees one.

When a person opens your product page, a lot happens that they never notice. The browser downloads the HTML, then downloads JavaScript files, runs them, fetches product data from your shop system, and assembles the finished page piece by piece: the price appears, reviews load, the FAQ accordion unfolds. The polished result is version one, the rendered page. Rendering is just the technical word for that assembly work.

Version two is what existed before any of that ran: the raw HTML response, exactly as your server sent it. For many modern shops that raw response is close to empty. That is especially true for so-called single-page applications built with React or Vue, sites where the browser constructs the entire page with JavaScript. The raw response then looks something like this:

view-source of a client-rendered product page

<body>
  <div id="root"></div>
  <script src="/assets/main.8f3a2c.js"></script>
</body>

No product name. No price. No description, no reviews, no FAQ. All of that arrives later, when JavaScript runs in a browser. And here is the part that matters: most AI crawlers never run that JavaScript. They request the page, read the raw response, and move on. If your product only exists in version one, then for ChatGPT, Claude and Perplexity your product does not exist.

diag-why-ai-cannot-read-javascript-0.svg

Which crawlers render JavaScript, and which do not

Executing JavaScript at crawl scale is expensive. It means running something like a full browser for every page of the web, so almost every AI crawler skips it. Based on the crawlers' published documentation and observed behavior at the time of writing:

Crawler

Feeds

Executes JavaScript?

GPTBot / OAI-SearchBot

ChatGPT, OpenAI search

No. Reads the raw HTML response.

ClaudeBot

Claude

No. Reads the raw HTML response.

PerplexityBot

Perplexity

No. Reads the raw HTML response.

Googlebot

Google Search, AI Overviews

Partially. Rendering happens in a deferred second pass.

Bingbot

Bing, Copilot

Limited. Do not count on it.

The pattern is clear: the one ecosystem with mature JavaScript rendering is classic Google, and even there rendering is delayed and not guaranteed for every page. The crawlers behind the fastest-growing AI assistants read raw HTML, full stop. Anyone telling you "Google renders JavaScript, so we're fine" is answering a question from 2019, not the one that decides whether an AI shopping agent can see your catalog.

What typically goes missing

The damage is rarely all-or-nothing. Even on shops that render the basic page server-side, specific high-value pieces are often injected by JavaScript afterwards and are invisible in the raw response:

  • Prices and availability loaded live from your shop system after the page loads.

  • Product variants where sizes, colors and their prices only exist after a script runs.

  • Reviews and ratings pulled in by a third-party review widget.

  • FAQ accordions whose questions and answers live in a JavaScript bundle, not in the HTML.

  • Delivery times, shipping costs and stock per location fetched on demand.

Notice that this list is almost exactly the information an AI assistant needs to recommend or buy a product. The marketing headline usually survives. The facts often do not.

diag-why-ai-cannot-read-javascript-1.svg

Test it yourself: three ways, no developer required

terminal · fetch a page the way GPTBot does

curl -A "GPTBot" https://www.example-shop.de/products/pour-over-set | grep "49.90"

If that search comes back empty while the price is clearly visible in your browser, you have just reproduced the entire problem in one line.

What actually fixes it

There are three honest routes, and which one fits depends on how much you can change your stack:

  • Server-side rendering (SSR). The cleanest fix. The assembly work moves to your server: it delivers the finished HTML, and JavaScript only adds interactivity on top. Modern frameworks (Next.js, Nuxt, Astro and others) support this well. The catch: on an existing shop this is a real engineering project, not a setting you toggle.

  • Pre-rendering. Pages are rendered to static HTML at build time or by a pre-render service, and crawlers get the finished version. Works well for catalogs that do not change every minute; gets harder with live stock and frequent price changes, because the snapshot can lag behind reality.

  • Deliver the facts server-side as structured data. Instead of rebuilding the storefront, you publish the machine-relevant facts (products, prices, availability, answers) as clean server-rendered HTML with structured data, meaning facts in a format machines can read directly. Klariton's schema export takes this route: a crawlable subdomain generated from your shop data, wired up with one line in your robots.txt (the small text file that tells crawlers where to go), or served through your own server API in around ten lines of code. Your shop stays exactly as it is; crawlers get a version they can read.

diag-why-ai-cannot-read-javascript-2.svg

By the way, the same logic applies beyond shops: if you sell services, the readability gap is usually even wider, which is why service providers and B2B companies need structured data even more than shops.

One caution on the honest side of the ledger: do not build your strategy on the engines that render. Google's rendering exists, but it is deferred, resource-capped and covers exactly one ecosystem. The direction of travel in agentic reach is toward more agents making fast, cheap, HTML-only requests, not fewer: Visa reports that 47% of US shoppers already use AI tools for at least one shopping task, and expects millions of consumers to buy through AI agents by the 2026 holiday season. Server-delivered content is the baseline that works for all of them.

Frequently asked questions

Ask Klariton

Ask your question about Klariton.

Grounded in Klariton’s own knowledge, cited rather than invented.

Or ask your own question:
Next step

How visible is your brand to AI?

The free AI visibility check shows you in under a minute how AI assistants see your shop today.