← Blog

JavaScript and AI crawlers: how to check whether your text is accessible

Your browser may show a complete page even when the server’s initial HTML contains almost no text. In that case, JavaScript adds the content after loading. This does not automatically make the page invisible to search: Google can run JavaScript, while other crawlers and tools differ. Check the page and the service you care about before deciding what to change.

Compare two versions of the page

Open a service page normally and identify its essential content: heading, offer, price conditions, service area and links. Then inspect the HTML returned by the server. A simple download can help:

curl -L https://example.com/service/ -o page.html

Find the actual heading, offer and conditions in the response; finding related words inside JavaScript code is not the same as finding readable page content. Check the response code and redirects too. A blocked request, rate limit or login screen needs a different fix from missing text. Changing the User-Agent header only changes the name sent with the request; it does not make it a verified crawler visit.

Choose a fix that fits the page

For important public pages, ask whether the site can generate the main text in advance or on the server. These approaches are known as static generation and server-side rendering. Calculators and forms can still use JavaScript. A change to the relevant template may be enough; a full rebuild is not always necessary.

After the change, compare the server response with the page in the browser. Check that important facts match, links work and canonical URLs still identify the intended pages. Use search-engine inspection tools and server logs if available. A readable response confirms what was accessible in that test, not whether an AI service will cite it.

Sources and further reading

Related guides

Related guides

Published: · Reviewed by the geo-rank.ai editorial team · How we check