← Blog

GPTBot, OAI-SearchBot and ChatGPT-User: how to check access to your site

A request with “OpenAI” in its name can serve different purposes. OpenAI documents OAI-SearchBot for search, GPTBot for content that may be used to train foundation models, and ChatGPT-User for certain user-initiated visits. Blocking or allowing one should not be treated as the same decision for all three.

If your aim is visibility in ChatGPT search, examine access for the search crawler. A GPTBot visit does not demonstrate that a page has been added to search, cited in an answer or used in training. The log records a request, not what happened afterwards.

Check where requests are allowed or blocked

Ask your developer to check robots.txt, redirects, response codes and rules in the firewall or content delivery network (CDN). Even if robots.txt allows access, another system may show the crawler a verification challenge or return a 403 denial. Successfully opening the page in your browser does not prove that the crawler receives the same response.

A developer can use curl to send a request with a crawler’s User-Agent, the header that identifies the software making the request. This can reveal rules based on that name, but it does not prove that the request comes from the real crawler. To identify genuine visits, use the verification information and IP ranges in OpenAI’s documentation.

Check the response again after changing access rules

After a change, inspect new requests and the content returned to them. Test ChatGPT search separately and save the answers and cited links. ChatGPT-User is used for certain user-initiated visits and can behave differently from a search crawler, including in how robots.txt applies. Consult the current documentation before applying one rule to all three. Successful access does not guarantee a citation.

Sources and further reading

Related guides

Published: · Reviewed by the geo-rank.ai editorial team · How we check