GPTBot, OAI-SearchBot and ChatGPT-User: how to check access to your site
A request with “OpenAI” in its name can serve different purposes. OpenAI documents OAI-SearchBot for search, GPTBot for content that may be used to train foundation models, and ChatGPT-User for certain user-initiated visits. OpenAI’s list now also includes OAI-AdsBot, which checks pages submitted as ChatGPT ads; it has nothing to do with search or training. Blocking or allowing one of the three agents discussed here is not a decision about the other two.
If your aim is visibility in ChatGPT search, examine access for the search crawler. A GPTBot visit does not demonstrate that a page has been added to search, cited in an answer or used in training. The log records a request, not what happened afterward.
Check where requests are allowed or blocked
Ask your developer to check robots.txt, redirects, response codes and rules in the firewall or content delivery network (CDN). Even if robots.txt allows access, another system may show the crawler a verification challenge or return a 403 denial. The page opening fine in your browser does not prove that the crawler gets the same response.
A developer can use curl to send a request with a crawler’s User-Agent, the header that identifies the software making the request. This can reveal rules based on that name, but it does not prove that the request comes from the real crawler. To identify genuine visits, use the verification information and IP ranges in OpenAI’s documentation.
Check the response again after changing access rules
After a change, inspect new requests and the content returned to them. Test ChatGPT search separately and save the answers and cited links. ChatGPT-User is used for certain user-initiated visits and can behave differently from a search crawler, including in how robots.txt applies. Consult the current documentation before applying one rule to all three. Successful access does not guarantee a citation.
Sources and further reading
Updated: · published: · Reviewed by the geo-rank.ai editorial team · Our methodology