llms.txt: when it helps and how to test it
llms.txt is a proposed Markdown file format for listing useful pages on a website. An assistant configured to read it can use those links to find documentation. Publishing the file does not establish that search services will use it or cite your site.
For a small service business with a handful of clear pages, improving those pages is usually the more direct task. For a software product with extensive documentation, a short list of current installation instructions, API references and examples may solve a real navigation problem.
Choose links that help with a specific task
Start with the project name and a short description. Group a few useful links by purpose and explain what each page contains. Include maintained pages rather than every URL on the site. The specification also allows an Optional section for material a tool can skip when it needs less context.
# Example product
> Documentation for the Example booking API.
## Documentation
- [Getting started](https://example.com/docs/start): Create a test booking.
- [Authentication](https://example.com/docs/auth): Issue and rotate API keys.
Test whether the file helps the intended assistant
Give the assistant a task, such as finding instructions for replacing an API key, and check whether it finds the right page using the file. Verify the links and examples. This test is separate from checking citations in AI search. Google does not require special AI text files for its AI search features, and llms.txt does not replace accessible pages with useful content.
Sources and further reading
Related guides
- JavaScript and AI crawlers: how to check whether your text is accessible
- GPTBot, OAI-SearchBot and ChatGPT-User: how to check access to your site
- AEO for knowledge bases: write instructions people can follow
Published: · Reviewed by the geo-rank.ai editorial team · Our methodology