If you are searching what is llms.txt and how to use it, start from the problem it tries to solve. A language model answering a question about your company works with limited context, and full web pages come loaded with menus, scripts, banners and footers. The file offers a clean, hand-picked map in Markdown that a model or an agent can read in seconds. It lives at yourdomain.com/llms.txt, next to robots.txt and sitemap.xml, yet it does a different job from both.
What is llms.txt and how to use it: the file format
Knowing what is llms.txt and how to use it begins with its structure, which the proposal defines in this order:
- A heading with the name of the company or project, the only required element.
- A blockquote with a short summary holding the facts needed to understand the rest of the file.
- Optional paragraphs or lists with more detail, without headings.
- Sections under second-level headings, each with a list of Markdown links followed by an optional note.
- An optional section named "Optional" for secondary links a model can skip when context is short.
For a service company, the lines of the file could look like this:
| Line in the file | Example | Purpose |
|---|---|---|
| # Name | # Acme Automation | Required title |
| > Summary | > Automation projects for food plants in Ohio | Context in one sentence |
| ## Section | ## Services | Groups the main pages |
| - [Page](URL): note | - [Line automation](https://example.com/automation): scope, steps, FAQ | Points to a page and says what it contains |
| ## Optional | ## Optional | Pages a model may skip |
How llms.txt differs from robots.txt and sitemap.xml
llms.txt differs from the other two root files in audience and in power: it informs, but it does not control access.
| File | Main reader | What it does | Controls access? |
|---|---|---|---|
| robots.txt | Crawlers | Says which paths a crawler may fetch | Yes, for crawlers that respect it |
| sitemap.xml | Search engines | Lists URLs for discovery | No |
| llms.txt | Language models and agents answering a question | Summarizes the site and curates key pages | No |
That last column holds the most common misunderstanding. llms.txt neither blocks nor allows AI crawlers; that is still done in robots.txt, with user agents such as GPTBot or Google-Extended. Publishing the file also does not make any model read your site.
Does llms.txt help with Google or AI search?
llms.txt has no documented effect on Google: Google has said it does not use the file, and both AI Overviews and AI Mode select sources from the regular Search index. Among AI providers, support is uneven. Documentation platforms and developer tools often generate the file, and some coding agents fetch it when pointed at a site, but no major AI search product has documented llms.txt as an input for ranking or citation. Your server logs are the reliable way to learn whether anything requests yours.
Our position is pragmatic. The file costs little to create and can help in specific cases, such as software documentation, APIs and knowledge bases, but it never replaces the work that shapes AI visibility. That work is explained in our guides on generative engine optimization and answer engine optimization.
How to create and publish your llms.txt
- Choose the pages that answer the main questions about your business: services, how pricing is defined, documentation, policies and the about page.
- Write the title and summary in plain language with the facts a model needs: what you do, for whom and where.
- Group links under sections, each with a one-line note saying what the page contains.
- If useful, offer Markdown versions of key pages, which the proposal suggests serving at the same URL with .md appended. Large documentation sites sometimes add a longer companion file named llms-full.txt, a convention some tools follow rather than part of the proposal.
- Upload the file to the root as plain text in UTF-8 and confirm it returns status 200.
- Update the file in the same release whenever a listed page moves or changes.
- Check server logs for requests to /llms.txt to see who reads it.
Common mistakes when publishing llms.txt
- Listing every URL on the site, which turns the file into a second sitemap and defeats its purpose.
- Including staging, private or gated pages that were never meant to be public.
- Writing sales copy instead of facts a model can repeat accurately.
- Using it in an attempt to block AI crawlers, a job that belongs to robots.txt.
- Letting it go stale after a redesign, so it points to redirects or missing pages.
For most business websites, the honest summary of what is llms.txt and how to use it is this: a small, optional file worth publishing carefully, with low expectations. Indexable pages, clear answers, consistent company facts and structured data do far more for AI visibility, and that is where our AI SEO agency work concentrates. Our team in Brazil serves businesses in the United States and Ireland remotely, drawing on more than 700 clients served in Brazil since 2021.
If you want a second opinion on your robots.txt, llms.txt and structured data, book a call with our team and we will review them with you.