How to write a useful llms.txt

An llms.txt introduces your site and links to pages an agent may need. Start with a short description, then group links by topic and explain what each destination contains.

Publishing a file does not guarantee that an assistant will fetch it, use your content or cite your site. Start with accurate, accessible source pages: an index can point to useful information, but cannot supply information that those pages lack.

Start with a complete example

The example below is for Harbour, a fictional service. Its example.com addresses are placeholders, not working documentation. Replace the name, description, links and notes with facts about your own site. Use .md addresses only for resources that exist.

# Harbour

> Harbour is a fictional team scheduling service. This index covers its public product documentation and support policies.

Use the setup guide for a first workspace and the reference for individual settings. Account-specific schedules require sign-in and are not part of this index.

## Documentation

- [Set up a workspace](https://example.com/docs/setup.md): Create a workspace, invite colleagues and set its time zone.
- [Schedule reference](https://example.com/docs/schedules.md): Available scheduling settings and how recurring events behave.

## Policies

- [Support](https://example.com/support.md): Support channels, the information to include in a request and the scope of assistance.

## Optional

- [Release notes](https://example.com/releases.md): Dated changes for readers checking whether a feature is available.

This sample passes our structure and quality checks at the versions reviewed for this guide. Before publishing your own file, check its facts, links and HTTP response.

Understand the format

The llms.txt v2 proposal is a community convention. It specifies an H1 title, followed optionally by a summary blockquote, introductory detail and H2 sections containing link lists. A list entry uses a Markdown link with optional notes after a colon. An Optional section holds secondary material.

The title is the only required section in the proposal. For a useful index, we recommend a summary and descriptive links too. Good for Bots evaluates usefulness as well as structure; its additional criteria appear under Current scanner criteria.

The proposal permits files at the root or within a subtree, such as /docs/llms.txt, covering URLs beneath that path. Our checks read only the root: a site that serves /docs/llms.txt and nothing at /llms.txt is reported as having no file. Each check's limitations are listed below.

Choose pages that answer real questions

Begin with the questions visitors bring to your site. For a documentation site, those might be how to install a tool, perform the first task and look up an option. For a service, they might be what it does, who it serves, how to get help and where its policies are published.

Link directly to pages that answer those questions. Leave out tag archives, search results, tracking URLs and duplicate language variants unless they serve a specific need.

Use titles that identify the destination. A note should add something the title does not: the task it supports, the information it contains or a relevant scope limit. In the example, “Schedule reference” is followed by the settings and behaviour the reader will find there.

Keep claims tied to the destination. If a support page lists contact channels but makes no response-time promise, do not invent one in its link note. When information changes frequently, explain where to find its current value instead of copying that value into several files.

Publish the file and check the response

For a site you control at the root, publish the file at https://your-domain.example/llms.txt. Use your host's mechanism for serving a static text file or an equivalent server route. The response body should be the Markdown you wrote, available without signing in or running browser JavaScript.

After deployment, open the public address and inspect its HTTP response. With curl, replace the example host and run:

curl --include https://your-domain.example/llms.txt

Check the status, headers and body together. A 200 response containing your application's HTML error page is not a published llms.txt. If you see a redirect, inspect its destination and repeat the request there. Prefer serving the file directly at its public address. If local and production responses differ, compare the two configurations.

Use a text media type appropriate to the response, such as text/plain; charset=utf-8. Check the body as well: renaming an HTML response does not turn it into Markdown. Our current detector's interpretation of the body is described in the methodology below.

You can also make a request with our crawler's identity:

curl --include \
  --user-agent 'GoodForBotsBot/1.0 (+https://goodforbots.com/bot)' \
  https://your-domain.example/llms.txt

This shows whether your server answers our user agent differently. It cannot show what our scanner receives from its own network. curl also ignores robots.txt, so a successful request says nothing about your rules: the Robots Exclusion Protocol applies to /llms.txt like any other path. Our crawler page shows the rules our bot obeys.

To check the file from our side rather than from your own network, use the llms.txt checker.

Fix common mistakes

A list with no descriptions

# Harbour

> Harbour is a fictional team scheduling service. This index points to its public documentation.

## Documentation

- [Setup](https://example.com/docs/setup.md)
- [Schedules](https://example.com/docs/schedules.md)

This example passes our structure check but fails our quality check as a bare list. Add a meaningful note after each link, using :, as in the complete example. Aim to help a reader choose a destination, rather than writing repeated filler to satisfy a counter.

Descriptions without an introduction

# Harbour

## Documentation

- [Set up a workspace](https://example.com/docs/setup.md): Create a workspace, invite colleagues and set its time zone.
- [Schedule reference](https://example.com/docs/schedules.md): Available settings and how recurring events behave.

The links are descriptive, but our quality check warns because there is no summary blockquote. Add a short introduction directly below the title, before the link sections. Explain what the site covers and any scope distinction a reader needs before following a link.

The application answers instead of the file

<!doctype html>
<html lang="en">
  <head><title>Harbour</title></head>
  <body><main><h1>Page not found</h1></main></body>
</html>

If /llms.txt serves this HTML with a successful status, our structure check fails it as an HTML page. Check static-file routing, catch-all handlers and the deployed file location. Fix the response body and route; changing only the filename or content-type header leaves the underlying problem in place.

A challenge, refusal or failed response is a different problem. Use the evidence and explanation in your report to distinguish access trouble from a missing or malformed file. Read what our bot fetches before changing access rules.

Give readers useful destinations

Follow every link you publish. Confirm that it reaches the intended content without a login, returns the expected representation and describes the same subject as its note. For Markdown alternatives, compare them with the corresponding visible page after a content change.

The llms.txt proposal recommends clean Markdown counterparts and discovery links using alternate and describedby relations. Prefer a working Markdown destination where you provide one; do not manufacture a .md URL by changing a suffix without checking the response.

Our Markdown-content check and Markdown-negotiation check explain how we assess Markdown responses. Check linked destinations yourself; a passing index does not verify them.

Keep the index useful after launch

Treat the index as part of your content release. When a linked page moves, disappears or changes purpose, update its entry in the same change. If your publishing system generates the file, inspect its output: generation can keep URLs current while still producing an unhelpful set of titles or descriptions.

Ask someone unfamiliar with your site to find an answer using the index. Watch which link they choose and check whether the page answers their question. Revise labels that lead them elsewhere.

Scan your site after publishing to see what Good for Bots can retrieve. Use the report's evidence and fixes alongside these manual checks. The score measures our declared technical criteria; your editorial review checks whether the information is accurate and useful.

Sources and review

Reviewed on 24 September 2026 against the llms.txt v2 proposal, the curl reference and RFC 9309. The examples and writing recommendations are ours. Scanner criteria come from the same reviewed catalogue as Standards and scoring; this guide's examples are also tested against the actual checks.