---
title: "How to serve Markdown to AI agents with content negotiation"
description: "Return Markdown from the same URL when an AI agent sends Accept: text/markdown, with tested responses, Vary and CDN caching, and nginx and Next.js setups."
url: "https://goodforbots.com/guides/markdown-negotiation"
date: "2026-10-03"
reviewed: "2026-10-03"
---

# How to serve Markdown to AI agents with content negotiation

By Good for Bots · Reviewed 2026-10-03 · [Page content](https://goodforbots.com/guides.md?category=page-content)

When a request's `Accept` header asks for `text/markdown`, answer at the same URL with a Markdown version of the page, labelled `Content-Type: text/markdown; charset=utf-8`, and add `Vary: Accept` so caches keep the two versions apart. Browsers do not ask for Markdown, so visitors keep receiving HTML. An agent that asks gets the page's text without the navigation, styling and scripts around it.

This is HTTP content negotiation applied to a registered media type. No standard says a website ought to offer Markdown: it is a convention that some agents and CDNs have adopted. It does not replace HTML that carries its content, and it does not guarantee that an assistant will fetch, use or cite your pages.

## Start with a complete example

An agent asks for the homepage of **Harbour, a fictional scheduling service**, and says it prefers Markdown to HTML:

```sh
curl --silent --include \
  --header 'Accept: text/markdown, text/html;q=0.9' \
  https://example.com/
```

The server answers with the Markdown version of the page:

```http
HTTP/2 200
content-type: text/markdown; charset=utf-8
vary: Accept

# Scheduling for teams that work in shifts

Harbour is a fictional scheduling service. A manager builds the rota once,
Harbour repeats it, handles shift swaps and warns when a shift is left without
cover.

## What Harbour does

- Recurring rotas with weekly, fortnightly or custom patterns.
- Shift swaps that a colleague accepts and a manager approves.
- Holiday requests checked against the minimum cover you set for each role.

## Who uses it

Clinics, cafés, warehouses and support desks with between 5 and 500 people.
Each workspace has its own time zone, so a team spread across two countries
sees every shift in local time.

## Plans

The free plan covers one team of up to 10 people. Paid plans are billed
monthly for each active member; the
[pricing page](https://example.com/pricing) lists the current rates.

The cover calculator, which estimates how many people each shift needs, runs
only in a browser:
[open it on the page](https://example.com/#cover-calculator).
```

`example.com` stands in for your domain. `text/markdown` is the media type registered for Markdown, and its registration requires the `charset` parameter. `Vary: Accept` tells any cache on the way that the answer depends on the `Accept` header. The body holds the page's main content, with the same headings, sentences and lists as the HTML and without the menu and footer. Links are absolute, and the calculator that needs a browser is described and linked rather than dropped. This response passes [our Markdown negotiation check](https://goodforbots.com/standards#markdown-negotiation) at the version reviewed for this guide.

A browser asking for the same URL never names Markdown. Firefox, for example, sends `text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8` when it opens a page, so it receives the HTML, unchanged.

## Which agents ask for Markdown

The clients we found asking are agents that fetch pages while they work for someone, most of them coding agents. Each entry comes from the tool's documentation or source code, read on 3 October 2026, unless it says otherwise.

- **Claude Code.** Its documentation says the WebFetch tool sends "an `Accept` header that prefers Markdown over HTML so servers that support content negotiation can return Markdown directly." We sent a request from version 2.1.286 to a service that echoes headers, and it carried `Accept: text/markdown, text/html, */*`. The two types have the same weight, so a server that settles ties in favour of HTML sends it HTML.
- **OpenCode.** Its fetch tool returns Markdown by default and asks with `text/markdown;q=1.0, text/x-markdown;q=0.9, text/plain;q=0.8, text/html;q=0.7, */*;q=0.1` (version 1.18.34).
- **Gemini CLI.** Puts `text/markdown` first only in its direct fetch mode, an experimental setting that is off by default (version 0.62.0).
- **Qwen Code and OpenClaw.** Both rank `text/markdown` above `text/html` in their fetch tools.
- **Cursor.** It publishes no source we could read. Checkly, a monitoring company, recorded it asking for Markdown in February 2026, in a test that found Codex, Gemini CLI and Windsurf did not.

None of these is a search or training crawler. A crawler that requests HTML still needs the content in the HTML, which [How to serve content AI crawlers can read without JavaScript](https://goodforbots.com/guides/server-rendered-html) covers.

## What HTTP requires and what is a choice

[RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-content-negotiation), the HTTP standard, calls this proactive negotiation: the client states its preferences and the server picks a representation. Three of its rules shape the implementation.

- **Quality values rank the types.** Each media range in `Accept` can carry a weight between 0 and 1. "If no "q" parameter is present, the default weight is 1", and 0 means "not acceptable". `*/*` stands for every media type, and when several ranges match one type, "the most specific reference has precedence".
- **The server may ignore the preference.** When it has nothing the client accepts, it "can either honor the header field by sending a 406 (Not Acceptable) response or disregard the header field". Sending HTML to a client that asked for Markdown is conformant.
- **`Vary` is recommended, not required.** "An origin server SHOULD generate a Vary header field on a cacheable response when it wishes that response to be selectively reused for subsequent requests."

[RFC 7763](https://www.rfc-editor.org/rfc/rfc7763.html) registers `text/markdown`. It is an Informational RFC, and it makes the `charset` parameter required, with "no default value". The IANA registry lists `text/markdown` and no `application/markdown`, so label the response with the registered name.

Two decisions in this guide are ours rather than HTTP's. We send Markdown only to a client that names `text/markdown`: a client that accepts `*/*` will take anything and has not asked for it. And a tie between Markdown and HTML goes to Markdown, because naming it was a deliberate choice. [Our check](https://goodforbots.com/standards#markdown-negotiation) has criteria of its own, listed at the end of this guide.

## Decide what the Markdown contains

Put in the page's main content and keep its facts identical to the HTML: the same headings, prices, dates, conditions, tables, code and links. Leave out what only makes sense on screen, such as the menu, the footer, cookie notices and buttons that open dialogs. The Markdown is not a place to tell agents something the page does not tell people.

- **Write links as absolute URLs.** Once the text is pasted into a prompt or saved to a file, a relative link has no address to resolve against. This is our recommendation; Markdown allows relative links.
- **Generate it from the same source as the HTML** when you can: a CMS field, an MDX file or a template. Converting the rendered HTML also works, and is what Cloudflare does, but the converter has to guess which parts are navigation.
- **Describe interactive parts.** A calculator, a map or a form has no Markdown equivalent. Say what it does and link to the page.
- **Keep one page per response.** A Markdown answer covers the page that was asked for. The whole site's content in one file is a separate convention, `/llms-full.txt`.

If you convert, use a maintained library rather than regular expressions: [Turndown](https://github.com/mixmark-io/turndown) for JavaScript, [rehype-remark](https://github.com/rehypejs/rehype-remark) in the unified ecosystem, [html-to-markdown](https://github.com/JohannesKaufmann/html-to-markdown) for Go or [markdownify](https://github.com/matthewwithanm/python-markdownify) for Python. Convert the page's `main` element rather than the whole document, so the menu and footer never reach the result.

## Serve it from your application

The negotiation fits in front of whatever renders your HTML. This TypeScript uses the standard `Request` and `Response` objects, so it runs in Node.js, Deno, Bun, Cloudflare Workers and the middleware of most JavaScript frameworks:

```ts
/** Whether `Accept` names Markdown and ranks it at least as high as HTML. */
export function prefersMarkdown(accept: string | null): boolean {
  const weights = new Map<string, number>();
  for (const range of (accept ?? "").split(",")) {
    const [type, ...parameters] = range.split(";").map(part => part.trim().toLowerCase());
    const q = parameters.find(parameter => parameter.startsWith("q="));
    const weight = q === undefined ? 1 : Number(q.slice(2));
    if (type && !Number.isNaN(weight)) weights.set(type, Math.max(weights.get(type) ?? 0, weight));
  }
  const markdown = weights.get("text/markdown") ?? 0;
  const html = weights.get("text/html") ?? weights.get("text/*") ?? weights.get("*/*") ?? 0;
  return markdown > 0 && markdown >= html;
}

/** Wraps the handler that renders your HTML; `markdownFor` returns null when a page has no Markdown. */
export function negotiate(
  renderHtml: (request: Request) => Promise<Response>,
  markdownFor: (path: string) => Promise<string | null>,
) {
  return async (request: Request): Promise<Response> => {
    const markdown = prefersMarkdown(request.headers.get("accept"))
      ? await markdownFor(new URL(request.url).pathname)
      : null;
    if (markdown !== null) {
      return new Response(markdown, {
        headers: { "Content-Type": "text/markdown; charset=utf-8", "Vary": "Accept" },
      });
    }
    const html = await renderHtml(request);
    const response = new Response(html.body, html); // a copy whose headers can change
    response.headers.append("Vary", "Accept");
    return response;
  };
}
```

`prefersMarkdown` reads the quality values, so `text/html, text/markdown;q=0.1` still receives HTML and `text/markdown;q=0` is a refusal. HTML takes its weight from the most specific range that covers it: `text/html`, then `text/*`, then `*/*`. Markdown has to be named. A page without a Markdown version falls through to the HTML, which HTTP allows. Both answers carry `Vary: Accept`, for the reason in the caching section below. The headers of a response obtained with `fetch` cannot be modified, which is why the code copies it before appending to them.

Where to put it depends on the framework:

- **Next.js.** [Its documentation](https://nextjs.org/docs/app/guides/backend-for-frontend#content-negotiation) rewrites requests whose `accept` header matches `(.*)text/markdown(.*)` to a Route Handler that returns the Markdown with `Content-Type` and `Vary`. That pattern ignores quality values, and the documented handler answers 404 for a page without Markdown, where the HTML would have served the client. To avoid both, call `prefersMarkdown` in `proxy.ts`, the file that replaced `middleware.ts` in Next.js 16, and rewrite only pages that have a Markdown version. Then check the HTML's `Vary`. On this site, a `Vary` set in `next.config` or in the proxy does not reach the HTML of App Router pages, which carry the values Next.js sets (measured with Next.js 16 on 22 September 2026, unchanged in production on 3 October). Cloudflare does not cache our HTML and our Markdown is sent with `no-store`, so no shared cache mixes them; if your HTML is cached, make sure the cache keys on `Accept`.
- **Astro.** Middleware can return its own `Response`, but for prerendered pages it runs at build time, not when a request arrives. Negotiate for a static Astro site in the web server or CDN in front of it; pages rendered on demand can do it in middleware.
- **SvelteKit.** The `handle` hook in `hooks.server` can return a `Response` for any request the server handles. Prerendered pages are served as static assets, which "are *not* handled by SvelteKit", so they need the same treatment as a static site.
- **Nuxt.** Server middleware "should not return anything", so it cannot send the Markdown itself. Negotiate in the server or CDN in front of Nuxt.
- **Static hosts.** Netlify's redirect rules cannot match a request header such as `Accept`, but an Edge Function can read it and return a rewrite. On Cloudflare Workers with static assets, set `run_worker_first = true`: by default a matching asset is served without running the Worker.

## Serve it from a static site with nginx

A static site generator can write a Markdown file next to each HTML file, following the [llms.txt](https://llmstxt.org/) convention: the page's address with `.md` appended, and `index.html.md` for an address that ends in a slash. nginx can then pick the file by the `Accept` header:

```nginx
map $http_accept $markdown_suffix {
    default           "";
    "~*text/markdown" ".md";
}

server {
    listen 80;
    root /var/www/site;

    include mime.types;
    types { text/markdown md; }
    charset utf-8;
    charset_types text/html text/markdown;

    location / {
        add_header Vary Accept;
        try_files $uri$markdown_suffix $uri/index.html$markdown_suffix $uri $uri/index.html =404;
    }
}
```

`map` sets the suffix to `.md` when the header mentions `text/markdown`, and `try_files` looks for the Markdown file before the HTML, so a page without a Markdown file still gets its HTML. The `types` line adds the media type to nginx's list rather than replacing it, and `charset_types` adds the `charset` parameter. We ran this configuration on nginx 1.30.5 on 3 October 2026: `/`, `/pricing/` and `/docs/setup.html` answered with Markdown or HTML according to the header, stylesheets kept their own type and every response carried `Vary: Accept`. The regular expression ignores quality values, so it also sends Markdown to a client that ranks it below HTML. If that matters to you, negotiate in application code.

## Let your CDN or documentation platform do it

**Cloudflare's Markdown for Agents** converts HTML to Markdown at the edge when the request's `Accept` includes `text/markdown`. You turn it on under AI Crawl Control, on a paid plan: on 3 October 2026 the documentation named Pro and Business in its setup steps and added Enterprise in its availability section. The converted response carries `Content-Type: text/markdown; charset=utf-8`, `Accept` in `Vary` and token estimates in `x-markdown-tokens` and `x-original-tokens`. Cloudflare strips headers, footers, navigation, scripts and styles, turns meta tags into YAML frontmatter and appends the page's JSON-LD as a code block. It converts HTML only, up to 2 MB per response.

It also adds `Content-Signal: ai-train=yes, search=yes, ai-input=yes` to the Markdown unless your server already sends a `Content-Signal` header. That header states how the content may be used, AI training included. If [your robots.txt](https://goodforbots.com/guides/robots-txt-ai-crawlers) says something different, send your own header from the origin so the Markdown carries the same choice.

**Documentation platforms** often negotiate already. Mintlify, Fern and ReadMe document that a request with `Accept: text/markdown` receives Markdown. GitBook documents a `.md` address for every page, and a request we sent with the header on 3 October 2026 also received Markdown. Docusaurus does not negotiate on its own: a community plugin writes Markdown files, which a server can then choose as in the nginx section above.

## Keep caches from mixing HTML and Markdown

A shared cache, such as a CDN, stores a response and hands it to the next client that asks for the same URL. [RFC 9111](https://www.rfc-editor.org/rfc/rfc9111.html#name-calculating-cache-keys-with) lets it reuse a stored response with `Vary` only when the request headers that `Vary` names match those of the original request. A response without `Vary` matches every request, and the standard describes the result: "Some resources mistakenly omit the Vary header field from their default response", which a cache then chooses "even when more preferable responses are available."

Send `Vary: Accept` with both answers. On the Markdown it keeps a browser from receiving Markdown; on the HTML it keeps an agent that asked for Markdown from receiving a cached page. If your server already sends `Vary`, for example `Vary: Accept-Encoding`, add `Accept` to the list rather than replacing it.

Then find out what your CDN does with the header. As each provider documented it on 3 October 2026:

- **Cloudflare:** "By default, Cloudflare does not consider vary values in caching decisions." It makes exceptions for `vary: accept-encoding`, for images and for the Vary setting in Cache Rules, which you have to turn on. If Cloudflare caches your pages, turn that setting on, or send the Markdown with `Cache-Control: no-store`, which Cloudflare never caches. This site does the latter.
- **Vercel:** its CDN "already includes the `Accept` and `Accept-Encoding` headers as part of the cache key by default."
- **Netlify:** headers named in `Vary` "are factored into cache keys for all response types". Its own `Netlify-Vary` header does not accept `Accept`, so use the standard one.
- **Fastly:** "`Vary` is an HTTP standard, and Fastly supports it per spec".

Give the two representations different entity tags as well. RFC 9110 defines an `ETag` as a validator "for differentiating between multiple representations of the same resource", including those produced by content negotiation. A tag computed from the body or the file already differs, as it did for the two files in the nginx test above. A tag derived from the page's revision alone is shared by both and needs the representation added to it. Cloudflare drops `ETag` and `Last-Modified` from the Markdown it generates, because conditional requests "cannot be honored for converted responses".

## Point agents to the Markdown version

An agent that has the HTML can find the Markdown without guessing if the page links to it. RFC 8288 defines the `Link` header and HTML has the same relation as a `<link>` element; either is enough, and this response carries both:

```http
HTTP/2 200
content-type: text/html; charset=utf-8
vary: Accept
link: </index.html.md>; rel="alternate"; type="text/markdown"

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Harbour: scheduling for teams that work in shifts</title>
    <link rel="alternate" type="text/markdown" href="/index.html.md">
  </head>
  <body>
    <main>
      <h1>Scheduling for teams that work in shifts</h1>
      <p>Harbour is a fictional scheduling service. A manager builds the rota once, Harbour repeats it, handles shift swaps and warns when a shift is left without cover.</p>
    </main>
  </body>
</html>
```

The [llms.txt specification](https://llmstxt.org/) names this relation: `rel="alternate" type="text/markdown"` "points to the markdown version of a page". RFC 8288 calls the `type` attribute "only a hint", so the target has to answer with Markdown itself, whatever `Accept` the client sends. Point the link at the `.md` address rather than back at the page, and only at an address that exists. This response passes [our typed discovery links check](https://goodforbots.com/standards#link-headers) at the version reviewed for this guide. [How to write a useful llms.txt](https://goodforbots.com/guides/llms-txt) covers listing the same `.md` addresses in an index.

## Check what your server sends

Ask for Markdown the way an agent does, and keep the headers:

```sh
curl --silent --location --dump-header - --output page.md \
  --header 'Accept: text/markdown' \
  https://your-domain.example/pricing
```

1. **Read the headers.** The last block should show status 200, `content-type: text/markdown; charset=utf-8` and a `vary` header that includes `Accept`. `--location` follows redirects, as our scanner does, so earlier blocks belong to the redirects.
2. **Read the body.** `page.md` should start with the page's heading or frontmatter, not `<!doctype html>`, and contain the text of the page.
3. **Ask as a browser.** Repeat the request with Firefox's header, `'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'`. The answer should be `text/html`, with `Accept` in `vary`.
4. **Ask through the cache.** Request the same URL as Markdown, then as a browser, then as Markdown again. If a CDN is mixing the two, one of the answers will have the wrong type.
5. **Try every template.** Check a page of each kind: an article, a product page, a documentation page. Our check reads a few sampled pages, not the whole site.

## Fix common mistakes

### HTML labelled as Markdown

```http
HTTP/2 200
content-type: text/markdown; charset=utf-8
vary: Accept

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Support | Harbour</title>
    <script type="module" src="/assets/app-3f9c1e.js"></script>
  </head>
  <body>
    <main>
      <h1>Support</h1>
      <p>Write to support@example.com from the address your Harbour workspace uses.</p>
    </main>
  </body>
</html>
```

The header promises Markdown and the body is the HTML page. This happens when a plugin or middleware sets the header and passes the page through unchanged. A client that trusts the header hands markup to a model as if it were text. Convert the body, or remove the header until the conversion works. Our check fails this response.

### Markdown without `Vary: Accept`

```http
HTTP/2 200
content-type: text/markdown; charset=utf-8

# Support

Write to support@example.com from the address your Harbour workspace uses.
Include the workspace name, the rota or shift concerned and what you expected
to happen.

We answer within one working day, Monday to Friday. On paid plans, a rota that
will not publish counts as urgent and gets an answer within four hours.
```

The body is correct, and a cache can still serve it to the wrong client: in this case, a browser that asks for the support page after an agent did. Add `Accept` to `Vary`, keeping any values already there. Our check gives this response a warning.

### Only some pages answer

Negotiation is decided per URL: RFC 9110 allows that a server "might not implement proactive negotiation for the requested resource". Sites often add it to the documentation or the homepage and leave the rest out. Put it in the shared request handling, so new pages receive it without further work, and check a page of every template.

### A 406 instead of a page

RFC 9110 allows a 406 when nothing the client accepts exists. When the request also contains `*/*`, though, HTML is acceptable, and a 406 refuses a page the client was ready to take. RFC 9110 also lets a server decide that a response outside the client's preferences is "better than sending a 406". Send the HTML when a page has no Markdown version. Our check counts a 406 as no Markdown.

## Sources and review

Reviewed on 3 October 2026 against these sources:

- [RFC 9110: HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html), sections 8.8, 12 and 15.5.7, and [RFC 9111: HTTP Caching](https://www.rfc-editor.org/rfc/rfc9111.html), section 4.1, both Internet Standards
- [RFC 7763: The text/markdown Media Type](https://www.rfc-editor.org/rfc/rfc7763.html), Informational, and the [IANA Media Types registry](https://www.iana.org/assignments/media-types/media-types.xhtml)
- [RFC 8288: Web Linking](https://www.rfc-editor.org/rfc/rfc8288.html), Proposed Standard, and [the /llms.txt file, v2](https://llmstxt.org/)
- [MDN: List of default Accept values](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation/List_of_default_Accept_values)
- [Claude Code: Tools reference](https://code.claude.com/docs/en/tools-reference), the source code of [OpenCode](https://github.com/anomalyco/opencode), [Gemini CLI](https://github.com/google-gemini/gemini-cli), [Qwen Code](https://github.com/QwenLM/qwen-code) and [OpenClaw](https://github.com/openclaw/openclaw), and [Checkly: The state of AI agent content negotiation](https://www.checklyhq.com/blog/state-of-ai-agent-content-negotation/), 19 February 2026
- [Next.js: Content negotiation](https://nextjs.org/docs/app/guides/backend-for-frontend#content-negotiation) and [proxy.js](https://nextjs.org/docs/app/api-reference/file-conventions/proxy), version 16.3
- [Astro: Middleware](https://docs.astro.build/en/guides/middleware/), [SvelteKit: Hooks](https://svelte.dev/docs/kit/hooks) and [Nuxt: server directory](https://nuxt.com/docs/4.x/directory-structure/server)
- [Netlify: Redirect options](https://docs.netlify.com/manage/routing/redirects/redirect-options/) and [Edge Functions API](https://docs.netlify.com/build/edge-functions/api/), and [Cloudflare Workers: static assets routing](https://developers.cloudflare.com/workers/static-assets/routing/worker-script/)
- [nginx: map module](https://nginx.org/en/docs/http/ngx_http_map_module.html) and [headers module](https://nginx.org/en/docs/http/ngx_http_headers_module.html)
- [Cloudflare: Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/), [Cache-Control](https://developers.cloudflare.com/cache/concepts/cache-control/) and [Vary](https://developers.cloudflare.com/cache/concepts/vary/)
- [Vercel: CDN cache](https://vercel.com/docs/caching/cdn-cache), [Netlify: Caching overview](https://docs.netlify.com/build/caching/caching-overview/) and [Fastly: Vary](https://www.fastly.com/documentation/reference/http/http-headers/Vary/)
- [Mintlify: Markdown export](https://www.mintlify.com/docs/ai/markdown-export), [Fern: Markdown](https://buildwithfern.com/learn/docs/ai-features/markdown), [ReadMe: AI discoverability](https://docs.readme.com/main/docs/ai-discoverability) and [GitBook: LLM-ready docs](https://gitbook.com/docs/getting-started/llm-ready-docs)

The examples are ours, and each is tested against the checks listed below at the version reviewed for this guide. The negotiation code was tested with Node.js 25 and the nginx configuration with nginx 1.30.5.

## Current scanner criteria

These criteria come from our current standards catalogue. They describe what Good for Bots checks, including usefulness rules of our own that the format does not require, and what our detection cannot see. Each report keeps the methodology of the scan that produced it.

### Markdown negotiation

active check · Reviewed 2026-09-25

We request the homepage and up to three pages chosen from the sitemap and homepage links with Accept: text/markdown, and inspect each response's status, media type, body and Vary: Accept header. Each page is graded on its own, and the median page decides. We reject HTML merely labelled as Markdown: a body that starts as an HTML document is HTML whatever its label, while Markdown that mentions HTML tags further in, in a code example for instance, is still Markdown.

**Results:** Pass: HTTP 200 labelled as Markdown (text/markdown, or the older text/x-markdown), at least 200 characters and Vary: Accept. Warn: useful plain text, a shorter Markdown response or a missing Vary: Accept. Fail: refusal, an error response, HTML or another representation, including the unregistered application/markdown.

**Limitations:** The size threshold is ours, and so is requiring Vary: Accept for a pass: HTTP recommends the header rather than requiring it. We do not establish that the Markdown contains every fact in the HTML. Pages outside the sample are not tested, and a sampled page that could not be read is left out rather than failed. Markdown at a separate URL is assessed separately.

[Full methodology and sources](https://goodforbots.com/standards#markdown-negotiation)

### Typed discovery links

active check · Reviewed 2026-09-25

We inspect the Link headers and link elements of the homepage and up to three pages chosen from the sitemap and homepage links for Markdown alternates, recognised discovery relations and absolute HTTP(S) extension relations. Ordinary canonical, icon and stylesheet links, and the REST API link WordPress adds to every page, do not earn points. Header parameters are read as RFC 8288 defines them, so a quoted type such as "text/markdown; charset=utf-8" is read whole.

**Results:** A page passes with a Markdown alternate or at least two distinct discovery relations, warns with one discovery relation and no Markdown alternate, and fails with no meaningful discovery link. The median page decides the result.

**Limitations:** Targets are not fetched by this check. Some accepted short relation names are vendor conventions, rather than registered relation types. The breadth threshold is ours and does not establish that a linked resource works. Pages outside the sample are not inspected.

[Full methodology and sources](https://goodforbots.com/standards#link-headers)

[Check your site](https://goodforbots.com/#scan)

## More guides

- [How to serve content AI crawlers can read without JavaScript](https://goodforbots.com/guides/server-rendered-html): Page content, reviewed Sep 25, 2026. Which AI crawlers run JavaScript, how to fix empty app shells in Next.js, Nuxt, SvelteKit or Vite, and how to test the raw HTML response yourself.
- [How to write robots.txt rules for AI crawlers](https://goodforbots.com/guides/robots-txt-ai-crawlers): Crawler access, reviewed Sep 25, 2026. Write robots.txt rules that treat AI training, search and user-triggered crawlers separately, with tested examples, the current token list and common fixes.
- [How to write a useful llms.txt](https://goodforbots.com/guides/llms-txt): Discovery files, reviewed Sep 24, 2026. Write an llms.txt that tells AI agents what your site covers: a complete tested example, the format explained, publishing checks and fixes for common mistakes.

[All guides](https://goodforbots.com/guides)

## Structured data

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "WebPage",
      "@id": "https://goodforbots.com/guides/markdown-negotiation#webpage",
      "url": "https://goodforbots.com/guides/markdown-negotiation",
      "name": "How to serve Markdown to AI agents with content negotiation",
      "description": "Return Markdown from the same URL when an AI agent sends Accept: text/markdown, with tested responses, Vary and CDN caching, and nginx and Next.js setups.",
      "inLanguage": "en",
      "isPartOf": {
        "@id": "https://goodforbots.com/#website"
      },
      "breadcrumb": {
        "@id": "https://goodforbots.com/guides/markdown-negotiation#breadcrumb"
      },
      "mainEntity": {
        "@id": "https://goodforbots.com/guides/markdown-negotiation#article"
      },
      "datePublished": "2026-10-03",
      "dateModified": "2026-10-03"
    },
    {
      "@type": "BreadcrumbList",
      "@id": "https://goodforbots.com/guides/markdown-negotiation#breadcrumb",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Home",
          "item": "https://goodforbots.com/"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "Guides",
          "item": "https://goodforbots.com/guides"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "How to serve Markdown to AI agents with content negotiation",
          "item": "https://goodforbots.com/guides/markdown-negotiation"
        }
      ]
    },
    {
      "@type": "TechArticle",
      "@id": "https://goodforbots.com/guides/markdown-negotiation#article",
      "url": "https://goodforbots.com/guides/markdown-negotiation",
      "mainEntityOfPage": {
        "@id": "https://goodforbots.com/guides/markdown-negotiation#webpage"
      },
      "headline": "How to serve Markdown to AI agents with content negotiation",
      "description": "Return Markdown from the same URL when an AI agent sends Accept: text/markdown, with tested responses, Vary and CDN caching, and nginx and Next.js setups.",
      "inLanguage": "en",
      "datePublished": "2026-10-03",
      "dateModified": "2026-10-03",
      "author": {
        "@id": "https://goodforbots.com/#organization"
      },
      "publisher": {
        "@id": "https://goodforbots.com/#organization"
      },
      "image": "https://goodforbots.com/og/guides/markdown-negotiation.png?v=3ctt1xjm5cm6c"
    },
    {
      "@type": "Organization",
      "@id": "https://goodforbots.com/#organization",
      "name": "Good for Bots",
      "url": "https://goodforbots.com",
      "logo": "https://goodforbots.com/icon-512.png",
      "founder": {
        "@id": "https://goodforbots.com/#person-mateusz-pawlica"
      },
      "contactPoint": {
        "@type": "ContactPoint",
        "contactType": "customer support",
        "email": "support@goodforbots.com",
        "url": "https://goodforbots.com/contact"
      }
    },
    {
      "@type": "WebSite",
      "@id": "https://goodforbots.com/#website",
      "name": "Good for Bots",
      "url": "https://goodforbots.com",
      "description": "Check if AI crawlers can access and read your website, free. Get a score out of 100 with every finding and fix prompts, then browse reports in the directory.",
      "inLanguage": "en",
      "publisher": {
        "@id": "https://goodforbots.com/#organization"
      },
      "potentialAction": {
        "@type": "SearchAction",
        "target": {
          "@type": "EntryPoint",
          "urlTemplate": "https://goodforbots.com/directory?q={q}"
        },
        "query-input": "required maxlength=120 name=q"
      }
    },
    {
      "@type": "Person",
      "@id": "https://goodforbots.com/#person-mateusz-pawlica",
      "name": "Mateusz Pawlica",
      "givenName": "Mateusz",
      "familyName": "Pawlica",
      "jobTitle": "Founder of Good for Bots",
      "description": "Web developer who builds directories and AI-powered products, and the founder of Good for Bots.",
      "url": "https://goodforbots.com/about#founder",
      "sameAs": [
        "https://pl.linkedin.com/in/mateusz-pawlica-65238816b",
        "https://pawlicaweb.pl/o-mnie"
      ],
      "knowsAbout": [
        "llms.txt",
        "robots.txt",
        "AI crawlers",
        "Markdown content negotiation",
        "Structured data",
        "Technical SEO",
        "Web directories",
        "Next.js",
        "TypeScript",
        "Generative AI"
      ],
      "worksFor": {
        "@id": "https://goodforbots.com/#organization"
      }
    }
  ]
}
```