XONER4

My website xoner4.github.io scored 22/100 on Good for Bots (Poor). Good for Bots measures one thing: whether a language model can read a site and cite it.

I need you to: mark the content up so a parser can find it.

The check it fails is "Content is marked up so a reader can find it", worth 13.04 points. What the scanner saw: The homepage is barely structured: No <main> element and nothing carrying role="main".

Review the site's public page templates and shared layout components. Fix missing
content roots, heading structure and landmarks wherever they occur, including
templates outside the reported sample. Check the server's HTML response from each
page type: a shared layout fix can introduce a second `<main>` on pages that
already have one.

Semantic elements help a reader of raw HTML find the page's content and distinguish
it from navigation and other surrounding material.

Three things, all from the HTML standard. Change the markup; do not change what the page
looks like.

**1. One content root.** Wrap the page's dominant content in a single `<main>`.

```html
<body>
  <header>…site header…</header>
  <main>…the article, the docs page, the product copy…</main>
  <footer>…site footer…</footer>
</body>
```

The standard allows exactly one `<main>` per document that is not marked `hidden`, and
its ancestors may only be `<html>`, `<body>`, `<div>`, an unnamed `<form>` or a custom
element, which makes a `<main>` inside `<section>` or `<article>` invalid. Put the real
content inside it: a `<main>` that holds a sliver of the page while the article sits in a
sibling `<div>` marks nothing. If a framework template makes the element impossible,
`role="main"` on the wrapper is an acceptable substitute.

**2. A heading outline that holds together.** Exactly one idea per level, starting at
`<h1>`, and never skipping a level on the way down.

```html
<h1>What this page is about</h1>
  <h2>First section</h2>
    <h3>A detail</h3>
  <h2>Second section</h2>
```

`<h1>` → `<h3>` is invalid; `<h3>` → `<h2>` on the way back up is fine. Use heading
elements for headings and CSS for size: a `<div class="text-2xl">` is invisible to the
outline. If a heading is an image or an icon, give it real text and hide it visually
rather than leaving the element empty.

**3. Landmarks around the content.** At least two of these, outside `<main>`:

```html
<header>   <!-- the site header: becomes the banner landmark -->
<nav>      <!-- navigation -->
<footer>   <!-- the site footer: becomes contentinfo -->
<aside>    <!-- related content -->
<search>   <!-- a search form -->
```

Two rules that surprise people. `<header>` and `<footer>` only count as the page's banner
and contentinfo when they are children of `<body>`; inside `<article>` or `<section>` they
are that section's own header and footer and are not landmarks at all. And a `<form>` is
only a landmark when it has an accessible name, so give a search form
`aria-label="Search"`.

Do not add landmarks that describe nothing: five unlabelled `<nav>` elements are worse
than one. And do not wrap the whole body in `<main>`: that puts the header and footer
inside the content root, which removes the landmarks this asks for.

The report's content and navigation breakdown is diagnostic. It measures extracted
text in marked regions, not content quality. Keep useful navigation; do not remove
links or add filler to change a percentage. If much of the text is labelled "other",
check whether the real main content has a meaningful wrapper. The reported start of
main text follows HTML order, which can differ from the visual layout.

The whole report, as markdown, is at https://goodforbots.com/r/xoner4.github.io.md?scan=cmuuy4cv8001x01shnis058o4. Read it first: this prompt covers one finding, and the report has everything the scan saw.

When you are done, summarise what you changed and how to check it against a running instance. Do not try to run the Good for Bots scan yourself: it only sees what is already deployed, it is rate-limited, and re-running it is the site owner's call once the change ships.

My website xoner4.github.io scored 22/100 on Good for Bots (Poor). Good for Bots measures one thing: whether a language model can read a site and cite it.

I need you to: serve the content as markdown at its own URLs.

The check it fails is "Serves markdown at its own URL", worth 12.5 points. What the scanner saw: No /llms-full.txt and no markdown versions linked from llms.txt.

A model that can only reach this site's content through HTML pays to parse markup and
loses structure doing it: navigation, cookie banners and script tags arrive alongside
the prose, and headings, tables and code blocks come out flattened. Markdown at its own
address removes HTML from the path entirely: what is fetched is the content and nothing
else.

Two routes satisfy this, and it is worth being clear about which one the specification
asks for. llmstxt.org asks for a markdown version of each page **at the same URL as the
original**, either with `.md` appended (`page.html.md`) or with the extension replaced
(`page.md`). `llms-full.txt`, the single bundle holding everything, does not appear in
the specification at all. It is a community convention that grew up around it and is
now widespread. Both are accepted here. The per-page form is the one the document
asks for.

Either of these passes:

- **The bundle.** `/llms-full.txt`, one document, real markdown rather than HTML,
  carrying the site's actual content rather than a placeholder.
- **Per-page mirrors.** A markdown twin for each page at `page.md` or `page.html.md`,
  **and linked from `/llms.txt`**. That second half is load-bearing: mirrors are counted
  from the links in the index, so twins that nothing points at are invisible to this scan
  and, more to the point, to anything crawling the site.

For the mirror route, the index carries them as ordinary link lines:

```markdown
## Docs

- [Quickstart](/docs/quickstart.md): create an account and take a first payment
- [Concepts](/docs/concepts.md): customers, subscriptions, invoices and proration
- [Webhooks](/docs/webhooks.md): events, signature verification, retry schedule
```

Both `/docs/quickstart` and `/docs/quickstart.md` have to answer 200: the HTML for
people, the markdown for everything else.

Serving markdown through content negotiation does **not** satisfy this, and that is
deliberate rather than an oversight. A negotiated response only exists for a client that
knows to send `Accept: text/markdown`, and most crawlers send no preference at all. A
`.md` address sits in the open: it is found by ordinary crawling, it can be linked to,
and it can be cited. If this site already negotiates markdown, the report says so in
these words: *"Markdown is served on request, but not at any URL a crawler would find on
its own"*. That is a different check passing on its own terms, not a contradiction.

Four things about how this is measured. The bundle is judged by size, by being markdown
rather than a page or JSON, and by not merely repeating `llms.txt`; its content is never
sampled beyond that, so a couple of kilobytes of boilerplate would pass. Do not take that
as permission, since the point of the file is the content in it. `.mdx` counts as markdown
although the specification names only `.md`, because excluding it would punish a large
part of the documentation ecosystem for its toolchain. The number of mirrors we want
before calling it a convention on this site is our judgement, not the specification's.
And the mirror URLs listed in the index are counted, never fetched, so a list of `.md`
addresses that 404 passes this check and helps nobody at all.

The whole report, as markdown, is at https://goodforbots.com/r/xoner4.github.io.md?scan=cmuuy4cv8001x01shnis058o4. Read it first: this prompt covers one finding, and the report has everything the scan saw.

When you are done, summarise what you changed and how to check it against a running instance. Do not try to run the Good for Bots scan yourself: it only sees what is already deployed, it is rate-limited, and re-running it is the site owner's call once the change ships.

My website xoner4.github.io scored 22/100 on Good for Bots (Poor). Good for Bots measures one thing: whether a language model can read a site and cite it.

I need you to: publish an llms.txt at the site root.

The check it fails is "llms.txt is present and well formed", worth 11.25 points. What the scanner saw: No /llms.txt on this site.

A model that arrives at a site with no `llms.txt` has to work out what is here from
whatever the navigation happens to expose. A sitemap does not help it. That file was
written for a search engine and lists addresses without saying what any of them are
for. `llms.txt` is the one file written for the other kind of reader: a short, curated
index that says what this project is and which pages are worth fetching.

The format is defined at llmstxt.org, currently at version 2. It is a convention with a
written specification and real adoption, not a standards-body document: there is no RFC
behind it and no working group. Worth knowing before treating any corner of it as
settled.

The file must:

- be served at `/llms.txt`, at the site root, answering 200 as text rather than HTML;
- open with an H1 naming the project (a `#`, a space, then the name). The specification
  makes this the one required element, so a file without it is not llms.txt at all;
- carry a blockquote directly under that title saying what the project is, in the terms
  someone who has never seen it would need;
- list the pages worth fetching as bullet links, `- [Name](url)`, each followed by `:`
  and a note saying what is behind it;
- group those links under `##` headings that mean something (`Docs`, `API`, `Optional`)
  rather than one undifferentiated pile.

By convention the section named `Optional` holds secondary information: links an agent
can skip when it needs a shorter context. It is the only place in the format where a
site can say "this matters less", so use it rather than leaving everything equally
weighted.

A minimal, valid example:

```markdown
# Acme

> Acme is a hosted billing API for subscription businesses. These pages cover the
> concepts, the REST endpoints, and the migration path from the v1 API.

Start with the quickstart. The API reference is generated from the OpenAPI document
linked below, so it is always current.

## Docs

- [Quickstart](/docs/quickstart.md): create an account and take a first payment
- [Concepts](/docs/concepts.md): customers, subscriptions and invoices, and how they relate
- [Migrating from v1](/docs/migrate-v1.md): what changed, and the cut-off date

## API

- [REST reference](/api/reference.md): every endpoint, with request and response shapes
- [OpenAPI document](/api/openapi.json): the machine-readable schema

## Optional

- [Changelog](/changelog.md): release notes, most recent first
```

Two things get this wrong more often than anything else.

**A catch-all route answering for the file.** A framework that serves its application
shell for every unknown path returns 200 and a page of HTML at `/llms.txt`, which is
worse than a plain 404 because nothing signals that the file is absent. Verify from
outside the application: `curl -s https://xoner4.github.io/llms.txt | head -5` must print markdown,
not `<!doctype html>`. On most stacks the answer is to put the file in the directory
that is served verbatim (`public/`, `static/`, `www/`) rather than in the router.

**Generating it from the sitemap.** A dump of every URL the site has is a sitemap with
different punctuation. This file is a curated index: a few dozen entries a person chose,
each carrying a note. A generated wall of bare links satisfies the format and helps
nobody, and it fails the companion check that asks whether the index describes anything.

Two limits of this scan are worth knowing, because neither is a reading of the
specification. The specification allows the file at any subpath (`/docs/llms.txt`
covers the URLs beneath `/docs/`), but this scan requests the site root and nothing
else, so a file that exists only under a subpath is reported as absent. And the URLs
inside the file are never fetched, so an index pointing at pages that have moved still
reads as well formed here.

The whole report, as markdown, is at https://goodforbots.com/r/xoner4.github.io.md?scan=cmuuy4cv8001x01shnis058o4. Read it first: this prompt covers one finding, and the report has everything the scan saw.

When you are done, summarise what you changed and how to check it against a running instance. Do not try to run the Good for Bots scan yourself: it only sees what is already deployed, it is rate-limited, and re-running it is the site owner's call once the change ships.