Agent view
Status: draft 0.1. Fixes friction #1 (HTML noise) and #5 (JavaScript-only pages) at the source.
agent-index extracts a clean view from any page. A site can do better by serving that view itself, so no agent has to guess what the main content is.
Asking for it
An agent sends:
GET /tide-pools HTTP/1.1
Accept: text/markdown, text/html;q=0.8
A site that supports the agent view answers with Content-Type: text/markdown; charset=utf-8 and Vary: Accept. A site that doesn't simply returns HTML, and agent-index extracts it as usual.
A site can also advertise the view from its HTML:
<link rel="alternate" type="text/markdown" href="/tide-pools.md">
What the Markdown contains
The main content only, with no navigation, ads, cookie notices or footers. It starts with YAML front matter carrying the same fields agent-index extracts:
---
title: How tide pools survive low tide
author: Ana Ruiz
published: 2025-06-02T08:00:00Z
modified: 2026-03-14T10:30:00Z
canonical: https://coast.example/tide-pools
lang: en
next: https://coast.example/tide-pools?page=2
---
# How tide pools survive low tide
...
| Field | Required | Meaning |
|---|---|---|
title |
yes | The page's own title, without the site name. |
canonical |
yes | The one URL this content should be cited by. |
modified |
yes | When the content last changed in a way a reader would notice. Not the build time. |
published |
no | First publication. |
author |
no | Person or organization. |
lang |
no | BCP 47 language tag. |
next, prev |
no | Pagination, so agents never have to click "Load more". |
Caching
Sites should send ETag or Last-Modified on the Markdown response. agent-index revalidates with them, so a page that hasn't changed costs one 304.
Adopting it
agent_index.site.AgentFriendly is a drop-in ASGI middleware that serves this view from a site's existing HTML. agent-index's fetcher asks for text/markdown and uses the site's own view, skipping extraction and rendering, when it gets one.
Why Markdown
It is what models read best, it is small, it diffs cleanly, and it degrades to plain text for anything that doesn't understand it.