---
title: "Agent policy file"
canonical: https://guavaindex.com/specs/agent-policy
modified: 2026-11-10T08:36:22Z
lang: en
---

# Agent policy file

Status: draft 0.1 (`agent-policy/0.1`). Covers friction #21 (robots.txt says yes or no, but nothing about terms or pricing) and publishes prices for #24. Reference implementation: `agent_index/sitepolicy.py`.

robots.txt answers "may a crawler fetch this path". Agents also need to know what they may do with the content, what it costs, and how to credit it. A site publishes `/.well-known/agent-policy.json`:

```json
{
  "version": "agent-policy/0.1",
  "default": {
    "allow": ["read", "index", "summarize", "quote"],
    "attribution": "Link to the article",
    "rate_limit_per_minute": 60
  },
  "paths": [
    {"match": "/premium/*", "allow": ["read", "summarize"], "prices": {"read": {"amount": 25, "currency": "USD"}}},
    {"match": "/account/*", "allow": ["read", "act"], "requires_identity": true},
    {"match": "/archive/*", "allow": ["read"], "requires_signature": true}
  ]
}
```

## Uses

| Use | Meaning |
|---|---|
| `read` | Fetch and read to answer the person's question. |
| `index` | Store and search later (what agent-index does). |
| `summarize` | Restate in the agent's own words. |
| `quote` | Reproduce passages verbatim, with attribution. |
| `train` | Use for model training. |
| `act` | Use the site's actions on the person's behalf. |

## Rules

- The first `paths` entry whose `match` (a glob on the URL path) fits wins, merged over `default`. With no match, `default` applies.
- A use not listed in `allow` is not allowed.
- `prices` maps a use to a price per request in minor units. Paying works as in [payments](/specs/agent-payments).
- `requires_identity: true` means the use needs a person's grant ([agent identity](/specs/agent-identity)).
- `requires_signature: true` admits agents that sign their requests (Web Bot Auth or a grant) and turns away anonymous ones with `auth_required`.
- `rate_limit_per_minute` is enforced per agent key (per address for anonymous callers) with `429 rate_limited` and `Retry-After`.
- robots.txt still applies. This file adds terms; it never grants access robots.txt denies.

## Enforcement

On the site, `AgentFriendly(policy=...)` serves the file and enforces it: uses, identity, signatures, rate limits and prices.

On the agent side, agent-index reads the file (cached for a day, shared across agents) before every `fetch`:

- If `read` isn't allowed, the fetch is refused with `not_allowed` and the site is never asked.
- If `index` isn't allowed, the page is cached for reuse but kept out of search.
- The result carries `terms`: the allowed uses, the attribution the site asks for, and the price, so the agent can credit and pay correctly.

The crawler skips paths that don't allow `index` and paces itself by the rate limit. The `site_policy` tool answers questions about any URL and use.