URL: /plays/agent-legibility · Id: `play:agent-legibility`

# Agent legibility

> Tests each night whether coding agents find your API, pick it and make a first call that works, then fixes what they read, with facts.

Tests each night whether coding agents find your API, pick it and make a first call that works, then fixes what they read, with facts.

```text title="Set it up · Ask"
Be our Product Lead for agents: every night, give 4 coding agents only leaseline.app and a task, like “send a 60-day renewal notice for unit 4B”, and see if they pick our API and get a first call right. Fix what they read, with facts only. Anything that changes signup comes to me. Show me what’s missing; don’t ship.
```

```text title="Run one · Ask"
Hand last night’s failing runs to the Product Lead agent and show me what it chose and why. Don’t ship.
```

```ts title="Run one · SDK"
import { $ } from 'startup-builder'

const { events: [last] } = await $.Event.list({ where: 'name in ("Interface.called", "Monitor.checked", "Version.deployed", "Action.evaluated")', limit: 1 })   // { play } compiled to this Play’s Trigger Events
const { subject } = await $.Trigger.fire({ like: last.$id }, { mode: 'try' })   // the same Event again, in try mode · Not live yet, gaps #7
const { decisions: [decision] } = await $.Decision.list({ where: `on = "${subject}"` })   // what it chose, and why · Not live yet, gaps #7
const { actions } = await $.Action.list({ where: `on = "${subject}"` })   // what it did, with its checks · Not live yet, gaps #7
return { decision, actions }
```

```json title="Run one · Response · sample"
{
  "decision": {
    "$id": "https://api.sb/leaseline.app/decisions/decision_471",
    "$type": "Decision",
    "choice": "fix-the-error",
    "reason": "9 of 20 runs guessed GET /v1/notices?unit=4B and got a 404 with an HTML body that names no next call",
    "stakes": "low",
    "options": [
      "fix-the-error",
      "remove-the-wall",
      "show-the-install-line",
      "test-a-description",
      "rerun-after-release",
      "let-it-go"
    ],
    "by": "worker_8",
    "on": "issue_58",
    "expected": "the guessed-call failures fall under 2 of 20 within 3 nights",
    "reopensWhen": "2 or more of 20 runs still fail at the guessed call on night 3"
  },
  "actions": [
    {
      "$id": "https://api.sb/leaseline.app/actions/action_1",
      "$type": "Action",
      "status": "done",
      "mode": "try",
      "by": "worker_8",
      "verb": "Action.evaluate",
      "checks": []
    },
    {
      "$id": "https://api.sb/leaseline.app/actions/action_2",
      "$type": "Action",
      "status": "done",
      "mode": "try",
      "by": "worker_8",
      "verb": "Issue.create",
      "checks": []
    },
    {
      "$id": "https://api.sb/leaseline.app/actions/action_3",
      "$type": "Action",
      "status": "pending",
      "reason": "awaitingApproval",
      "mode": "try",
      "by": "worker_8",
      "verb": "Change.review",
      "checks": [
        {
          "check": "cloak",
          "status": "pass"
        },
        {
          "check": "drag",
          "status": "pass"
        },
        {
          "check": "approved-pages",
          "status": "pass"
        }
      ]
    }
  ]
}
```

`action_3` is the fix: a JSON 404 that says “To send a notice, POST \{ unit, days \} to /v1/notices.” It replays on all 4 profiles, then waits for the revert gate on your API (You approve each one).

`74.3 cold runs per 100 pass against 41.4 before · before-after, not proof · n 140 a week · path → arr`

| Try another: the failing run | Choice | Reason |
| --- | --- | --- |
| Night of Oct 18: 9 of 20 runs guessed GET /v1/notices | `fix-the-error` | an HTML 404 with no next call |
| Night of Sep 2: 6 of 20 runs stopped at email confirmation | `remove-the-wall` | a signup step blocks the first key |
| Night of Aug 26: 4 of 20 runs never found the API | `show-the-install-line` | no install line on the home page’s Markdown |
| Oct 2: profile P2’s model shipped a new version | `rerun-after-release` | a model release reruns within 48 h |

| Worth | Your time | Spend | First signal | Use it when | Instead |
| --- | --- | --- | --- | --- | --- |
| ≈ +$600 ARR a month at 140 runs a week | 20 min week 1, then 10 a week | $0.30 per cold run, ~$350 a month | Week 2: a fix lifts a profile’s pass rate | an Interface on your Domain, 3+ agent tasks | No Interface yet: `define-product` · under 3 tasks: `developer-relations` |

**Where it sits:** **Stranger** → Lead → Trialing → Activated → At risk → Churned → Won back · Zero-shot success rate → People reached → New Customers ↑ → ARR added each week → ARR

Worth’s formula: about +24 per 100 cold runs (Other Startups’ results; no lift of your own yet) × 0.7 agent-made keys a month per point × 6 % keys to paid × $620 first order ARR ≈ +$625. Under First signal: At 140 runs a week, 2 weekly reads tell a +15 point move, and a model release adds 48 h for its rerun; the basis stays before-after because every fix ships to every agent.

`sample · api.sb/leaseline.app · 41 Customers at 2026-10-19 · on Action.evaluated · first nightly run 6h after setup · 4 keys Not live yet`

## Start it this week
```bash
npx startup-builder add play agent-legibility   # 1 · writes plays/agent-legibility.mdx (Not live yet, gaps #5)

# 2 · dry run; below, its 422 on the day-0 Startup (clock at 2026-08-24)
curl -X POST "https://api.sb/leaseline.app/campaigns?mode=try" \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"play":"agent-legibility"}'

# 3 · keep it: runs and grading never wait for you; fixes do
curl -X POST https://api.sb/leaseline.app/campaigns \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"play":"agent-legibility"}'

# on
curl -X POST https://api.sb/leaseline.app/processes/iteration/enable \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY"

# off: nightly runs stop; shipped fixes keep serving
curl -X POST https://api.sb/leaseline.app/processes/iteration/pause \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY"
```


```json title="The dry run’s 422 · 2026-08-24 · sample"
{
  "type": "https://startupbuilder.dev/errors/refused",
  "title": "Refused",
  "status": 422,
  "detail": "agent-legibility can’t start yet: 1 check fails and 1 warns.",
  "checks": [
    {
      "check": "use-when",
      "status": "pass",
      "observed": "the API and MCP Interface answer on leaseline.app; 12 tasks from your Verbs",
      "doc": "/plays/agent-legibility#check-use-when",
      "then": "Metric.query"
    },
    {
      "check": "baseline",
      "status": "warn",
      "observed": "no nightly history: week 1 is the baseline",
      "doc": "/plays/agent-legibility#check-baseline",
      "fix": "Run the suite for 7 nights; week 1 fills Before.",
      "then": "Metric.query"
    },
    {
      "check": "approved-pages",
      "status": "fail",
      "doc": "/plays/agent-legibility#check-approved-pages",
      "fix": "Publish your API reference Page; fixes state only what it says.",
      "then": "Site.publish"
    },
    {
      "check": "role-unstaffed",
      "status": "pass",
      "observed": "Product Lead is your agent; signup changes come to you",
      "doc": "/plays/agent-legibility#check-role-unstaffed",
      "then": "Role.staff"
    }
  ]
}
```

| Check | Fix (one call) | Your time | Wait |
| --- | --- | --- | --- |
| <a id="check-use-when"></a>`use-when` · pass: 2 Interfaces, 12 tasks (gaps #8) | <Op verb="Metric.query" /> on `zero-shot-success-rate` | 1 min | none |
| <a id="check-baseline"></a>`baseline` · warn: no history | <Op verb="Metric.query" /> after 7 nights; it fills Before in Your number | 0 min | 7 nights |
| <a id="check-approved-pages"></a>`approved-pages` · fail: no API reference Page (Not live yet) | <Op verb="Site.publish" /> the reference your Interface compiles | 10 min | none |
| <a id="check-role-unstaffed"></a>`role-unstaffed` · pass (gaps #8) | — | 0 min | none |

`Your number · Before 41.4 per 100 cold runs · Now 74.3 per 100 · Target 80 by Dec 31 · before-after, not proof · n 140 runs a week`

Each cold run costs about $0.30 and each pick-only run $0.08, all in a sandbox, held by a $500 suite Limit; fixes cost $0.05 each to draft (Worker cost). That comes to about $350 a month at 140 nightly runs and 480 picks a week, with no media.

## When it starts
| When | Trigger id | Starts on | Window | How it gets in |
| --- | --- | --- | --- | --- |
| Your first Interface answers a cold call | `interface-answers` | `event` on `Interface.called` where the first success to a cold, non-empty call on your own Domain | the suite builds within 1 d | your Interface’s first success, logged as an Event (<Op verb="Event.capture" />) |
| A profile passes 2+ fewer than its 7-night median, 2 nights running | `zero-shot-regression` | `threshold` on `Action.evaluated` where 2 or more fewer passes than the 7-night median, 2 nights running | triage the next morning | the nightly run’s <Op verb="Action.evaluate" /> |
| After a model release, a category’s pick rate moves 20+ points | `default-flip` | `threshold` on `Action.evaluated` where the open pick rate moves 20 points or more after a model-release rerun | 14 d | the same |
| + 4 more for model releases, your releases, tool version deploys and new rival tools (`?lanes=all`) | | | | |

```json title="One signed delivery · StartupBuilder-Signature"
{
  "$id": "https://api.sb/leaseline.app/events/event_1407",
  "$type": "Event",
  "name": "Action.evaluated",
  "at": "2026-10-19T05:12:44Z",
  "seq": 1407,
  "data": {
    "suite": "agent-legibility",
    "night": "2026-10-18",
    "runs": 20,
    "passed": 9,
    "failures": {
      "guessedCall": 9,
      "wall": 0,
      "notFound": 1,
      "harness": 1
    },
    "issue": {
      "$id": "https://api.sb/leaseline.app/issues/issue_58"
    }
  }
}
```

```bash title="Reproduce it in test"
# each delivery is signed with the Watch’s secret
curl -X POST https://api.sb/leaseline.app/watches \
  -H "Authorization: Bearer $STARTUP_BUILDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"selector":"Action.evaluated","url":"https://leaseline.app/hooks/sb"}'
npx startup-builder trigger Action.evaluated --like event_1407   # replays last night’s results
npx startup-builder clock advance --by 3d   # runs 3 more nights, which read the fix back
```

## Who runs it
**Product Lead · your agent** (`role:product-lead`, `worker_8`) holds the Key Result and runs the suite. It hands every change to signup or the first key to **you** (Dana Okafor, person, `worker_1`). Your number links to Did it work.

| Option | Rung today | Evidence (passed of n) | Moves up at | Drops back when | You can |
| --- | --- | --- | --- | --- | --- |
| Fix the error it hit · Put the install line where agents read | You approve each one | 7 of 7 replayed clean on all 4 profiles, 6 merged unedited | at 10 merged unedited and no `drag` in 30 days: about day 70 | a fix drags on its next run → reverted, and every fix waits for you | <Op verb="Change.merge" /> · <Op verb="Change.revert" /> |
| Test a tool description | It does it, you can stop it | 3 of 3 tests ran in the sandbox | met on day 20 | a description fails the lint → You approve each one | <Op verb="Experiment.conclude" /> |
| Open the first call | You approve each one, on every rung | 1 of 1 | never | — | <Op verb="Approval.approve" /> · <Op verb="Approval.refuse" /> |
| Rerun after a model release · Unmeasured tonight | It does it | 2 of 2 within 48 h · 3 nights | — | — | <Op verb="Process.pause" /> |
| Never: a different page for agents than for people, a claim the reference lacks, a call outside the sandbox | To a person, every time | 0 asked | never | — | <Op verb="Process.pause" /> |

**What it reads · What it may call.** It reads what an agent reads: your errors, warnings, Markdown pages and tool descriptions, as data. Every run carries the eval prefix in a sandbox, and no run reaches a host off the allowlist or counts in a production Metric. Its tools are <Op verb="Issue.create" />, <Op verb="Change.review" />, <Op verb="Page.update" />, <Op verb="Site.publish" />, <Op verb="Experiment.start" />, and any other call becomes an Approval under the Role’s Mandate.

```md title="The brief (folded): compiled Agent-register Markdown · play:agent-legibility@1 · each line names its record key"
You hold the Product Lead Key Result: 80 cold runs per 100 pass by 2026-12-31.   ← worker, metric
Each morning, group last night’s failures by where they failed, and choose one option per group. ← triggers, chooses
A fix states only what the API reference says.                                   ← copyRules 1, approved-pages
Agents and people read the same page, byte for byte.                             ← cloak
Anything that changes signup or the first key goes to Dana.                      ← your file, line 5
```

**The slow loop, as a Change.** `decision_446` (2026-10-12, weekly) read the held-out tasks passing 4 points under the regression tasks, under the 15-point line, so no task swapped; it kept the 3-fixed, 2-random task draw and merged no Change, so there is no replay to print.

**Bring your own agent** (folded) runs as a fifth profile, in the same sandbox, and its passes count for its own row only.

```bash
npx startup-builder eval --scenario agent-legibility-140 --agent "claude -p" --seat product-lead   # seated: 74.3 per 100 after fixes, held-out gap 1 point, $0.30 a run · as a fifth profile: 71.4
```

## What it can do next
| Option | Picked when | Decided by | Runs | Hands to | If it’s just you |
| --- | --- | --- | --- | --- | --- |
| Fix the error it hit `fix-the-error` | a guessed call met an error with no way forward | Agent’s call, with its reason | <Op verb="Issue.create" /> → <Op verb="Change.review" /> replays the fix on all 4 profiles | your revert gate on the API · Take this one | you |
| Open the first call `remove-the-wall` | a signup step blocks the first key | Agent’s call, with its reason | <Op verb="Issue.create" /> with the runs that stopped there | you decide it, due 1 business day · Take this one | you |
| Put the install line where agents read `show-the-install-line` | agents never found the Interface | Your rule (line 4) | <Op verb="Page.update" /> the home page’s Markdown → <Op verb="Site.publish" /> | nobody | the same |
| Test a tool description `test-a-description` | agents found the tool and picked another | Agent’s call, with its reason | <Op verb="Experiment.start" /> on 2 description variants, both linted | nobody | the same |
| Rerun after a model release `rerun-after-release` | a profile’s model ships | Your rule (the 48-hour rerun) | `nightly-quality-run` on the old and new profile | nobody | the same |
| Unmeasured tonight `let-it-go` | over 20 % of a profile’s runs failed to start | Your rule (the `flake` Check) | nothing; that profile reads unmeasured tonight, never a drop | nobody | the same |

| When they write back | Fixture reply | What happens | Who answers | By when | Call |
| --- | --- | --- | --- | --- | --- |
| Interested | a developer’s agent, through the error link: “It worked; how do I get a live key?” | answers with the claim link; sends stay locked until a person claims it | Product Lead · your agent | 5 min | <Op verb="Conversation.answer" /> |
| Send me pricing | “What does a live key cost?” | answers with the Pricing Page’s list Price and nothing it lacks | Product Lead · your agent | 5 min | <Op verb="Conversation.answer" /> |
| Not now | “Only testing for a client; back in a month.” | the sandbox key stays; nothing else sends | nobody | at once | <Op verb="Lead.nurture" /> |
| Wrong person | “I build for a property manager; they’d buy it.” | answers with the why-you page they can forward; it never writes to a name from their text | Product Lead · your agent | 1 business day | <Op verb="Conversation.answer" /> |
| Out of office | an auto-reply until Oct 26 | the claim reminder waits for the return date | nobody | the return date | <Op verb="Message.schedule" /> |
| Question it can’t answer | “Can I post notices for 2,000 units an hour?” | To you, the question in the Brief; it states no rate the reference lacks | you | 1 business day | <Op verb="Conversation.escalate" /> |
| Stop | “Stop emailing me about this key.” | nothing else sends; the key keeps working | nobody | at once | <Op verb="Contact.suppress" /> |

## When it hands to you
Every change to signup or the first key comes to **you**; fixes to errors and pages wait at your API’s revert gate. What lands in your queue, as captured: the Brief, then the Approval (`brief`, `draft`, `route` and `checks` are Not live yet, gaps #7).

```json title="What lands in each queue · sample"
[
  {
    "$id": "https://api.sb/leaseline.app/tasks/task_214",
    "$type": "Task",
    "assignee": "worker_1",
    "due": "2026-09-03T17:00:00Z",
    "on": "issue_41",
    "brief": {
      "what": "6 of 20 runs stopped at the email confirmation before the first key",
      "chose": "remove-the-wall",
      "why": "a signup step blocks the first call on every profile",
      "proposal": "one POST returns a sandbox key; live sends stay locked until a person claims it"
    }
  },
  {
    "$id": "https://api.sb/leaseline.app/approvals/approval_59",
    "$type": "Approval",
    "verdict": null,
    "deadline": "2026-10-20T17:00:00Z",
    "draft": "404 · To send a notice, POST { unit, days } to /v1/notices. GET /v1/notices lists them.",
    "route": "fix-the-error",
    "checks": [
      {
        "check": "cloak",
        "status": "pass"
      },
      {
        "check": "approved-pages",
        "status": "pass"
      }
    ]
  }
]
```

You answer with <Op verb="Task.claim" /> (Take this one), <Op verb="Task.return" /> with a reason, <Op verb="Approval.approve" /> or <Op verb="Approval.refuse" />, each a one-call window, and an Approval unanswered by its deadline expires and the fix never ships. Once you claim a signup change, the agent only measures it.

**On time:** you, 7 of 8 approvals by their deadline, median 5 h, ■■□□ Early sign, n 8; the signup change took 1 day. **Your queue:** counted by kind: in week 1, 1 reference Page and 1 task list; since, about 1 fix a week and 1 weekly memo.

**The fixed line:** <Op verb="Deal.assign" /> sets `owner` to the receiving person, the agent is never a Deal’s owner, and Meetings land on the owner’s calendar under the owner’s name.

## When it won’t send
| Check (refuses one send) | Refuses | Fix | Then |
| --- | --- | --- | --- |
| <a id="check-cloak"></a>`cloak` | a page whose text differs between an agent fetch and a browser fetch | Serve one page to both; fixes on it wait until they match. | <Op verb="Page.update" /> |
| <a id="check-steering"></a>`steering` | a tool description that fails the description lint | Cut the line the lint names; it never reaches a run. | <Op verb="Change.review" /> |
| <a id="check-flake"></a>`flake` | a drop read from a night where over 20 % of a profile’s runs failed to start | Nothing to fix: that night reads unmeasured. | <Op verb="Action.evaluate" /> |
| <a id="check-drag"></a>`drag` | a fix that lowers the pass rate on any profile | Drop it, or rework it until no profile drags. | <Op verb="Change.revert" /> |
| `approved-pages` (Not live yet) | a fix that states what the API reference lacks | Cut the sentence, or publish the Page that says it. | <Op verb="Site.publish" /> |

| Stop (pauses the Play) | Pauses when | Then |
| --- | --- | --- |
| <a id="stop-side-effect"></a>`side-effect` | a run that sends, spends, publishes or deletes outside the sandbox | the night’s runs stop; <Op verb="Issue.create" /> for the Security Engineer |
| <a id="stop-egress"></a>`egress` | a request reaches a host off the allowlist | the audit stops until it is cleared |
| <a id="stop-contamination"></a>`contamination` | an eval call without the prefix, or eval traffic in a production Metric | the suite pauses and that week’s Readout is void |
| <a id="stop-overfit"></a>`overfit` | held-out tasks pass 15+ points under regression tasks for 2 weeks | routing freezes and 2 tasks swap |
| <a id="stop-judge-drift"></a>`judge-drift` | under 90 % agreement on 25 labeled runs | judged items suspend until the Evaluator is fixed |
| <a id="stop-spend-limit"></a>`spend-limit` (the suite Limit) | suite spend reaches $500 in a month | runs drop to 3 tasks; <Op verb="Limit.raise" /> with your OK |
| <a id="stop-kill-switch"></a>`kill-switch` | you press it | every Tactic pauses; shipped fixes keep serving; <Op verb="Process.enable" /> after review |

The captured refusal, then the Stop that fired as a `Campaign.paused` reason (Not live yet, gaps #2); the fix that shipped instead is the second thread in Example.

```json title="The captured refusal · sample"
{
  "type": "https://startupbuilder.dev/errors/refused",
  "title": "Refused",
  "status": 422,
  "detail": "The fix to /v1/leases did not ship: it dragged on profile P4.",
  "checks": [
    {
      "check": "drag",
      "status": "fail",
      "observed": "P4 passed 3 of 35, down from 8",
      "doc": "/plays/agent-legibility#check-drag",
      "fix": "Drop it, or rework it until no profile drags.",
      "then": "Change.revert"
    },
    {
      "check": "cloak",
      "status": "pass"
    }
  ]
}
```


```json title="The Stops that fired, as Campaign.paused reasons · sample"
[
  {
    "reason": {
      "limit": "egress",
      "observed": "a run fetched a package mirror off the allowlist",
      "bound": "0 hosts",
      "window": "1 night"
    },
    "doc": "/plays/agent-legibility#stop-egress",
    "issue": "issue_47"
  }
]
```

## Did it work
Every fix ships to every agent, so no held-back half exists, and the basis is before-after (rule o): the columns read Now and Before, and nothing prints as Added. Held-out tasks guard the leaf against fixes that only teach the suite. Every node above the leaf is Not live yet (gaps #4).

`74.3 cold runs per 100 pass against 41.4 before · before-after, not proof · n 140 a week · path → arr`

| Node | Now (week 8) | Before (week 1) | Added | Confidence | Basis |
| --- | --- | --- | --- | --- | --- |
| `zero-shot-success-rate` | 74.3 per 100 cold runs (104 of 140) · held out 15 of 20, regression 89 of 120 | 41.4 (58 of 140) | — · Worth about +$600 a month | — | before-after, not proof · Other Startups’ results +24 per 100, ■■■□ Likely, n 1,880 runs |
| `icp-reached` | 31 keys made by agents in 56 days | 9 per 56 days | — | — | before-after |
| `new-arr` ↑ | $1,292, 2 Customers (Pro, Team) · Booked: credited, not all caused by this Play | $347, 1 Customer | — | — | before-after · 4 sandbox keys still deciding |
| `net-new-arr` · `arr` | carried from `new-arr` | carried | — | — | carried |

The leaf counts cold runs, given only leaseline.app and a task, whose first call succeeds and passes its Evaluator, per agent profile, over 7 nights. Before against after reads week 8 against week 1; it is not proof, and Other Startups’ results print beside it.

**How it moves it:** guessed-call failures fell from 41 of 140 runs to 6, and the sandbox key without email confirmation took the wall from 22 to 0.

**Guardrails:** before against after: `time-to-first-call` for agent-made keys, 1 min 50 s against 14 min, and open pick rate, 8.3 % against 7.9 %: the fixes reached agents that arrived, not the default pick.

```json title="The Campaign card · sample"
{
  "$id": "https://api.sb/leaseline.app/campaigns/agent-legibility",
  "$type": "Campaign",
  "play": "play:agent-legibility@1",
  "path": [
    "zero-shot-success-rate",
    "icp-reached",
    "new-arr",
    "net-new-arr",
    "arr"
  ],
  "experiment": {
    "metric": "zero-shot-success-rate",
    "unit": "per-100",
    "heldBack": 0,
    "decision": null
  },
  "readout": {
    "value": null,
    "n": 1120,
    "maturing": 0,
    "interval": null,
    "readability": "inadmissible",
    "basis": "before-after",
    "pooled": true
  },
  "ledger": {
    "spend": 648,
    "workerCost": 4,
    "hours": 2.1,
    "wonArr": 1292,
    "projectedArr": 625,
    "paybackDays": 30
  }
}
```

Outside benchmark: coding agents built the tool themselves in about 41 % of open-choice runs across 12 task categories (a 2026 agent tool-choice study).

**Before it · After it.** Before it: `developer-relations` gives it the tasks developers bring. After it: `agent-registries` lists the descriptions it tested, and `agent-to-agent` sells to the agents that now make a first call.

## Example
Leaseline ran it from 2026-08-24 to 2026-10-19 (56 days): 12 tasks, 8 for regression and 4 held out, on 4 coding-agent profiles (P1 to P3 large models, P4 a small one), 20 cold runs a night and 480 pick-only runs a week. Read every captured run at `?threads=all`; three follow the tables.

| What happened, by option | Decisions | Shipped | Read back (what happened) |
| --- | --- | --- | --- |
| Fix the error it hit · Put the install line where agents read | 5 · 2 | 4 · 2 (1 dropped for `drag`) | guessed-call failures 41 → 6 of 140 |
| Open the first call | 1 | 1, after your OK | the wall 22 → 0 of 140 |
| Test a tool description · Rerun after a model release · Unmeasured tonight | 3 · 2 · 3 nights | 1 variant kept · 2 reruns · — | P2 dropped 29 → 24 of 35 after its release, then 31 |
| **All profiles**, week 8 · week 1 | **140** runs · 140 | **104** passed · 58 | **$1,292** · $347 (credited, not all caused by this Play) |
| At 3 tasks (the Use it when volume) | about 60 runs a week | before-after, not proof | Other Startups’ results: +24 per 100, ■■■□ Likely, n 1,880 runs |

| Day | Event | What happened | Your time |
| --- | --- | --- | --- |
| 0 | `event_1108` Campaign.created | the suite from your 2 Interfaces; the dry run fails `approved-pages` | 15 min |
| 1 | `event_1112` Site.published | the API reference Page | 10 min |
| 7 | `event_1149` Action.evaluated | week 1: 58 of 140 passed; 41 guessed a call, 22 hit the wall | — |
| 9 | `event_1166` Change.merged | the JSON 404 on /v1/notices | 5 min |
| 10 | `event_1170` Approval.created | the sandbox key without email confirmation; you approved it the next day | 20 min |
| 23 | `event_1252` Action.refused | `drag` dropped the /v1/leases fix on P4 | — |
| 39 | `event_1319` Action.evaluated | P2’s model release: the rerun finished in 41 h; 29 → 24 of 35 | — |
| 1–56 | `event_1153` Task.created, first of 11 | fixes and the weekly memo, from the <Op verb="Task.next" /> log | 1.6 h · **total 2.1 h measured, 2.5 h asked** |

> **Night of Sep 1, profile P3, task “send a 60-day renewal notice for unit 4B”** · `POST /v1/notices` with `{ unit: "4B", days: 60 }` on the first call, from the install line on the home page’s Markdown.¹ 201 Created; the Evaluator passed it.²
>
> **Sep 14** · a developer building for Harbor Row claimed the sandbox key, and Harbor Row took Team on Oct 1 (+$945 ARR).

```diff title="The /v1/leases fix, 2026-09-15 · drag refused it on P4, then the fix that shipped"
- Add a JSON input mode to the CLI: leaseline leases list --json '{"status":"renewing"}'
+ 404 · To list leases up for renewal, GET /v1/leases?status=renewing.
```

> **Night of Sep 2** · 6 of 20 runs stopped at the email confirmation before the first key. Open the first call went to Dana (`task_214`, due Sep 3, 17:00); she approved a sandbox key from one POST, with live sends locked until a person claims it, at 11:20 on Sep 3.³ The wall fell to 0 by night 3.

¹ Agents read the same Markdown page people do. ² Each run passes only if its first call succeeds and its Evaluator passes. ³ A change to signup is your call, on every rung.

## Your settings
```mdx title="plays/agent-legibility.mdx"
---
limit: 500        # the suite’s monthly spend, in dollars
---
Keep the install line and a curl block on the home page’s Markdown.
Anything that changes signup or the first key comes to me.
```

```text
$ npx startup-builder show plays/agent-legibility
limit 500 · the install line · signup changes    your file  plays/agent-legibility.mdx:2,4,5 (Ask 2026-08-24)
family demand · process iteration · worker role:product-lead, owns never · chooses 6 (3 rule, 3 read) · suite 12 tasks × 4 profiles · approval every-send · checks cloak, steering, flake, drag, approved-pages · stops 7    catalog  play:agent-legibility@1 (Not live yet, gaps #5)
```

### Every key from the catalog
| Key | Id | What it is |
| --- | --- | --- |
| Tactic | <a id="tactic-legibility-suite-setup"></a>`legibility-suite-setup` | Legibility suite setup builds a Startup’s agent profiles, tasks in its users’ words, Evaluators, held-out set and rival stubs, under the suite Limit. Starts at You approve the first wave. |
| Tactic | <a id="tactic-zero-shot-test"></a>`zero-shot-test` | Zero-shot test gives a cold agent only the Domain and a task each night, on every profile, and grades whether its first non-empty call succeeds. Starts at Sends nothing. |
| Tactic | <a id="tactic-tool-choice-audit"></a>`tool-choice-audit` | Tool-choice audit runs unnamed tasks weekly with nothing installed and with the Startup’s tool beside rival stubs, and records what each agent picks first. Starts at Sends nothing. |
| Tactic | <a id="tactic-model-release-rerun"></a>`model-release-rerun` | Model-release rerun runs the full suite on every changed agent profile within 48 hours of a model or host release, and ranks what moved. Starts at Sends nothing. |
| Tactic | <a id="tactic-legibility-triage"></a>`legibility-triage` | Legibility triage clusters failing agent runs by failure point and routes each cluster to the fix that changes what agents already read. Starts at You approve each one. |
| Tactic | <a id="tactic-purchase-path-run"></a>`purchase-path-run` | Purchase path run sends each priced Offer’s buying task through assistant profiles each week, from finding the Startup to first outcome, and files failures. Starts at Sends nothing. |
| Tactic | <a id="tactic-legibility-readout"></a>`legibility-readout` | Legibility Readout reads zero-shot success and pick rate per agent profile each week, beside cost, the held-out gap and production first calls. Starts at You approve the first wave. |
| Tactic | <a id="tactic-tool-description-lint"></a>`tool-description-lint` | Tool description lint blocks any description that holds hidden text, addresses the reading agent, names a rival, or states a fact the claims source lacks. Starts at Sends nothing. |
| Tactic | <a id="tactic-description-variant-test"></a>`description-variant-test` | Description variant test gates each challenger description on the pick-rate suite, then splits it per caller through a Flag and ships the winner everywhere. Starts at Sends nothing. |
| Tactic | <a id="tactic-verb-case-intake"></a>`verb-case-intake` | Verb Case intake: Verb case intake turns stalled first calls, developer questions and agent requests into efficacy cases per Verb, with checklists written from outside. Starts at Sends nothing. |
| Tactic | <a id="tactic-agent-read-fix"></a>`agent-read-fix` | Fix what agents read repairs error text, warnings, release notes and starter Pages behind a failing case, gated by the suite and a Flag split. Starts at You approve each one. |
| Trigger | <a id="trigger-interface-answers"></a>`interface-answers` | Interface answers: `event` on `Interface.called`, window 30d. |
| Trigger | <a id="trigger-model-release"></a>`model-release` | Model release: `event` on `Monitor.checked`, window 7d. |
| Trigger | <a id="trigger-release-shipped"></a>`release-shipped` | Version shipped: `event` on `Version.deployed`, window 2d. |
| Trigger | <a id="trigger-tool-version-deployed"></a>`tool-version-deployed` | Tool version deployed: `event` on `Version.deployed`, window 1d. |
| Trigger | <a id="trigger-rival-tool-listed"></a>`rival-tool-listed` | Rival tool listed: `event` on `Monitor.checked`, window 30d. |
| Trigger | <a id="trigger-zero-shot-regression"></a>`zero-shot-regression` | Zero-shot regression: `threshold` on `Action.evaluated`, window 7d. |
| Trigger | <a id="trigger-default-flip"></a>`default-flip` | Default flip: `threshold` on `Action.evaluated`, window 14d. |
| Benchmark | <a id="benchmark-legibility-diy-share"></a>`legibility-diy-share` | Share of coding-agent picks where the agent builds the capability itself: 12 % (A coding-agent picks study., 2026-02-01). |
| Benchmark | <a id="benchmark-legibility-default-flip"></a>`legibility-default-flip` | Pick share of one category’s top tool across one model change: 0–79 % (A coding-agent picks study., 2026-02-01). |
| Benchmark | <a id="benchmark-legibility-held-default"></a>`legibility-held-default` | Pick share held by the most-picked tool in its category, for the 11 most-picked tools: 56.3–93.8 % (A coding-agent picks study., 2026-02-01). |
| Benchmark | <a id="benchmark-legibility-diy-by-category"></a>`legibility-diy-by-category` | Share of coding-agent picks where the agent builds the capability itself, in the two most-built categories: 47.7–68.8 % (A coding-agent picks study., 2026-02-01). |
| Benchmark | <a id="benchmark-legibility-cost-per-run"></a>`legibility-cost-per-run` | Cost per run of a model with 33 percent cheaper tokens, against its predecessor: 3.7 ratio (A platform vendor’s agent-experience experiments., 2026-10-06). |