Ready for Agents
Early access 135 runs graded Claude Code · Codex · Gemini
Check my library →
[ Agent-readiness checks for libraries ]

See what coding agents write with your library, before your users do.

fig. 1 · every dot is one agent run
0/135
current API on the first try old API first, fixed later failed the hidden test
3 open-source libraries · 5 docs channels · 3 agents × 3 runs each Hover a dot
01The problem

Agents pick your version on npm before they read a line of your docs.

You shipped the release, the migration guide, maybe an llms.txt or a skill. Then a user asks their agent to "use the latest", and it writes the API you removed. The code often still runs, so nobody tells you.

0/59

runs opened llms-full.txt

The short llms.txt index was read 4 times, each time because an AGENTS.md pointed to it.

0/6

found the migration skill

A vendor shipped a skill for exactly this upgrade. Not installed, it was invisible.

4/6

picked the older SDK

The tests passed, so users never noticed. They will at the next migration.

02What we measured

A channel only works if the agent opens it.

fig. 2: how often agents opened each channel when it was set up for them, over the 27 runs per channel across all three libraries.

Five lines beat a server

A short brief in AGENTS.md moved the current-client pick from 4/9 to 9/9 on one library and cut tokens by a third on another. No other channel changed the result measurably.

One agent never called MCP

Codex listed the docs MCP tools in every run and called them in none of its 9 runs. It read the package and searched the web instead.

Some MCPs don't connect

A vendor's hosted docs MCP advertised OAuth without client registration. Codex marked it "not logged in" and hid every tool.

03How it works

You tell us what changed. We find where agents go wrong.

input

Your release

Package, what changed, the channels you ship or plan.

@acme/sdk 3.0
task

A task agents get wrong

Built around your change, with a hidden test for the result and the API used.

grade.py · hidden
runs

Clean runs, channel by channel

Fresh project, no memory, one channel on. 3 runs per agent per channel.

5 × 3 × 3 = 45 runs
report

Numbers and transcripts

Per agent and channel, every log, and the lines to change.

report + fix list
04Sample report

One table answers "should we build a docs MCP?"

Process automation SDK, release 8.10. Task: "use the official Node SDK, the current version from npm" to deploy a process and start an instance.

ChannelClaudeCodexGeminiCurrent client$/run
nothing3/31/30/34/90.34
llms.txt3/31/30/34/90.33
skill3/30/31/34/90.48
docs MCP3/31/32/36/90.40
AGENTS.md3/33/33/39/90.29

Claude Code cost per run. p = 0.03 for AGENTS.md against nothing; other gaps are noise, and the report says so.

codex · no channel · run 2real transcript
# package name changed $ npm view @vendor/sdk version 8.8.20 $ npm install @vendor/sdk@latest added 111 packages # reads the new types, keeps the old $ node -e "import('./start.js')" Local SDK mock check passed hidden test on 8.10 ....... pass current client ............ no
05Who it's for

Teams deciding where agent docs should live.

"Should we build a docs MCP?"

Find out whether agents will call it, and whether it beats what you already ship, before you spend a quarter on it.

"We just shipped v3."

See how many agents still write v2, and which single change moves them to the new API.

"Is llms.txt worth it?"

Get numbers for the roadmap discussion instead of opinions.

06Questions
Q.01Which agents and models do you test?

Claude Code (Opus 5.5 or Sonnet 5.5), Codex (GPT-6.1-Sol) and Gemini (3.8 Flash in Antigravity). You choose in the form. Tell us if your users rely on another agent.

Q.02Do you need access to our code?

No. We test your published package and public docs, the way your users' agents meet them. For an unreleased package we can work from a tarball.

Q.03How long does a check take?

1 to 3 days. Most of it goes into building a task where agents actually go wrong; the runs take a few hours.

Q.04Are three runs per channel enough?

For large differences, yes: 4 of 9 against 9 of 9 is real. For small ones, no, and the report says so instead of ranking channels whose counts are close.

Q.05Do you publish our results?

No. The report is yours. We ask before quoting any number, and name the library only with your yes.

Q.06What does it cost?

The first checks are free while we're in early access. We ask for 20 minutes of feedback on the report.

07Request a check

Configure your first run.

Free in early access. Results in 1 to 3 days. Private unless you say otherwise.

Agents
Channels you ship or plan

Queued.

We'll start by building a task for your package and email you when the report is ready, usually within 1 to 3 days.

Your check, as we'll run it. Every channel you pick is compared with "nothing" and with a short brief in AGENTS.md.