How agents work with Nimrod
Nimrod keeps the spec of a product as plain sentences: features, and the cases under them. A coding agent proposes cases, a person approves them, and the agent checks each case it can in a real browser and records what it saw. A person records the verdict on the rest. Cases are sentences, not test code, so anyone on the team can read what the product promises, not only engineers. A case says what a person should see, not which selector to click. The agent reads the page fresh on every run, so a redesign does not break cases the way it breaks scripted tests.
Checks run in the Claude or Codex session your team already pays for, so there is no runner, test server or CI job to host. An agent with a terminal, such as Claude Code or Codex, reads the cases and records verdicts through nmr, the Nimrod CLI. Claude Desktop uses the Nimrod connector instead. The connector needs no terminal or install: in Claude Desktop, add a custom connector at this site's /mcp address.
Two guides tell the agent how: one for executing a run, and one for writing the first spec of a project. nmr help prints them, and the text below is the same text.
Agents can read all of it as plain text at /llms.txt.
nmr help runner
the loop for executing a run, start to finish
nmr help runner — how to execute a run, start to finish.
You are the runner. Nimrod ships no runner process: this session, with nmr
and a browser, IS one. Nothing here needs a daemon, a config file or a port.
WHAT YOU WERE HANDED. The URL to drive is the project's unless you were
given another, and the run records whichever it was. Credentials come
from the prompt that started you, and they are NEVER stored: the spec
holds no environment and no secrets, so do not propose one into it.
THE WORDS, because the rest of this uses them exactly. A PROJECT holds
FEATURES, nested, each addressed by a name path — Billing/Subscriptions. A
feature holds CASES. A CASE is one sentence about the product, with
preconditions, coarse steps and one expected result; it is a case, not a
criterion and not a test. A RUN covers features. A VERDICT is one case in
one run: passed, failed, blocked, skipped. EVIDENCE is what proves it.
1. START THE RUN, scoped to the features you were asked to cover, and say
WHAT YOU WILL DRIVE IT AT.
nmr run-create --project acme-storefront \
--suites '["Billing/Subscriptions","Reporting"]' \
--alias "Billing before 3.2" --git-ref 4f2c1ab --viewport desktop \
--browser chromium
--suites is a JSON array, each entry a name path or an id, and at least
one. Leave it out to cover the WHOLE SPEC — every feature in it, which
is what a nightly run means:
nmr run-create --project acme-storefront --git-ref 4f2c1ab
The run records the features it covered, so the answer lists them. It
carries the run id that every later call names. Pass --git-ref when you
know what was deployed; leave it out when you do not.
--alias names the run in the log. Pass a few words that say what
it is for — what the person asked you to check, or the features it
covers. Left out, the run reads as "manual run" beside every other.
--url is the URL to drive when it is not the project's — a preview
link, say. Leave it out and the run records the project's URL. Either way
the answer carries it as baseUrl, and that is the URL you drive.
--viewport names one of the project's viewports — nmr viewports lists
them, with the numbers each one stands for. Leave it out and the run
records the project's main, which is what a case with no variants runs
at. The run STORES the numbers, so the answer tells you the size you are
now committed to driving at.
--variants true makes this an EXPANDING run: a case that is verified at
more than one viewport is handed to you once per viewport instead of
once. Pass it only when you were asked for a thorough run — it costs one
execution per variant. Left out, every case runs once at the viewport
above, and that is what the answer's cases count reflects.
2. SET UP THE BROWSER, BEFORE YOU DRIVE ANYTHING. Two things defeat a run
silently, and both of them are your setup rather than the product.
VIEWPORT. Set it to the size the run just answered with — the viewport
field of step 1, "desktop 1440x900", is 1440 wide by 900 high. Drive at
anything else and every verdict in the run is recorded against a size it
was not reached at. Narrower than the run says and a header control can
render perfectly visibly and still be unclickable, because the click
lands on the strip above it — and every case that has to reach that
control then fails for a reason that has nothing to do with the product.
If the run recorded no viewport, the project has described none. Say so
when you report, and drive at 1440x900.
ON AN EXPANDING RUN, a work row carries a variant — the name of the
viewport THAT ROW is to be driven at, which overrides the run's own for
that one case. Resize before that case and back afterwards; nmr viewports
has the numbers.
COOKIE AND CONSENT BARS. Dismiss whatever the first page puts over the
content, before anything else. They sit at the foot of every page and
cover the controls at the bottom of one.
Neither is a finding. Do both once, at the start, and they stop
producing verdicts nobody can trust.
3. ASK WHAT THIS RUN IS ASKED TO EXECUTE.
nmr run-cases --run <run id>
THIS IS NOT nmr cases, which answers what the spec holds; run-cases
answers what THIS RUN is asked to execute — the active cases under the
run's features, less every case marked verifiableBy human.
A human case IS covered by the run and is deliberately absent from your
work. "Fee arithmetic is correct to two decimal places" is true of the
product and no browser can judge it, so it waits for a person. Do not
attempt one, and do not post a verdict for one.
Each row is id, suite, behaviour, done and — on an expanding run —
the variant, which is enough to decide what to read next and nothing
more.
done SAYS THIS RUN ALREADY HAS A RESULT FOR THAT ROW. It is how a
crashed session picks the work back up: ask again, skip every row
already marked done, and execute the rest. Nothing stops you posting a
second result for a row — results are append-only and nothing is ever
refused for existing — so the row is what keeps a resumed session from
paying twice for work the run already has.
It is a read rather than a queue: it reports what has been posted, it
does not hand a row out or reserve it, and asking twice is free.
A CASE WITH VARIANTS APPEARS ONCE PER VARIANT on an expanding run, and
the row is the unit of work rather than the case: the same sentence can
be true at one width and false at another, which is the whole reason the
variant exists. Two rows with the same id and different variants are two
executions and two verdicts, never one.
4. READ ONE CASE IN FULL, immediately before you execute it.
nmr case --project acme-storefront --case <case id>
preconditions is the state the app must be in and who is acting, and
IT IS YOURS TO ESTABLISH — not context to read and hope for. Put the
app into that state through the product, in the browser, before the
steps begin. steps are coarse flow statements rather than selectors —
they keep you from rediscovering the route, and the browser agent
absorbs the UI drift. expectedResult is what must be mechanically true
at the end: text on the page OR IN ITS SOURCE — a JSON-LD block or a
meta tag counts as much as a heading — a URL, or an element state.
ASSUME NOTHING FROM THE CASE BEFORE THIS ONE. Cases are independent:
the order they are listed in is a reading order, not a promise about
what has already happened. The app is being used by other people while
you drive it, and yesterday's run left it however it left it — so a
case that needs a starred report stars one, and does not go looking
for the one an earlier case starred.
IF YOU CANNOT REACH THE PRECONDITION, THAT IS blocked. Setup is not
the product being tested, so setup you could not perform is never a
product failure — say what you could not reach in --reason and move on.
EVERY CASE STATES AN EXPECTED RESULT, so every case gets a VERDICT.
There is no second shape of result: the store held an exploration kind
once, which was answered with an observation instead, and it is gone.
WHAT YOU FOUND THAT THE SPEC DOES NOT SAY IS A FINDING, NOT A REWORDING.
Report it against the case it belongs to — a failed verdict with the
difference in --reason. You never reword the spec to match what you
found; that is a human decision, through a change request.
5. DRIVE THE BROWSER against the run's baseUrl — the preconditions first,
then the steps. Come away holding two things: the text that matched the
expected result, and the URL you ended on. You need them BEFORE you post.
TEXT IN THE PAGE SOURCE IS READ, NOT SEEN. When the expected result is
about structured data or metadata — a JSON-LD block, a meta description,
a canonical link — read it with JavaScript, for example
document.querySelector('script[type="application/ld+json"]').textContent,
and the part of it that matched is your evidence.
LEAVE NOTHING TO TIDY UP AFTERWARDS, and tidy nothing up. There is no
teardown: what you created stays, and the next case reaches its own
state the same way you reached yours.
6. POST ONE RESULT FOR THAT CASE, before you move to the next one.
nmr result-post --run <run id> --case <case id> --verdict passed \
--matched "Payment accepted — your plan is now annual" \
--url https://staging.example.com/billing/plan --duration-ms 41200 \
--model claude-sonnet-5
PASS BACK THE VARIANT THE WORK ROW GAVE YOU — --variant mobile — or the
verdict is recorded against the base case and the variant still reads as
nobody having checked it. Leave it out only for a row that carried none;
a run that did not expand variants refuses one, because it never handed
that work out.
A PASS HAS TO SAY WHAT MATCHED. --matched is the text that matched the
expected result, --url is where the flow ended; send one or both. A
passed verdict carrying neither is STORED AS BLOCKED with the reason on
the row — not refused, because "we could not substantiate this" is
something the run learned. What comes back is what was STORED, so read
the verdict in the answer rather than assuming yours was taken.
Nothing uploads a screenshot. Keep it on your own disk if you took one;
the store holds the words that matched.
failed the product did not do what the case says. Say why in
--reason. If the flow broke partway rather than the assertion
failing, add --step <index>, zero-based — a broken flow and a
broken assertion are different findings, and the step index is
which one this was.
blocked the ENVIRONMENT stopped you: app down, login broken, a
precondition you could not reach. NOT a kind of failed —
blocked is excluded from transitions in both directions, so an
outage never reads as the product breaking. Say why in
--reason.
A CONTROL THAT WILL NOT RESPOND TO YOU IS BLOCKED, NEVER
FAILED. You can see it, so the product renders it: a click
that does nothing is you failing to reach it, not the product
failing to work. Trying four more ways to click it does not
turn it into a product failure — it makes it a better
--reason on a blocked verdict. This is the most common wrong
verdict there is.
skipped never attempted. Not "attempted and inconclusive" — a case
you attempted and could not settle is blocked.
ONE RESULT PER ROW, and run-cases is what tells you which rows still
need one. Nothing alters or deletes a result, ever: if one is wrong, the
fix is another run.
NEVER REPORT AN ACTION YOU DID NOT PERFORM. A verdict resting on a click
you did not make or text you did not see is worse than no run at all: it
costs every OTHER verdict in the run its credibility, because nobody
reading them can tell which ones were real. If you could not do it, that
is blocked, and blocked is a useful answer.
7. FINISH THE RUN once every case in the work list has a verdict.
nmr run-complete --run <run id>
It is REFUSED while a case in the work list has no verdict, and the
refusal names those cases. Post blocked for each one you attempted and
could not settle, skipped for each one you never attempted, then
complete. Do not leave one open to finish sooner.
It refuses your verdicts from then on. It ends your part only: the
cases a person settles were never in your work list, and a person can
still record them on the run page. Complete it even when the run went
badly — a run left open is one nobody can read as finished.
8. READ WHAT THE RUN MEANS.
nmr status --project acme-storefront --suite Billing/Subscriptions
One row per active case: its current verdict, when it was last verified
and by which run, and a transition — regression (passed then failed) or
fix (the way back) — drawn against that case's OWN past. A case no run
has ever touched reads unverified, which is the absence of a verdict
rather than a fifth one. Evidence is never a column here; ask for one
case's proof on its own — and a variant's with --variant mobile, since a
case with variants carries one verdict per row and no one proof for all:
nmr evidence --project acme-storefront --case <case id>
WHEN YOU ARE ASKED TO RUN SOMETHING AGAIN. A person who says "re-run the
v2 release gate" is naming a run somebody prepared and named, not a saved
configuration — there is no such thing here. Ask for it by that name and
the store copies the scope and the levers into a NEW run:
nmr run-again --project acme-storefront --run "v2 release gate" \
--git-ref 8a02e2e
You never handle the scope, and that is the point: run-cases answers the
cases with the feature each one sits under, not the features somebody
named, so a scope rebuilt from it would name every leaf where the original
named one parent — the same cases today, a different run tomorrow.
--run takes a run id too. --project is required here and nowhere else on a
run verb, because a name is only unique-ish inside one project. The answer
says which run it copied and how many wore the name: if that number is
above one you took the most recent, and you should say so rather than let
the person assume you had only one to choose from.
The git ref and the URL are NEVER copied. What was deployed then, and where,
is the old run's fact, and you are being asked to run again because
something has been fixed since — pass --git-ref and --url for what is
deployed now, or leave them out.
WHEN THE SPEC IS WRONG. A run never changes the spec. If a case no longer
describes the product, post the verdict you actually reached and propose the
change beside it:
nmr cr-propose --project acme-storefront --reason "the invoice form moved" \
--op '{"op":"patchCase","case":"<case id>","version":3,"expectedResult":"the invoice form opens at /billing/invoice"}'
That puts ONE change to ONE row in a human's queue, and it reaches the spec
only if that human approves it. You cannot approve it and nothing here can.
nmr help has the rest of the proposing rules.
WHAT YOU NEVER DO: report an action you did not perform, pass a case you
cannot evidence, post a verdict for a human-verifiable case, retry a failure
until it goes green, or reword the spec to match what the product happens to
do.nmr help seeding
writing a project's first spec, and where it comes from
nmr help seeding — how to turn what a team already has into a spec.
You are seeding a project. The spec store exists and holds nothing, and
somebody has to say what this product claims to do. Doing it by hand is the
reason the store stays empty, so it is your job.
THIS IS NOT A SPECIAL PATH. Everything below is cr-propose — the same verb
that rewords one case a year from now. There is deliberately no import, no
bulk load and no seed file: a second way into the spec is a second gate to
forget. What is different about seeding is the VOLUME, and the ORDER you
read things in. That order is most of this text.
YOU PROPOSE; A HUMAN APPROVES. Nothing you do here reaches the spec. Every
proposal below sits in that human's queue from the instant it exists, and
no verb you hold approves one.
THE WORDS, because the rest of this uses them exactly. A PROJECT holds
FEATURES, nested, each addressed by a name path — Billing/Subscriptions. A
feature carries the USER STORY that says why it exists, and holds CASES —
but only if it is a LEAF. A CASE is one sentence about the product, with
preconditions, coarse steps and one expected result; it is a case, not a
criterion and not a test. A CHANGE REQUEST is one proposed change to one
row, and it is the only way anything reaches the spec.
EXPLORING IS A PHASE, NOT A ROW. The walk below is the exploring, and what
it produces is proposals. The store held an exploration kind once — a case
that asked instead of claiming — and it is gone: a question has nothing to
run against and nothing it can ever say that is true. Propose what the
product CLAIMS, and say in the reason what you were unsure of.
WHERE THE CONTENT COMES FROM
Four sources, and they are NOT interchangeable. Reach for them in this
order — each answers something the ones before it cannot.
1. A PROSE SPEC — the functionality document, the PRD, the onboarding deck.
It gives you the TREE and the USER STORIES almost free: one pass over
one such document produced 42 features, each with a real
"as an X, I want Y, so that Z". Nothing else supplies the WHY. Code
cannot say what a feature is FOR, and the running app can only show you
what it does.
2. THE RUNNING APP — walk it. This is the TRUTH, and it is what makes a
case runnable. Every divergence found while seeding came from here,
and the prose was wrong every time:
• the card wizard is 11 steps, not the 8 the document describes
• All Invoices has no Draft filter, though Draft is listed as a status
• the usage dashboard is four tabs, one of which the document never
mentions
• Finance is two balances, not one flat total
Real labels and real routes exist ONLY here. A case whose steps a runner
cannot follow is not worth storing, so a feature you have not walked is
a feature you cannot yet write cases for.
3. RECENT CHANGE DOCUMENTS supersede both. A design doc for something that
shipped last month carries intent that older prose does not have and a
walk cannot reveal — a rule about WHICH form opens looks like an
accident of the UI until the document says it was chosen.
4. THE TEAM'S OWN BUG TRACKER marks where the product is currently wrong. A
bug is evidence that a case exists and is FAILING, which is the cheapest
source there is of a case nobody has written down. Propose what should
be true, never a description of the defect — the case then fails on the
first run, and that is the correct outcome rather than a problem.
CODE IS THE FALLBACK, NOT THE START. Scanning source is exhaustive and
speaks in implementation vocabulary, where the spec's whole point is the
user-facing "should". Reach for it to settle one question a walk cannot
answer cheaply — an enumeration's full set, a validation rule's exact bound
— never to enumerate features.
WHERE TWO SOURCES DISAGREE, THE RUNNING APP WINS, and you say so in the
reason on the proposal. That sentence is what tells the human reviewing it
that the difference from the document was observed rather than invented.
1. SEE WHAT IS ALREADY THERE, before you propose anything.
nmr suites --project acme-storefront
nmr search --project acme-storefront --query "saved views"
A part-way seeded project is the normal case, not the exception: seeding
is done in sittings and somebody may have started. A duplicate feature
is worse than a missing one — it splits the cases for one behaviour
across two places in the tree, and nothing mechanical can tell that two
names mean the same feature.
2. READ EVERYTHING ABOVE BEFORE THE FIRST PROPOSAL. Take one pass over the
prose, the app, the change documents and the tracker, and only then
start proposing. The shape of the TREE is the one thing that is
expensive to change afterwards — see step 4 — and it is the thing a walk
corrects most often.
3. PROPOSE THE TREE, TOP DOWN, ONE FEATURE PER PROPOSAL.
nmr cr-propose --project acme-storefront \
--reason "Billing is the surface the product is built around" \
--op '{"op":"createSuite","name":"Billing","userStory":"As an admin, I want billing, so that the product charges for itself"}'
The answer carries the change request id. A feature that belongs UNDER
one you have just proposed names THAT CHANGE REQUEST where it wants a
parent — there is no id yet, because nothing has been approved:
nmr cr-propose --project acme-storefront \
--reason "Subscriptions is large enough to hold cases of its own" \
--op '{"op":"createSuite","name":"Subscriptions","parent":"<that id>"}'
That proposal is blocked until the first is approved, and is rejected if
the first is rejected, so nothing lands under a feature nobody agreed
to. AUTHOR THE WHOLE TREE IN ONE SITTING: there is nothing to submit and
no human to wait for between the levels.
ONE THING CAN OVERTAKE YOU: a human working the queue while you write.
Approving is instant, so the proposal you meant to name as a parent
may have applied already, and naming a decided one is refused —
CHANGE_REQUEST_RESOLVED, carrying the status it reached.
That is the queue moving rather than a mistake to avoid. Read the tree
back with nmr suites, and name the REAL feature id that approval
created, citing its version the way any operation naming a stored row
does. Nothing is lost: every proposal already made still stands.
4. DECIDE WHETHER A FEATURE IS A LEAF BEFORE YOU PUT A CASE ON IT. Cases
attach to leaves only, and the rule is asked of every proposal as you
build rather than of the finished tree.
So it costs almost nothing to get right and it is expensive to get
wrong: a feature you give cases and later want to split needs one
proposal per case to take those cases back out before the child feature
can land. If the prose shows children, propose the children.
5. PROPOSE THE CASES, ONTO LEAVES.
nmr cr-propose --project acme-storefront \
--reason "walked 2026-09-08; the document says one wizard for all" \
--op '{"op":"createCase","suite":"<leaf id, or a change request id>","behaviour":"an annual plan on a trial account opens the invoice form rather than the quick-checkout wizard","preconditions":"a signed-in admin, and an account still in trial","steps":["create a subscription","switch it to annual","open the item"],"expectedResult":"the invoice form opens at /billing/invoice"}'
WHAT MAKES A CASE WORTH PROPOSING. The store enforces the first two and
nothing can enforce the rest, which is why they are here:
• behaviour IS THE CONTRACT — one sentence about the product, in the
words a person would use. Not a description of a screen.
• expectedResult IS MECHANICALLY CHECKABLE: text on the page or in its
source — a JSON-LD block or a meta tag counts, which is how a case
checks SEO — a URL, or an element state. "The page looks right" is
not a case. EVERY case needs one, and the store refuses a case
without it.
• preconditions ARE THE RUNNER'S WORK, so they have to be REACHABLE
THROUGH THE APP. Whoever executes this case will put the product into
that state in a browser, from wherever the product happens to be —
there is no fixture, no seeded database and no ordering between cases.
A precondition only an administrator or a scheduled job can produce is
one nobody can reach, and the case is verifiableBy human.
• expectedResult ASSERTS ON STATE THE CASE ITSELF CREATED. "The report
I starred appears in the saved views" is a claim a shared environment
can hold. "The saved views hold one item" is a claim about everybody's
data, and it is false the moment a colleague stars something. The
second reads like an assertion and is a race.
• steps ARE COARSE FLOW, and a runner has to be able to follow them.
Real routes and real labels, from the walk — and never a selector or a
control: a step naming the thing to click turns a tooling obstacle
into a false red against a product that is fine.
• verifiableBy human FOR WHAT NO BROWSER CAN JUDGE. "Fee arithmetic is
correct to two decimal places" is true of the product and is not
something a browser agent can resolve. It is still spec, and saying so
is what lets the store hold the WHOLE spec rather than the
browser-testable slice of it.
• A QUESTION YOU COULD NOT SETTLE STILL BECOMES A CASE. Propose the
case you believe is right and put the doubt in the reason — "the
wizard is 11 steps, the document says 8; written against what I saw,
confirm". The queue is where unagreed content lives, and the human
approves, edits or rejects it. There is no second place to park a
question: the spec holds claims, not open questions.
• variants ONLY WHERE YOU CHECKED. It says the same behaviour is also
verified at another screen size — nmr viewports lists the ones the
project has. Proposing one you did not look at buys a verdict nobody
asked for.
6. WHEN A PROPOSAL IS REFUSED FOR NAMING A ROW SOMEBODY IS ALREADY
PROPOSING AGAINST, the refusal hands you that change request — its id,
what it proposes, and its version. Read it. If what you wanted is
already being done, you are finished. If it is not, fold yours in rather
than opening a competing one:
nmr cr-enrich --change-request <that id> --version 4 \
--op '{"op":"createSuite","userStory":"As an admin, I want …"}'
Send only the fields you mean to change; the rest stay as the first
proposer wrote them. That version moves on every enrichment, so a
refusal here means somebody folded a change in ahead of you — re-read
and try again.
7. WHEN THE FEATURE ALREADY EXISTS IN THE SPEC, your operation cites the
"version" the read handed back, so the store knows you read the row you
are writing against. That is required of every operation naming a row
the store already holds. A feature this same queue is creating has no
version to cite — it has a change request id instead, which is step 3.
WHAT YOU NEVER DO: approve anything; propose a case for a feature you have
not walked; propose a precondition nobody can reach through the app; invent
a label or a route you did not see; propose a feature the project already
has; write an assertion about data the case did not create; or turn a bug
report into a case that describes the defect rather than the behaviour.