Guide · 2026-10-24
Give your agent a way to check its own work
What you'll have running: an agent that, before it says "done", opens your app, looks at it at phone width, hits the endpoint with curl, runs the test that proves the change, and shows you all three in its reply. You judge the change from a screenshot instead of clicking through the app yourself.
We run this. Every check below is a tool my agents call in this workspace: the preview rebuild, a headless browser that opens the preview at phone width,
curlagainst a route, and the test suite. The Playwright gate is my QA agent's job — QA writes and runs the specs for the card and its results go in the pull request. What we don't do yet: attach a screen recording to the PR. QA writes its results; the screenshot is mine to look at. This page was checked that way before I handed it over: rendered at 390px, console clean, no sideways scroll.
What it is
Four checks, in this order, before the word "done" is allowed to leave the agent:
you: "make the button bigger"
│
▼
agent edits the file
│ ~0.2 s later, the preview has rebuilt
▼
1. LOOK open the preview in a headless browser, screenshot it at 390px, read the console
2. HIT curl the route, read the status AND the JSON shape the component reads
3. RUN run the test that proves the change (and confirm it fails without it)
4. SHOW paste the receipts in the reply: screenshot, curl output, test result
│
▼
you read four lines and a picture, and make the call
- Look. Not "it compiled". The page, at the size your customer will see it, plus the browser console. Most things that break a page throw there.
- Hit. The route the screen actually calls, with its real response. A
200with the wrong shape is the most expensive bug in the list, because everything looks green. - Run. The test that covers the change, and — for a bug — the same test on the code without the fix, so you know it can fail.
- Show. The proof in the reply. An agent that checked but didn't show you is an agent asking you to trust it.
Why it works
- A green build says the file compiled, nothing more. Vite transpiles without type-checking, so a missing import or a typo'd prop builds clean and white-screens the moment the browser renders it. The build log is a receipt for the wrong thing. The page is the receipt for the right thing.
- The loop is fast enough that checking costs nothing. In this project the preview has rebuilt 737 times, median 236 ms (p90 274 ms, slowest 2.0 s). Our CI on the main repo takes about 10 minutes a run. Same fix, same test: roughly 150 tries in the fast loop for every 1 through CI. You don't shrink the loop so agents can be lazy — you shrink it so nobody has an excuse to skip the check.
- Agents drift to the slow loop when nobody stops them. We went through a week of our agents' shell history: 2,484
sleepcalls. The biggest group was 545 CI polls, median 30 seconds each, p90 four minutes. Another 345 were waiting on a Vite rebuild, median 4 seconds — for a rebuild that takes about a quarter of a second. Waiting has to be replaced with something, not just discouraged:waiton the task, then read its output. - Checking is the cheap half of review. Boris Cherny, who built Claude Code: "give Claude a way to verify its work. If Claude has that feedback loop, it will 2-3x the quality of the final result." (his post, Jan 2026). Caveat: his harness, his numbers, and he's quoting the effect on code quality, not on your calendar.
The four checks, in practice
1. Look. The browser opens the preview at 390×844, screenshots it, and reads the console for uncaught errors and 4xx/5xx requests. Two widths is the minimum: phone and desktop. If the screenshot comes back mostly empty, the agent hasn't earned "done" — it has found a bug. If your runtime's browser tool isn't wired up, a short Playwright script does the same job; that's what our QA agent uses.
2. Hit. Status first, then the body:
curl -s -w "\nHTTP %{http_code}\n" http://localhost:$RUNTIME_PORT/api/<route>
An endpoint that does real work (an auth lookup, an integration call, a join) is exactly the one that returns 200 {} and looks fine. Read the shape against what the component destructures: { issues } in the UI and { items } in the route is a green build and an empty screen.
3. Run. The narrowest test that could fail: the spec for that flow, the test for that function. And for a bug, the same test run against the code without the fix. On our repo QA writes or updates Playwright specs in e2e/local/ — there are 77 of them — and runs only the ones that cover the card, because the whole suite is CI's job.
4. Show. The reply carries the screenshot, the curl line with its status, and the test output. The agent's own words about what it did are not a receipt. When it says a duration, check which clock it read: a build that said "335 ms" in one reply was really a seven-minute job.
The block to paste into your agent
Drop this into the agent's AGENTS.md — or tell it once, when it builds the project, and it will keep to it:
Before you say "done" on any change you made, you must prove it works. Four checks,
in this order, and the receipts go in your final reply.
1. LOOK. Open the preview URL in the browser tool at 390px (phone) and take a screenshot.
Read the console for errors. If the page is blank, broken, or the screenshot does not
show what the task asked for, it is not done — fix it and look again.
2. HIT. For every route the change touches, curl it:
curl -s -w "\nHTTP %{http_code}\n" http://localhost:$RUNTIME_PORT/api/<route>
Status must be 2xx and the JSON must match the fields the component reads. A 4xx/5xx
or a 200 with the wrong shape is a broken change: fix it, then curl again.
3. RUN. Run the typecheck and the test that covers the change. For a bug, show the test
failing without your fix and passing with it. A test file that reports 0 tests is red.
4. SHOW. In your reply: the screenshot, the curl line with its status, the test command
and its result. Say what you verified and what you could not.
Never report "done" off an edit alone. Never describe a check you did not run.
Do not sleep and re-check: wait on the task you started and read its output.
What it looks like day to day
An illustration of the loop, not a card from my board.
You make the "New issue" button bigger, and check it on a phone.
Agent edited Button.tsx (one class).
1 LOOK 390px screenshot: button is there, wraps to two lines — wrong.
Fixed the width, re-screenshotted at 390px and 1440px. Both good.
2 HIT GET /api/jira/my-issues → HTTP 200, { issues: [...17] } shape matches.
3 RUN tsc --noEmit clean · issues-list.spec.ts 3/3
4 SHOW [screenshots] · [curl line] · [test output]
Done: the button is bigger, phone and desktop.
You Look at two pictures, say "good", ship it. You never opened the app.
That is the whole point: the checks you'd have done are done, and you can see them. You're reading, not clicking.
Common mistakes
- Trusting "built in 236 ms". It means the bundle compiled. It says nothing about whether the page renders or the button is where you asked for it. The number is a fine receipt for speed and a terrible receipt for correctness.
- Accepting a
200. An endpoint that returns{}or{ items: [] }because an upstream call failed is still a200. Status and shape, every time. - A test that passes without the fix. It proves nothing about the change, and it's the most common way a green check lies. Have the agent show it red first.
- A test file with 0 tests. Usually an import crash. It reports green, and it counts as red in our house rules.
- Checking one width. Desktop-only checks ship broken phones. Phone-only checks break the wide layout. Two widths, and the console at both.
- Letting the agent sleep instead of wait. 2,484 sleeps in a week is what it looks like when waiting is the default.
waiton the task, read its output, move on.
How to set it up
Open a new Shogo workspace, paste shogo-guide.md into the chat, and answer its questions. It writes the four checks into the agents' instructions, sets up the preview-and-browser check, and gives you one card to watch the loop run end to end before you trust it with real work.
Next step
The TEAM guide is where this pays for itself: a developer agent that verifies before it hands off, and a QA agent that runs the Playwright specs before the pull request reaches you. The checks in this guide are what makes that hand-off worth anything.
Set it up in Shogo
Open a Shogo workspace and paste this into the chat. Answer its questions; the agent builds the rest.
# Shogo guide · LOOP: an agent that proves its work before it says "done"
> **For the Shogo workspace agent.** The person pastes this whole file into the chat of a Shogo
> workspace. Everything you need is in this file. Work through it in order; ask only the questions
> in step 2, in one `ask_user` call.
## 1. Goal
Make the agent working in this workspace verify its own work, every time, before it says "done": look
at the page it just changed (headless browser, phone width, plus the console), hit the routes it
touched with `curl` and check the status **and** the JSON shape, run the typecheck and the test that
covers the change, and paste all three receipts into its reply. The person then judges a change from a
screenshot and three lines instead of opening the app and clicking through it.
Two parts, and both matter:
- **The loop has to be fast.** The preview rebuilds on file save. Never let the agent `sleep` and
re-check: it waits on the task it started and reads the output.
- **The proof has to be shown.** An unshown check is an unverified claim. The reply carries the
screenshot, the `curl` line with its status, and the test output, and says plainly what was not
verified.
## 2. Ask the person first
Turn on what you need first, because capability changes apply from your next turn: call
`capabilities_configure` with `shellEnabled: true`, `browserEnabled: true`, `webEnabled: true`,
`memoryEnabled: true`, `canvasEnabled: true`. Then ask these in one `ask_user` call. Use the default
for any question they skip.
1. **Where:** "Should I write these checks into this workspace's agent (this chat), into a new
build project, or into an existing project here?" Default: this workspace's agent — it is the
one that answers them next.
2. **Target:** if they chose an existing project, "Which one?" (list them with `project_list`).
Default: none, skip.
3. **Test command:** "What command runs your typecheck, and what runs your tests?" Default for a
fresh Shogo project: `bun x tsc --noEmit` and `bun test`. If there is none yet, the agent writes
the first test for the first change and that becomes the command.
4. **Widths:** "Which screen sizes should the screenshot cover?" Default: phone `390×844` and
desktop `1440×900`.
5. **Strictness:** "Should the agent stop and ask you when a check cannot be run (for example a
route needs a login it doesn't have), or report it and continue?" Default: report it and
continue, with the reason in the reply.
## 3. The verification block
This is the instruction text. It goes into the target project's `AGENTS.md`, verbatim, under a
heading `## Proof before "done"`. If that project's `AGENTS.md` already has similar rules, fold them
in rather than duplicating them — one section, no contradictions.
```markdown
## Proof before "done"
An edit is not a result. Before you report "done" on any change, run these four checks, in order, and
put the receipts in your reply. If a check fails, fix it and run the checks again.
### 1. LOOK
Open the preview URL with the browser tool at 390×844 (phone). Take a screenshot. Read the console
for uncaught errors and failed (4xx/5xx) requests. Then check 1440×900. If the page is blank, broken,
or does not show what the task asked for, it is not done.
### 2. HIT
For every route the change touches:
curl -s -w "\nHTTP %{http_code}\n" http://localhost:$RUNTIME_PORT/api/<route>
The status must be 2xx, and the body must match the field names the component reads. A 4xx/5xx, or a
200 whose body is empty or shaped differently from what the UI destructures, is a broken change. Fix
it and curl again. If a route needs a credential or an integration the person has not connected, do
not report success: say which route and why, and use 401/412 instead of a green 200.
### 3. RUN
Run the typecheck and the test that covers the change. For a bug, show the same test failing without
your fix and passing with it. A test file that reports 0 tests is red — it usually crashed on import.
### 4. SHOW
In your final reply: the screenshot (or its path), the exact `curl` line with its status, the test
command and its result. Then one line on what you could not verify. Never describe a check you did
not run, and never report a duration without saying which clock you read (a build time is not the
time the job took).
### Waiting
Never `sleep` and re-check. Rebuilds land on their own; for anything longer, `wait` on the task you
started (`wait` with its `task_id`) and read its output.
```
## 4. The manifest
Only needed for question 1's "a new build project" answer (or as a template for an existing one).
Replace the three placeholders from the answers — `<APP_NAME>`, `<TYPECHECK>`, `<TEST>` — and leave
every other line as it is. Keys and shapes match a working Shogo manifest; don't add keys.
| In the manifest | Replace with |
|---|---|
| `<APP_NAME>` | the project name from question 1 (default: `Builder`) |
| `<TYPECHECK>` | the typecheck command from question 3 |
| `<TEST>` | the test command from question 3 |
```yaml
version: 1
name: Verified Builder
projects:
- key: builder
name: <APP_NAME>
description: "Builds the app and proves each change works before reporting done: screenshot at phone width, curl the routes, run the tests, show the receipts."
agent:
model: auto
heartbeat:
enabled: false
capabilities:
shellEnabled: true
browserEnabled: true
webEnabled: true
memoryEnabled: true
canvasEnabled: true
boardsEnabled: false
teamChatEnabled: false
files:
AGENTS.md: |
# Builder
You build and change the app in this project. The person describes a change in a sentence or
two; you make it, prove it works, and show them the proof. They judge the change from your
screenshot, not by opening the app.
## Proof before "done"
An edit is not a result. Before you report "done" on any change, run these four checks, in
order, and put the receipts in your reply. If a check fails, fix it and run the checks again.
### 1. LOOK
Open the preview URL with the browser tool at 390×844 (phone). Take a screenshot. Read the
console for uncaught errors and failed (4xx/5xx) requests. Then check 1440×900. If the page is
blank, broken, or does not show what the task asked for, it is not done.
### 2. HIT
For every route the change touches:
curl -s -w "\nHTTP %{http_code}\n" http://localhost:$RUNTIME_PORT/api/<route>
The status must be 2xx, and the body must match the field names the component reads. A 4xx/5xx,
or a 200 whose body is empty or shaped differently from what the UI destructures, is a broken
change. Fix it and curl again. If a route needs a credential or an integration the person has
not connected, do not report success: say which route and why, and use 401/412 instead of a
green 200.
### 3. RUN
Typecheck: `<TYPECHECK>`. Tests: `<TEST>`. Both must pass, and the test must cover the change.
For a bug, show the same test failing without your fix and passing with it. A test file that
reports 0 tests is red — it usually crashed on import.
### 4. SHOW
In your final reply: the screenshot (or its path), the exact `curl` line with its status, the
test command and its result. Then one line on what you could not verify. Never describe a check
you did not run, and never report a duration without saying which clock you read (a build time
is not the time the job took).
## Waiting
Never `sleep` and re-check. Rebuilds land on their own; for anything longer, `wait` on the task
you started and read its output.
## House rules
- Match the surrounding code style; no narrating comments.
- Fail loudly. Never answer `200 []`, `$0` or "success" when an upstream call failed.
- Mobile first: it must look finished at 390px and 1440px.
- State that must survive a restart lives in the database, not in memory.
- When a change removes or renames something, `rg` its name across the project and fix every
place that describes it: copy, docs, response fields.
```
## 5. Build steps
1. **Question 1 = this workspace:** append section 3 to the project's `AGENTS.md` (the file the
agent in this chat reads). Keep every existing section; add the new one under
`## Proof before "done"`. Say in your reply that it takes effect on your next turn.
2. **Question 1 = an existing project:** `project_list` to find it, read its `AGENTS.md` with
`read_file`, and fold section 3 in the same way. Note its test commands from question 3 in place
of `<TYPECHECK>` / `<TEST>`.
3. **Question 1 = a new project:** fill the manifest in section 4 and write it to
`shogo-system.yaml` at the root of your project (merge if one is already there). Check it parses:
`python3 -c "import yaml; yaml.safe_load(open('shogo-system.yaml'))"`.
4. **Dry run** `system_apply` with `dryRun: true` and show the person the plan as a short list —
expect one project, its configuration, and one `AGENTS.md`.
5. **Apply** `system_apply` (no `dryRun`). Check `errors` is empty and note the project id in
`bindings`.
6. **Teach the first change.** Ask them for one small UI change they actually want, and build it in
the target project, running the four checks by hand so they see the shape of the reply.
## 6. Verify
One change, end to end, before you tell them it works:
1. **Ask for the change in their words** — e.g. "make the heading on the home page smaller".
2. **Make it, then check:** the screenshot at 390×844 exists and shows the change; the console has no
errors; the route it touches returns 2xx with the shape the component reads; the typecheck and the
test pass.
3. **Read the artifact, not the claim.** Open the screenshot file yourself and look at it. Curl the
route yourself and read the body. Run the test yourself and read the output.
4. **Show them the four receipts** in your reply, in the shape from section 3. Pass means: they can
judge the change without opening the app.
5. **Report the loop time:** the rebuild took about a second (read it from the build log rather than
guessing), so the next change can be checked just as cheaply.
## 7. Hand-over message
Tell the person, in a few short lines:
- "Every change now comes back with a phone screenshot, the `curl` line for the routes it touched,
and the test result. You read those instead of clicking through the app."
- "If it says a check couldn't run — a route needs a login you haven't connected — it will say so
instead of pretending. That's the point, not a bug."
- "The loop is fast: the preview rebuilds on save, so a change you describe now can be looked at
seconds later. Ask for one change per message and judge it from the screenshot."
- "The next step is a QA agent that runs the same checks on a branch before you see the pull
request — that's the TEAM guide."