Working with a coding agent

For when a coding agent works in your own repo: four things to set up first, a gate you can copy, and three ways a green pull request still goes wrong.

1. Instructions file: AGENTS.md at the root says how to test, what done means and what not to touch.
2. Branch protection: nobody pushes to main. A merge needs the gate to pass and a person's review.
3. Gate job: one required check that runs your tests and your saved eval set (Agent evaluation). A miss stops the merge.
4. Secrets out of reach: the agent's jobs get no deploy or production secrets; only the deploy job on main does. An issue can ask for one (Prompt injection). If your eval calls an AI model, give it a test key with its own limits.

A sketch, not a tool. Start the instructions file like this:

# AGENTS.md (sketch: your commands, your paths)
Test: npm test, then npm run eval.
Done: the issue's "Done when" line passes.
Scope: one change. Ask before touching 10+ files.
Never edit: .github/ ci/ deploy/ settings/
Never read, print or ask for secrets.

Then one script your CI runs on each agent pull request:

#!/bin/sh
# Sketch: ci/gate.sh. Paths and commands are yours.
# CI runs main's copy, so a branch can't edit it:
#   git show origin/main:ci/gate.sh | sh
# Fetch main with history first, not a shallow clone.
set -eu
BASE="${BASE:-origin/main}"
MAX="${MAX_FILES:-10}"
LOCKED='^(\.github/|ci/|deploy/|settings/)'
TEST='\.test\.js$'
stop() { echo "GATE STOP: $*"; exit 1; }
hold() { echo "GATE HOLD: $*. A person decides."; exit 1; }
diffs() { git diff --name-only "$@" "$BASE"...HEAD; }

ALL=$(diffs)
N=$(printf '%s\n' "$ALL" | grep -c . || true)
[ "$N" -gt 0 ] || stop "empty diff"
[ "$N" -le "$MAX" ] || stop "$N files. Split the change"
if printf '%s\n' "$ALL" | grep -Eq "$LOCKED"; then
  hold "locked path"
fi
if diffs --diff-filter=MD | grep -Eq "$TEST"; then
  hold "an existing test changed"
fi
CODE=$(printf '%s\n' "$ALL" | grep -E '\.js$' |
  grep -Ev "$TEST" || true)
if [ -n "$CODE" ]; then
  printf '%s\n' "$ALL" | grep -Eq "$TEST" ||
    stop "code changed, no test"
fi
npm test || stop "tests"
npm run eval || stop "saved eval set"
echo "GATE PASS: $N changed"

• Green CI, no test of the change: nothing covers the new code, so old tests pass. The gate stops code without a test. Better: run a new test once on main. It should fail there.
• Wide diffs: 40 files for a one-line bug won't get a real review. The gate stops past your file limit.
• Workflow-file edits: a pull request that edits its own check can turn it green. The gate holds locked paths, and CI runs main's copy.

A stop goes back to the agent; a hold goes to a person, as Glance leaves deploy settings with their owner. The gate checks a change's shape, not its meaning: you still read the diff.

Your agent's pull request adds a feature, touches no test and edits ci/gate.sh. CI is green. What should have stopped it, and why distrust the green?