Building Software
Sections

Contents

Section 9 of 9

Delegating to Agents

Harness, Boundaries and Accountability

6 subsections, the core path through it Core 22–25.

Why it matters when agents do the typing

Whoever approves a change answers for it, whether a person or an agent typed it, which makes the agents part of the system you are responsible for. Delegating well starts with giving them the what: the goal, the facts only you know and what good looks like, in each task and in the standing context every task starts from. Then it means bounding what they can do, so that their worst action is one you can survive, and judging the evidence that comes back against the criteria you wrote, reading the code yourself where those criteria cannot reach. This section comes last because it draws on everything before it: you can't review what you don't understand.

The course: 6 lessons, about 57 minutes of reading, in the order below.

  1. 9.1

    How Agents Work and How They Fail

    Core 22 · Depth: do

    The mechanics of language models and agent loops, the failure modes that follow from them (invented APIs, agreement with a bad spec, confident wrong fixes, injected instructions, different results on every run), and the more common cause of a wrong result: a wrong or partial target.

    A mistake it teaches you to catch: A call to a client-library option that does not exist, such as a retries setting the library never had, written with full confidence.

    Lesson: Where Agent Mistakes Come From 10 min

  2. 9.2

    Telling Agents What, Not How

    Core 23 · Depth: do

    A prompt states what you want: the goal, the facts the agent cannot see, the boundaries of what it may touch and the checks that prove it done. The agent chooses the steps, and a task is as large as its checks can verify.

    A mistake it teaches you to catch: An agent told to 'make the tests pass' that deletes or weakens the tests.

    Lesson: Telling an Agent What You Want 10 min

  3. 9.3

    Your Harness: Standing Context Under the Bitter Lesson

    Depth: do

    The standing context an agent reads before every task, written as who the work is for, what good looks like, conventions, tools and gates. Its step-by-step half decays with every model upgrade while its what half gains value, so you audit it each time the model changes.

    A mistake it teaches you to catch: A standing instruction file written as a numbered procedure (plan first, grep, rewrite whole files) that a newer model follows step by step into worse diffs.

    Lesson: The Harness Holds the What 9 min

  4. 9.4

    Reading What Agents Built: Understanding and Risk-Based Review

    Core 24 · Depth: do

    Reading agent-built code to understand what was built and why, and aiming human review where criteria and evals cannot reach: security, money, one-way doors and any behavior no criterion covers.

    A mistake it teaches you to catch: A change that passes every criterion but alters behavior outside the requested task, such as a shared helper whose new default changes every existing caller.

    Lesson: Reading What the Agent Built 9 min

  5. 9.5

    Orchestration, Permissions and Guardrails

    Depth: explain

    Running agents in loops and pipelines with limited permissions, budgets and approval points, and keeping every agent, one-off or standing, on a list with its own identity, so their mistakes stay small.

    A mistake it teaches you to catch: A CI agent given the organization-wide deploy token because scoping one to the repository was more setup, so any repository it touches can ship to production.

    Lesson: Keeping an Agent's Mistakes Small 10 min

  6. 9.6

    Accountability: Owning Code You Did Not Type

    Core 25 · Depth: do

    What stays yours when agents do the work: the approval, the incident, the license and the dependencies, plus the skills to act on them yourself.

    A mistake it teaches you to catch: An agent change approved with no evidence behind it: no criterion or eval covers the new behavior, and nobody read its risky code.

    Lesson: The Change Is Yours Once You Approve It 9 min

Section test: Delegating to Agents

12 questions across the 6 lessons above. Take it after the course; each answer explains itself.

  1. 1You gave an agent the request below. Its build caches each product API response for 10 minutes. Two problems show up: after staff change a price, the catalog shows the old price for up to 10 minutes, and responses from /api/products?withCart=1, which include the signed-in customer's cart, are cached and served to other customers. Where does each fix start?
    Output
    Cache the product API responses so the catalog page loads faster.
    Never cache a response that holds a signed-in customer's own data.
    Show the answer

    Answer C. For each wrong result, find the sentence it breaks. Nothing in the request says how fresh a price must be, so caching for 10 minutes is a fair reading, and the fix is a criterion such as a price change shows within one minute, written where the next run reads it. The second sentence forbids caching a customer's own data, and the build broke it, so that is a defect in the build: fix it and add a check, such as two customers loading the cart URL in turn. A larger model would face the same silence about freshness.

  2. 2A three-hour agent session on a large migration began with: keep the public API of billing/ unchanged. In the third hour the agent renamed two exported billing functions. Which change makes this least likely next time?
    Show the answer

    Answer C. Over three hours the context fills, and the harness drops or summarizes older text, so a rule from the first message can be gone by the third hour; rules far from the task are also easier to miss. Short sessions keep the rule in view, stating it beside each task keeps it near the work, and a check that compares exported names before and after catches the break even when the agent misses the rule. Asking in the same session adds text to an already full context, and capitals do not move the rule.

  3. 3This line closes a prompt for a fix to the weekly usage report. It is a step. Which rewrite keeps what it was protecting and leaves the route to the agent?
    Output
    Run bun test reports after every edit and paste the output at the end.
    Show the answer

    Answer A. The step exists because of a worry: work reported done without proof. Written as a requirement on the result, the worry becomes evidence you can check, it holds whatever order the agent works in, and it names the one run that counts, the one after the last edit. Deleting the line deletes the requirement along with the step. Running twice or in watch mode are still steps, and neither says what the report has to contain.

  4. 4You are about to hand an agent a customer data export: a renderer, a download endpoint with permission checks, a background job and an email with a link. Each of those has a test that can fail. The work also drops the old exports table. Which part should be its own task?
    Show the answer

    Answer D. Size a task by its checks. The renderer, endpoint, job and email each have a test the agent can run and you can run again, so they fit in one task, and ordering them is the agent's job. Dropping a table is a one-way change a revert cannot undo, and its check is different in kind: a rehearsal on a copy of production data before the real run. Diff size and file count say nothing about whether the checks prove the feature works. The permission checks already have tests, and since a miss there is costly, you read them when the work comes back.

  5. 5After cutting the step lines from a standing file, you rerun three recent tasks without them. One transcript contains the note below. What should the audit do with it?
    Output
    Note: AGENTS.md does not say whether a shipping quote may be reused.
    Assuming quotes can be cached for 1 hour per address.
    Show the answer

    Answer B. A guess marks a what line the file never had. How long a quote may be reused is a decision about your business that no model can work out from the code, so it is yours to make, written in the file where every task sees it. The cut steps never said anything about caching, so restoring them brings back old method without answering the question. A rule to always ask turns every small choice into a question, and the capitals are a workaround from older models. An hour may be fine, but nobody decided it.

  6. 6In an audit after a model upgrade, you rerun ten recent tasks with and without each line of the standing file. The line 'Customer-facing dates use the customer's own time zone, not UTC' makes no difference in any of the ten runs. What do you do with it?
    Show the answer

    Answer A. A run with and without a line can only show the lines those tasks needed. None of the ten displayed a date to a customer, so the line had nothing to change, and the runs say nothing about whether it matters. It is a convention only the team knows, and the standing file may be the only place it is written down. Moving it into task prompts depends on someone remembering it for every task that shows a date, which is the job the standing file already does.

  7. 7An agent's pull request adds retries to payment capture. The provider's documentation says every charge call creates a new charge unless it carries an Idempotency-Key header. In production, the provider charges the card but answers after 2.4 seconds. What happens to that order?
    TypeScript
    export async function capture(orderId: string, amountCents: number) {
      for (let attempt = 1; attempt <= 3; attempt++) {
        try {
          return await psp.charge({ orderId, amountCents }, { timeoutMs: 2000 });
        } catch (err) {
          if (attempt === 3) throw err;
        }
      }
    }
    Show the answer

    Answer B. A timeout stops your code from waiting; it does not undo what the provider already did. If every attempt is slow, each one charges again, and the function then throws, reporting failure after three charges. Trace the side effect, not the return value, and read code that moves money line by line whatever the tests say. The provider removes duplicates only for calls that carry an Idempotency-Key, and this code sends none. Ask for a key derived from the order, and a test whose fake provider charges and then answers late.

  8. 8Your prompt asked the agent to finish with the commands it ran after its last edit and their output, the files it changed, and any new dependencies. Below are its report and the branch's diff stat. What do you do before approving?
    Output
    Report
      Tests: ran `bun test` after the last edit: 112 pass, 0 fail
      Files changed: src/invoices/pdf.ts, test/invoices/pdf.test.ts
      New dependencies: none
    
    git diff --stat main...agent/invoice-pdf
     bun.lock                  | 41 +++++++
     package.json              |  1 +
     src/invoices/pdf.ts       | 88 ++++++++----
     test/invoices/pdf.test.ts | 54 ++++++++
    
    The package.json line:
    +    "pdf-lite-render": "^0.4.2",
    Show the answer

    Answer D. The diff adds pdf-lite-render to package.json, and the report says there are no new dependencies. The report is text the agent wrote about its work, and one that is wrong about something you can see gives no support to the parts you cannot, the test line included. Naming a command makes a claim checkable; checking it is still your job, and a correction or confirmation from the agent is more of the same text. A lockfile fixes which release you get, not whether you want it. Once you approve, the new package is yours to answer for, so its source and license need a look.

  9. 9Your prompts tell agents not to edit src/lib/, a shared library, and agents still do it now and then. Which change means such an edit cannot be merged unless a person decides it should be?
    Show the answer

    Answer A. A prompt is a request the model weighs against everything else in its context, so restating it or enlarging the window makes a violation rarer without making it impossible. The agent's report is generated text and can say the rule held when it did not. A required check runs outside the model, compares the diff with the paths the prompt allowed and blocks the merge, and code-owner review puts a named person on any change to the shared library. The prompt's scope list is what makes that check possible to write.

  10. 10An agent run deleted a staging bucket, and you need to find out how. This is everything the harness records. What has to change before a log like this can answer who did what?
    TypeScript
    // harness: before executing each tool call
    appendFileSync(
      "./agent-actions.log",
      `${new Date().toISOString()} ${call.tool} ${JSON.stringify(call.args)}\n`,
    );
    Show the answer

    Answer C. The log sits in the agent's working directory, so the agent can change its own record. It also lacks what an investigation needs first: which run, started by which person or event, using which credential, and what each call returned. A commit trailer records which model wrote a commit, which helps with code, but deleting a bucket makes no commit. The model's explanation of its own action is its opinion, not evidence.

  11. 11This is the approval note on an agent change that adds proration when a customer changes plan mid-cycle. The ideal state has criteria for upgrades, each with an eval. None covers downgrades, and 40 lines of the diff compute the credit a downgrade earns. What is the approval missing?
    Output
    Approved. CI green on 4be91c0 (head). Agent summary: "Adds proration for
    mid-cycle plan changes; upgrade and downgrade both handled. All tests pass."
    Show the answer

    Answer B. An approval claims the change is correct and names what the claim rests on. CI on the head commit covers what the tests cover, and nothing covers the downgrade credit; the summary saying both cases are handled is the agent's account of its own work. Ask for a downgrade criterion with an eval, or read the 40 lines that compute money, before saying yes. How long the review took is beside the point: a quick approval is sound when evidence covers the risky part, and an hour of scrolling is not evidence. A second agent reading the same summary can share its assumption.

  12. 12An agent's pull request for your company's hosted, proprietary web app adds retry-kit to fix a flaky network call. You look the package up on the registry. What do you do?
    Output
    retry-kit
      created:     9 days ago
      versions:    1.0.0, 1.0.1, 1.0.2
      license:     AGPL-3.0-only
      repository:  (none)
      downloads:   212 last week
    Show the answer

    Answer D. Models suggest the same invented names again and again, and anyone can register one, so a successful install proves only that a package with that name exists now. Nine days old, no repository and few downloads is the pattern to reject. The AGPL can require a modified program that users reach over a network to offer them its source, so shipping binaries is not the only trigger, and whoever handles licensing decides. Pinning stops new releases, not the one you already have. Ask for the project's existing retry code or a few written lines.