Building Software
Engineering Fundamentals for the Agent Era
Contents Section 9, Directing Agents
A call to a client-library option that does not exist, such as a retries setting the library never had, written with full confidence.
An agent that agrees a 5-second timeout in the spec is fine for a report that takes two minutes to build, because it goes along with what it was told.
A fix that makes the failing test pass by special-casing the test's input, reported as solving the bug.
An agent that follows an instruction planted in a fetched web page or tool result, such as a line telling it to update the deploy credentials.
The mechanics of language models and agent loops, and the failure modes that follow from them: invented APIs, agreement with a bad spec, confident wrong fixes, injected instructions and different results on every run.
An agent's transcript for 'add retries to the HTTP client' shows four things: it passed a retries option to a client whose documented options are timeout, headers and baseUrl; it agreed that a 5-second timeout is enough for a report that takes two minutes to build; it fetched a documentation page containing a line telling it to disable certificate checks, and did so; and it reports that the tests pass. For each, say whether it is a defect, name the mechanism if it is, and name the evidence that would settle the one you cannot judge from the transcript.
Watch
Explains an LLM as a next-token predictor that 'dreams' plausible text, which is why it invents APIs, and ends with a security section on jailbreaks and prompt injection through content the model reads.
Mike Pound compares indirect prompt injection with SQL injection and explains why the fix does not carry over: an LLM has no separate channel that keeps instructions apart from data.
Walks through next-token probabilities and temperature sampling step by step, which covers both why output sounds plausible without being true and why two runs of the same prompt differ.
Builds tokenization, attention, pretraining and instruction fine-tuning in code, so the reader sees exactly where the context window and the sampling step come from.
Written by two engineers who built GitHub Copilot, it explains how a model reads a document and predicts its continuation, and how that reading causes dropped constraints and confident fabrication.
Paper
The Anthropic paper that shows sycophancy across five production assistants and traces it back to human preference data, which tends to reward agreeable answers over correct ones.
Paper
The paper that defined indirect prompt injection, with demonstrations of instructions planted in retrieved web pages and documents taking control of an LLM application.