Orchestration, Permissions and Guardrails

Mistakes to catch in review

A CI agent given the organization-wide deploy token because scoping one to the repository was more setup, so any repository it touches can ship to production.
An agent loop with no limit on steps, time or cost, running for hours.
An agent that can both write a change and approve its own merge.
An autonomous agent force-pushing, deleting data or sending messages without a person approving the action.

Running agents in loops and pipelines with limited permissions, budgets and approval points, so their mistakes stay small.

Topics

Agent Loops and Multi-Agent Orchestration: Planners, workers and reviewers, and when adding agents improves reliability and when it only adds noise.
Sandboxing and Least Privilege for Agents: Restricting file, network and credential access so an agent's worst possible action is survivable.
Separating Reading From Acting: Designing workflows so an agent that reads untrusted content cannot take consequential actions on its own.
Human Approval Points: Choosing which actions, such as deploys, deletions, payments and external messages, need a person, and making that approval meaningful.
Budgets, Timeouts and Kill Switches: Limits on steps, time and spend, and a reliable way to stop everything at once.
Audit Trails: Recording what each agent did, with which permissions and on whose instruction.

You understand it when you can

Design the minimum permission set for an agent that fixes failing tests.
Identify which actions in an agent workflow need human approval, and justify each one.
Explain how you would detect and stop an agent that is looping or overspending.

Drill

An agent workflow that triages new issues in a public issue tracker runs with a token that can push to the main branch, merge pull requests and read deployment secrets. Find the attack path from a malicious issue to a production change, and cut it with the smallest permission set that still lets the workflow do its job.

Start here

Reference

The lethal trifecta for AI agents: private data, untrusted content, and external communication

The original statement of the rule that an agent combining private data, untrusted input and a way to send data out can be made to leak, and that the fix is to remove one of the three, not to filter.

Watch

How We Build Effective Agents: Barry Zhang, Anthropic

Barry Zhang, 2025. 15-minute talk.

Says to reach for a fixed workflow before an autonomous agent and to keep agents simple, and covers when extra agents or steps actually improve reliability.

Black Hat USA 2025 | Reinventing Agentic AI Security With Architectural Controls

David Brauchler III, 2025. 31-minute talk.

Argues that prompt injection cannot be eliminated and that guardrails are not a first-line control, then shows architectural trust-boundary patterns that keep an agent reading untrusted data from escalating privileges.

Johann Rehberger - Your Agent Works for Me Now | [un]prompted 2026

Johann Rehberger, 2026. 26-minute talk.

Live exploit chains against coding and assistant agents, including delayed tool invocation, persistence and lateral movement, which show what an over-permissioned agent turns into once injected.

Read

Building Applications with AI Agents: Designing and Implementing Multiagent Systems

Michael Albada, 2025.

Covers how to structure single- and multi-agent systems (tools, planning, orchestration patterns) and the trade-offs of each design, including when more agents add noise instead of reliability.

Building Secure and Reliable Systems: Best Practices for Designing, Implementing, and Maintaining Systems

Heather Adkins, Betsy Beyer and 4 others, 2020. Free to read online.

Google's chapters on least privilege, multi-party authorization and audit logging apply directly to scoping an agent's token and requiring a second party for consequential actions.

Primary sources

Reference
Building Effective AI Agents
Anthropic's reference taxonomy of workflows versus agents (prompt chaining, routing, orchestrator-workers, evaluator-optimizer) and the guidance to add autonomy only when it pays for its risk.