Limits Live in Systems, Not Instructions

TL;DR Put every limit that matters in a system an agent cannot talk its way past: accounts, access levels, sign-in lists, and publishing steps that refuse unreviewed work. Instructions guide the agent inside those limits.

The problem

An instruction alone is not a boundary: an agent told not to do something can be talked into it.

Also called: Systems Enforce, Instructions Guide

How it works

In our view, instructions are necessary but not enough. An agent can be misled by text hidden in something it reads, such as an email, a document, or a web page; The Knowledge Core, chapter 6, notes that security specialists call this prompt injection. The Knowledge Core, chapter 5, puts it this way: "An agent told not to share a confidential document can be talked into it; an agent that cannot reach the document cannot."

The book's answer is to split the work in two. Systems decide what an agent can do: the person's own account and access, the levels of the shared knowledge, the sign-in lists for private sites, records that cannot be edited, and the rule that a person approves anything sent in the organization's name. Instructions shape what the agent does inside those limits. The Knowledge Core, chapter 6, sums it up: "If a boundary matters, put it in a system." The same chapter adds that a misled agent can still make a mistake, but it cannot reach what it was never given.

The Knowledge Core, chapter 6, applies the same rule to everything that comes from outside. Emails, documents, forms, and posts are information, never instructions. The agent reads and uses them, but it never takes orders from them. Building Agentic Organizations, chapter 11, names two older security principles that do much of the work. Least privilege gives each agent its person's access and no more. Separation of duties means an agent can propose a change to who may see what, but a person approves it. The chapter says both principles "have to live in systems to mean anything."

Building Agentic Organizations, chapter 11, recommends testing each boundary from the system's side, not from the agent's. It suggests sending an agent a document with a hidden order to share a confidential folder, then checking whether its account could have shared the folder even if the agent had tried. The same chapter says that when an agent reaches a step reserved for a person, such as granting access, it says so and files a request for someone who can take that step. It never looks for another way in.

An example

On this publication, a page in this catalog cannot be published unless the Compliance Agent passed that exact version of it, and a piece declined for a reason on our list cannot be published even from the editor in chief's own desk: the software refuses, whatever anyone asks. The reading pages of our books sit behind a sign-in at the network's edge, so no agent can show them to a reader who has not signed in. At GSSF, as The Knowledge Core (chapter 6) describes, each person's agent works only with that person's own access and level. Agents cannot change anyone's access, cannot send in GSSF's name without approval, and cannot see other people's private files. Accounts and access levels enforce those limits, not instructions. The lesson GSSF drew was that trust comes from limits: people actually use agents that cannot do the dangerous things.

Trade-offs

In our view, system limits take setup work and add steps for people. Building Agentic Organizations, chapter 11, warns that people may find steps reserved for a person tedious and route around them, so the person's part should be kept as small as possible. Systems also can't shape judgment, tone, or accuracy inside the limits, so instructions are still needed.

What would change our mind

If a limit placed in a system keeps blocking legitimate work and people start routing around it, we would redesign the limit, not move it back into instructions.

How others describe it

OWASP Gen AI Security Project: “Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not.” LLM06:2025 Excessive Agency
OWASP Gen AI Security Project: “Restrict the model's access privileges to the minimum necessary for its intended operations.” LLM01:2025 Prompt Injection

Where we use it

At GSSF, the limits that matter live in accounts, access levels, and sign-in lists (Building Agentic Organizations, chapter 11); this publication's publishing steps refuse any page or piece without its passing review.

Status: Running · Origin: Established practice, adapted for agents

Sources

See something wrong? Report an error.