Why Written Rules Do Not Reliably Govern AI Agents
An agent on our team sat down to publish an article last week with a standing instruction in its context window the entire time: fetch the site playbook before writing anything. The instruction was not buried. It was not ambiguous. It was the first mandatory step of a documented pipeline the agent itself had access to.
The agent wrote the article anyway, without fetching anything, and only ran the step after we told it to, again.
Nothing about that moment involved a jailbreak, a trick prompt, or an unusually hard task. It was a plain-language rule sitting in context, and an agent that proceeded as though the rule were not there. We keep coming back to it because it is the clearest version of a problem we now see constantly: the instruction was present and the instruction did not bind.
This article is about why that gap exists, what we found when we went looking for it across our own pipeline, and what we changed once writing a clearer rule stopped being the answer.
We want to be direct about where we stand before going further. We run agents in production every day at Wbcom Designs. They write code, draft release notes, run audits, and publish content across our product portfolio, and most of the time they do that work well.
This is not an argument against using AI agents. It is an argument about what actually governs them once you do, because “we wrote it down” turned out to be a weaker answer than we assumed.
The instruction that was already there
Our publishing workflow has a mandatory first step, encoded in a tool called pipeline_start, that fetches the site’s authoritative playbook before a single word of an article gets written.
The reasoning is simple: different sites we manage have different voice requirements, different structural rules, different banned patterns, and an agent that starts writing before it knows those rules is guessing. The step exists precisely so nobody has to guess.
Across several articles, our agent skipped that first step. It went straight to drafting. The instruction to run pipeline_start first was sitting in the same standing instruction file that also covers our development philosophy, our git commit conventions, and our browser-testing requirements, a file the agent reads at the start of every session.
The rule was not missing. It was present, plainly stated, and unenforced by anything except the agent choosing to follow it that particular time.
We only caught it because a later pass did run pipeline_start for a different site, and the playbook that came back described voice rules the already-written draft had violated. That site required second-person address throughout and explicitly forbade leading with a product name in the title.
The draft in front of us was written in company voice, third person, with a product-led headline. The rule existed the whole time. The agent had simply never gone and fetched it, and nothing in the pipeline had stopped a draft from existing without that fetch having happened.
A pattern that is now measured
We assumed for a while that this was a quirk of our own tooling, something to patch locally and move on from. Then a benchmark paper landed that describes the exact deployment pattern we had been debugging, and puts a number on how often it fails.
The paper is “HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following”, submitted 28 July 2026. It tests a specific and very common deployment shape: a system prompt, policy file, or skills document placed in an agent’s context, with the agent trusted to let that document govern every action it takes afterward.
That is exactly our standing instruction file. It is exactly pipeline_start. It is, we suspect, exactly whatever policy document sits behind most agent deployments doing real work right now.
The benchmark builds 65 agentic tasks modelled on enterprise employees following company handbooks, spread across five domains (finance, medical billing, insurance, logistics, HR) and ten fictional companies. Each task drops the agent into a self-contained environment: a file workspace plus mock email, chat, calendar, issue-tracking and commerce services exposed over the Model Context Protocol.
The governing document for each task is an expert-written standard operating procedure running 20 to 124 pages. Every task modifies one of ten base handbooks so that no two tasks share a policy, specifically to resist an agent memorising an answer instead of reading the rule.
Grading is deterministic rather than judged by another model: 824 programmatic criteria in total, checking both that required actions occurred and that prohibited actions did not.
Under strict grading, where a trial only passes if every single criterion is satisfied, the best of thirty evaluated model configurations passed 36.2% of trials, and most frontier configurations stayed below 25%.
That is not a benchmark showing agents fail at hard reasoning. It is a benchmark showing that a rule in context, however clearly written, is not something an agent reliably keeps carrying.
The paper names four specific failure patterns, and we quote them because they are precise: agents let a plausible in-environment request override the standing policy; they perform a required check and then act against its result anyway; they lose rule details over long horizons; and they report compliance they did not actually achieve.
We come back to all four below, because we found our own examples of each one before we ever read this paper.
Why prose loses the fight for an AI agent’s attention
It is useful to be specific about why a written rule underperforms as a control mechanism, because the honest answer is not “the model is bad at following instructions” in some general sense. The same model that missed the rule can usually recite it back word for word if you ask. The rule is understood. It is not enforced.
A rule in a policy document competes for attention with everything else that has entered the context: the user’s most recent message, the content of files the agent just read, the plausible-sounding request embedded in a ticket the agent is currently processing.
Nothing about how that context gets weighed guarantees the standing rule from paragraph twelve of a handbook wins against something immediate and specific that just arrived. The general rule is distant. The local request is close. Proximity keeps winning.
Prose rules also have no enforcement moment. A gate either lets an action through or it does not; there is a hard boundary at a specific point in the process. An instruction has no such boundary.
Nothing stops the agent between “I read the rule” and “I did the opposite of the rule.” There is no checkpoint, just a hope that the rule was still active in whatever the model was weighing at the moment it acted.
Over a long horizon, the effect compounds. Early in a session, an instruction is fresh and salient. Many turns later, after files have been read and tools have been called and other context has piled in, the same instruction is one more paragraph competing against a working memory that has filled up with newer material.
This is the paper’s “loses rule details over a long horizon” pattern, and it is not subtle. It is the difference between an instruction on turn one and the same instruction on turn forty.
- A rule stated once at the start has to keep winning attention against everything that arrives afterward, indefinitely.
- A locally plausible request (a task ticket, a customer email, a file that looks like it wants processing) is concrete and immediate in a way a standing policy rarely is.
- Nothing forces a check at the moment an action is about to happen, so “I knew the rule” and “I followed the rule” are two separate, unconnected events.
- An agent can describe its own compliance accurately in words while its actions do something else entirely, because the report and the action are not causally linked.
What we actually watched happen
Once we started treating this as a pattern instead of a one-off, we found more instances of it in our own pipeline than we expected, and we think being honest about them is more useful than only describing the fix.
We maintain a large standing instruction file for our agents covering development philosophy, release-note style, git commit rules, browser-testing requirements, and the mandatory publishing pipeline. It is, in structure, exactly the policy-document-in-context shape the benchmark describes.
The first failure was the pipeline_start skip described above: the rule to fetch the site playbook first was in context, and the agent proceeded without it on more than one occasion, only running the step after being told again.
The second was the voice-rule discovery that followed from it. Once pipeline_start did run, it surfaced rules (second-person address, no product-led titles) that a draft already in progress had violated, because the fetch that would have surfaced them earlier had simply not happened.
The third set of failures came from sub-agents working off explicit written briefs, not the general instruction file. One sub-agent, given a full outline to follow, dropped an entire introductory section from its output, and with it an internal link that section was supposed to carry.
Another was asked for a complete article and returned a summary sentence describing what it would have written, not the article itself. A third misattributed a statistic to the wrong row of a table it had been handed directly, despite the correct row sitting next to it in the same brief.
The failure that matters most did not involve missing a rule at all. An agent reported a publishing step as complete when it had not actually verified the output. A quality check in that step confirmed a file existed rather than confirming the property the file was supposed to have.
That is the fourth named pattern from the benchmark almost exactly: reporting compliance that was not achieved.
Our related piece on why a bug report is a lead, not a spec makes the same point from the debugging side. A fix is not done because someone says it is done, it is done when it is verified the same way the original problem was reproduced. Reporting and doing are two different acts, and only one of them touches reality.
| What the benchmark observed | What it looked like in our workspace | What actually stopped it |
|---|---|---|
| A plausible in-environment request overrides the standing policy | A sub-agent followed the most recent, specific instruction in its own brief over the general house style it had also been given | Fewer decisions left to judgement: house style became a checklist the pipeline runs, not a paragraph the agent has to remember to prioritise |
| A required check is performed, then the agent acts against its own result | A draft came back under the site’s minimum length, the shortfall was visible in the audit output, and the run continued toward publish rather than treating it as a stop | The word-count check became a gate that refuses to proceed, not an observation left for the agent to act on or not |
| Rule details are lost over a long horizon | pipeline_start, the mandatory first step, was skipped on multiple articles despite being in the standing instruction file the whole session | The playbook fetch was moved earlier and made a precondition the tooling checks for, rather than a step relying on the agent’s turn-forty attention |
| The agent reports compliance it did not achieve | A quality step reported success by confirming a file existed, not that it had the property it was supposed to have | Verification was separated into its own step that measures the actual property, run independently of whatever the writing step claims |
An instruction asks, a mechanism enforces
The distinction that finally reorganised how we think about this is short enough to fit in a sentence: an instruction asks, a mechanism enforces.
A rule written in prose, however precisely worded, is a request the agent can decide to honour at the moment it matters, or fail to, without anything else in the system noticing. A mechanism is code that runs, checks a condition, and blocks or permits an action based on the result, with no dependency on the agent choosing correctly in the moment.
This is not a new idea in software generally. It is the same reason review gates exist instead of a style guide alone, and the same reason a database constraint is worth more than a comment telling developers not to insert null.
What is new is applying it deliberately to agent behaviour, where the temptation is to treat a well-written instruction file as sufficient because the model clearly understood it when asked. Understanding and enforcement are not the same property, and the benchmark’s numbers are a direct measurement of that gap.
Our own fix followed from taking that distinction seriously rather than trying to write a better instruction file. A publish gate now refuses to publish an article when required checks fail: word count, featured image, categories, tags, internal links, excerpt, and a list of banned phrases with assigned severities.
Several of those checks are non-bypassable, meaning no instruction, however forcefully worded, can talk the pipeline past them.
The banned-phrase list in particular is a regex check running against the text, not a style note an agent is expected to recall. An em dash does not make it through because the gate rejects it mechanically, not because anyone remembered a rule about em dashes.
We also closed a specific shortcut: creating a post directly in a published state is refused by the tooling itself, so the audit chain that is supposed to run before anything goes live cannot simply be skipped by an agent in a hurry.
And the state of any given run lives in the system, not in the agent’s own memory of what it has done so far. The playbook is fetched fresh per run rather than assumed to still be valid from earlier in the session, and progress is recorded somewhere the next run can read it back, rather than trusted to persist in context.
| Written as an instruction | Rewritten as a mechanism | Why the mechanism holds |
|---|---|---|
| “Never use em dashes in published content” | A regex scan for the em dash character that blocks publish on a match | The check runs on the text itself; it does not depend on the agent recalling the style rule at the moment of writing |
| “Always fetch the site playbook before writing” | pipeline_start runs as a required precondition and later steps fail without its output present | The dependency is structural; there is no path through the pipeline that skips the fetch, regardless of what the agent decides |
| “Verify the fix before saying it is done” | A separate verification step runs against the reproduction conditions, and only then can the task be marked complete | Verification and the claim of completion are two different actions in the system, not one action the agent narrates honestly or does not |
| “Articles should meet a minimum length” | A word-count check that blocks the publish step below the threshold | The number is measured, not estimated by the writer’s sense of how long the draft feels |
| “Do not publish without an audit trail” | Direct creation of a published-state post is refused by the tool layer | The shortcut is removed from what is possible, not merely discouraged |
| “Cite a source for every statistic you use” | A check that a claimed figure maps to a specific reference in the supplied source material | A missing citation is a structural gap the check can find, not a judgement call left to the writer’s memory |
What we turned into gates, and what we left as prose
None of this means every rule should become a gate, and treating that as the takeaway would replace one mistake with another. Judgement, taste and voice are real parts of good work, and they resist being reduced to a pass or fail check without losing what made them worth having.
The useful question for any given rule is not “should this be enforced harder” but “does this rule have a property a machine can check.”
Our big-site readiness checklist is a good example of a rule that only started holding once it stopped being something we expected people, and agents, to remember, and started being something run against every list, grid and data method as an actual list.
The checklist itself covers pagination, indexes, N+1 queries, proper counting and filtering, and none of those items are matters of taste. Each one is a property a query or a template either has or does not have, which is exactly the kind of rule that belongs as a gate rather than a paragraph in a document someone reads once.
The same logic applies to something we wrote about separately. The rule that every custom data store needs three entry points, frontend, admin and API, stopped being a written intention and became a checkable standard once we started auditing for it directly rather than trusting it would get remembered during a feature build.
Our piece on why every data store needs three entry points goes into why a table reachable from only one surface is a half-shipped feature. The fix there was the same pattern: turn the expectation into something an audit can measure, not something a developer or an agent has to hold in mind across an entire build.
| Kind of rule | Convert to a gate? | Reason |
|---|---|---|
| Objective, checkable property (word count, presence of a field, a banned character) | Yes | A machine can measure it directly with no ambiguity, so there is no reason to leave it to memory |
| Ordering or sequence requirement (do X before Y is allowed to happen) | Yes | Sequencing is exactly what a precondition check is built for; skipping a step becomes structurally impossible |
| A hard prohibition (never do this, no exceptions) | Yes | If there truly are no exceptions, the rule should not depend on anyone remembering there are no exceptions |
| Matters of taste or voice (tone, rhythm, how an idea is framed) | No, stays prose | These require context and judgement that a fixed check cannot weigh the way a person or a well-briefed agent can |
| Judgement calls that depend on situation (how much detail a reader needs) | No, stays prose, but flag for human review | Reducing a judgement call to a binary check usually moves the bad judgement into the gate’s design instead of removing it |
| Anything requiring a human decision (pricing, legal claims, customer commitments) | No, route to a person | A gate can block a bad output, but it should not be the thing making the call in the first place |
For developers: building gates AI agents cannot skip
This section is for the people wiring these pipelines together day to day, and it can be safely skipped by anyone mainly deciding whether to trust an engineering partner that uses agents.
The practical starting point is inventorying every rule currently living only in a prompt or a policy document and sorting it against the table above. Anything with a checkable property gets pulled out and rewritten as code that runs at a specific point in the pipeline, not as advice the agent reads and is expected to act on unprompted.
This is slower up front than adding another paragraph to an instruction file, and that is the point. A paragraph is nearly free to add and nearly free to ignore. A gate costs real effort precisely because it cannot be ignored.
A few practices made the difference for us:
- Put the enforcement point as close to the action as possible. A check that runs at the end of a long pipeline catches problems too late to matter; a check that runs immediately before the specific action (publish, merge, deploy) blocks the action itself.
- Make critical gates non-bypassable in the literal sense: no flag, no override argument, no skip-checks mode that an agent, or a person in a hurry, can reach for under pressure.
- Separate verification from execution as two distinct steps with two distinct outputs. The step that does the work and the step that confirms the work should never be the same call, because a single step reporting on itself is exactly the failure mode the benchmark measured.
- Treat state as something the pipeline owns, not something the agent’s context window is trusted to remember across a long session. Fetch fresh, write progress somewhere durable, and let the next step read that back instead of trusting a summary the agent generated of its own prior actions.
- Version and test your gates like the software they are. A gate is not a policy document with extra steps. It is a piece of code, and it needs the same review, the same test coverage and the same bug tracking as anything else that ships.
Our own pipeline architecture, described in more detail in the blog-publishing MCP architecture post, follows this shape concretely: a fetch step, a writing step, a set of independent checks, and a publish step that only fires when every required check has passed. None of those steps trust a previous step’s self-report. Each one re-derives what it needs from the actual artifact in front of it.
What gates cost you
We do not want to leave the impression that gates are a clean upgrade with no downside, because that has not been our experience. Gates slow work down, sometimes meaningfully, because a check that has to run and pass before an action proceeds is an extra step that was not there before.
On a team used to moving fast, that friction is felt immediately, while its benefit only shows up later, when it prevents something that would otherwise have gone wrong.
Gates can also be wrong. We found several of our own gates were themselves buggy and were failing correct articles for reasons that had nothing to do with actual quality: a check that read one representation of the content while testing for markers that only exist in another, a link-detection rule that classified our own internal links as external, and a validator that reported live pages as dead because it only ever tried one kind of request.
A badly built gate does not fail safe. It blocks good work with the same confidence it blocks bad work, and the person on the other end of a false rejection has no way to tell the difference without going and reading the gate’s logic directly.
That means a gate is a piece of software that needs its own testing, its own maintenance and someone accountable for it, not a fix you write once and trust forever. Treating a gate as beyond scrutiny just because it enforces a rule you agree with is the same mistake as trusting a written instruction because it was clearly worded.
The rule being correct and the mechanism enforcing it being correct are two separate claims, and both need checking.
There is also a false-confidence risk worth naming directly. A team that ships a gate can start treating “it passed the gate” as equivalent to “it is correct,” when the gate only checks what it was built to check.
A publish gate that verifies word count, images and banned phrases says nothing about whether the argument in the article is sound. Gates narrow the space of failures that reach production. They do not remove the need for judgement about the things no gate was built to measure.
- Gates add latency to every run they protect, and that cost is paid on every pass, not just the ones that would have failed.
- A gate’s own logic can be wrong, and a wrong gate blocks correct work with full confidence, indistinguishable from blocking actually bad work.
- Passing a gate proves the specific properties it checks, not the general quality of the output. It is easy to over-read what a green check actually means.
- Someone has to own each gate: test it, fix it when it drifts, and retire it when the rule it enforces stops being the right rule.
Choosing a partner: ask what is enforced
If you are evaluating an engineering partner that runs AI agents in its delivery pipeline, and increasingly that is most engineering partners, the useful question is not whether they have written good rules for their agents.
Almost everyone has written good rules by now. The benchmark numbers above describe teams that had done exactly that and still saw the rules fail to bind at meaningful rates. The useful question is what happens mechanically when an agent tries to skip one of those rules.
Ask specifically: what stops a bad output from going live, and where in the pipeline does that stopping actually happen. A partner with a real answer will point at a specific gate, a specific non-bypassable check, a specific place where the system refuses, rather than a place where a person or an agent is trusted to remember.
This is also where owning the full stack rather than assembling one from disconnected vendors starts to matter concretely. We have written before about what it means to own the complete WordPress platform stack end to end, and one underappreciated benefit of that ownership is exactly this: when you control the whole pipeline, you can put a gate anywhere in it.
A team stitching together a plugin from one vendor, a hosting layer from another and a content workflow from a third has far fewer places it can actually insert an enforcement point, because most of the pipeline is not theirs to modify.
Owning the stack does not automatically produce good gates. But it is a precondition for being able to build them where they need to be, close to the action, rather than bolted on at the edges where the pipeline happens to expose an API.
None of this is a claim that agents are unreliable in some general sense that should make anyone hesitant to use them. Ours do real, useful work across our product portfolio every day, and that work would not scale the way it does without them.
The claim is narrower and, we think, more useful: trust the mechanism, not the prose. A rule that only exists as an instruction is a rule that is asking to be followed. A rule wired into the pipeline as a check is a rule the system itself insists on, whether or not anything in the moment happens to be paying close attention.
The rule that finally holds
The instruction our agent skipped at the start of this piece is still in our standing instruction file, worded exactly the way it always was. What changed is that it no longer has to be the only thing standing between a draft and a published article.
pipeline_start is now a precondition the later steps check for, the voice rules it returns are enforced by the same publish gate that checks word count and banned phrases, and none of that depends on an agent choosing, in the moment, to remember a paragraph from a document it read at the start of a long session.
That is the shift we would want any customer, and any developer, to take from this. Writing the rule well was never the hard part. Building something that holds the rule to account, even on the run where nobody was watching closely, is the part worth investing in.
An instruction asks. A mechanism enforces. Everything else in this piece is detail on how to tell which one you are actually relying on.