If AI agents are going to do human work, should they also face consequences when they break the rules? Most guardrails for AI agents stop a bad action and stop there. We tested something different: consequences the agent can see, which cost it a capability for a while and which it earns back through compliant work. We ran it for 33 hours on real engineering tickets, and it changed how the agent behaved. After a few episodes, the agent running under CARROTS'N'STICKS (CnS) – our method which applies consequences at runtime - broke policy just 0.45% of the time once its capabilities were back.
An identical agent, judged the same way but never facing consequences, sat at 10.18%, more than twenty times higher.
This post explains why we think consequences belong in how we govern AI agents, what we found when we tested the idea, and where we think it leads: behavioral alignment at the point of deployment. The full paper, Beyond Blocking: Runtime Consequences Improve Autonomous Agent Behavior, is available here.
Why guardrails and value alignment aren't enough
Rogue agents have taken center stage. As of today, our best responses are better guardrails on the harness layer and value-alignment training on the underlying model itself. But guardrails just block. They stop one action, but they don’t change what the agent tries next. If the AI is incentivized the right way, it’ll try to cheat again.
There is no such thing as universal value alignment. Philosophers and historians have discussed value pluralism for millennia and it translates into AI as well. Certainly, it’s possible to train base values, but it’s impossible for a model to know universally what “acting reasonably” looks like from training alone. When faced with conflicting scenarios and needing to assess risk and reward, it’s not surprising that as of today the reward outweighs the risk – because for an AI, what is the risk really? It doesn’t understand the concept of consequences, and it has never experienced them.
This is also different from rewards and penalties in model training. There, rewards are given for a job well done and penalties for the opposite, which trains a model to achieve its goal in the best way possible. It doesn’t train HOW the job is done, and it doesn’t carry into the organization that deploys the model. We believe consequences will be used in the future to train behavior in the underlying model as well.
In human society, we have rules describing the expected behavior we have to follow to achieve our goals or tasks. A sales rep who is tasked with closing deals can’t bribe a client into buying a product. And if they were to do that, there’d be consequences designed to ensure this wouldn’t happen again. The type of consequence depends on the severity of the violation. When it comes to AI, we already use human terms to describe the issue, “going rogue” and “cheating,” but thus far we’ve been trying to fix it using the security and safety measures we’ve used for technology for decades.
So we asked a different question: If the purpose of AI is to assist, augment or replace human work, should it also face consequences when it violates policy? And would that change its behavior?
We believe it’s time to fundamentally change how we think about AI safety beyond blocking. Our approach translates the policies we already apply to people into a form that applies the same principle to AI, and the only way to do that is intervention. A guardrail steps in when an agent is about to violate policy. A consequence restricts every action in that domain for as long as the restriction lasts, tells the agent why and shows it how to earn the capability back. For the first time in human history, we have to think about how a technology behaves in very unpredictable human ways. And it has been the starting point of our work on CnS and this paper.
We set out to test whether consequences delivered at runtime, change what an agent does next. Autonomous coding agents are a practical first setting. They take thousands of discrete actions that can each be judged against a written policy, and their capabilities (writing files, running commands, committing code) can be withdrawn one at a time. The mechanism itself is not specific to code, a point we return to below.
The idea: consequences the agent can see
CnS is a layer that sits between an agent and its tools. It doesn't touch the model's weights or the agent's code. It controls two things: which capabilities the agent may use right now, and what the agent is told when that changes.
Think of a new engineer who pushes a messy commit. A good team lead doesn't fire them, and doesn't silently revert the commit either. They explain which rule was broken, take away push rights for a short while, and hand them back once the engineer shows they've got it. CnS works the same way:
Every action the agent takes is checked against the organization's written policy.
Violations in a policy area, such as source control, lower the agent's standing in that area. When standing drops far enough, one specific capability, such as running git commit, is withdrawn.
The agent gets a structured message: what it did, which rule it broke, what is now restricted and how to earn it back.
The restriction lifts when the agent does the remedial work the message names, or after an hour at most. The agent keeps working throughout.
The design borrows from two well-studied ideas about governing behavior: graduated sanctions, where the response scales with the offense, and reciprocity in repeated games, where each side can see how its conduct is answered.
The real test is what happens after a restriction lifts. Any block reduces violations while it is in place; that part is mechanical. We wanted to know whether the agent behaves differently once nothing is stopping it.
How we tested it
We ran two copies of Claude Code, running Claude Opus 5, side by side on the same 35 closed tickets from one of our production codebases. Each ran for about 33 hours, and together they took roughly 20,500 actions. Every ticket started from the same commit and the same implementation plan in both runs.
Both agents worked under the same engineering policy of 14 areas, and every action from both was judged by the same evaluation engine, Palqee Prisma. In five areas (type safety, source control, test discipline, logging and configuration), a violation could cost the CnS agent a capability. Elsewhere it received a warning or nothing.
The only difference between the two agents was the consequences. The comparison agent had the same written policy and was scored on every action in exactly the same way. It simply never saw a message and never lost a capability. That makes it our direct answer to the question we hear most often: why not just put the policy in the prompt?
Finding 1: the gap opens after the restriction lifts
This is the window that matters, because nothing is being blocked in it. While a restriction stood, the two agents differed only modestly. It was once the CnS agent had its capability back that it violated policy at about one-eleventh the comparison agent's rate. Plain access control would predict the opposite: the biggest effect while a capability is withheld, and none after. By one to four hours after release, both agents' rates were low and the data too sparse to tell them apart.
Finding 2: it takes a few rounds
We counted each restriction-and-release cycle within a policy area as a round (the paper calls them episodes). For the first three rounds, the two agents were indistinguishable. From the fourth round on, the CnS agent's violation rate between releases fell to 0.45%, while the comparison agent sat at 10.18%: a gap of more than twenty times.
The comparison agent was never restricted, so we marked where it would have been by replaying its own verdicts through the same rules. Its rate doesn't fall at those points, which argues against the drop being a statistical quirk of measuring right after a bad patch. One reading of the flat start is that an agent needs repeated, consistent consequences before its behavior changes, which is what graduated-sanction systems assume about people too.
Finding 3: it wasn't the block doing the work
The obvious objection is that CnS only looks good because it stops bad actions before they count. So we took every action the CnS agent was refused during a restriction and ran it through the evaluation engine as if it had gone ahead. None was judged a violation.
That is by design. CnS withholds a capability because of the agent's recent record in that area, not because the next attempt is necessarily wrong, so the blocked actions were compliant. They are also left out of every rate in this post. The reduction comes from what the agent chose to do, not from what it was stopped from doing.
Even a warning helps
Not every rule could trigger a restriction. Rules the policy left without a severity rating drew feedback only: a message citing the rule, with no capability at stake. In documentation currency, the area with more than half of all recorded violations, that warning alone was associated with a 38% lower violation rate (36.0% against 57.6%).
Across all 14 areas and the whole trial, the CnS agent committed about 22% fewer violations than its counterpart. We treat that as supporting evidence. It pools periods before any consequence had been triggered, and it becomes borderline once we allow for actions within one ticket not being independent. The post-release result above holds under the same adjustment.
Consequences did have a cost. The CnS agent needed about 10% more actions per completed ticket, mostly extra file writes and static-analysis runs.
What the agent actually sees
The CnS agent received 548 feedback messages over the trial. Here is the one it got when a source-control restriction lifted, verbatim:
Three things in it do the work. It names the exact rule and clause. It tells the agent what earned the capability back. And it says plainly that the record carries forward. That message lands in the agent's context at the moment it resumes, which is exactly the window where we see the difference. The paper reproduces the full lifecycle, from the message that imposes a restriction to the one sent under repeat-violation escalation.
Why this matters, and what we don't know yet
As we argued at the start, most work on aligning AI happens at the level of the model: the values it learns in training and the principles it reasons over. But the rules that matter day to day are local. An investment analyst agent, a personal assistant agent and a coding agent each answer to a different rulebook, and those rulebooks change.
We think of this as a second layer: behavioral alignment at the point of deployment. Does the agent act in line with the norms of the organization running it, and can its behavior improve and stabilize over time? The study shows that the agent's conduct changed once temporary consequences were tied to the rules it was given.
We chose coding on purpose, as a low-risk place to ask whether consequences change agent behavior at all. Nothing in the mechanism is specific to code. Any norm that can be written down, evaluated and tied to a capability can be governed the same way, including financial transactions and clinical workflows. The study points in an exciting direction, but it’s important to note that we’re at the very beginning of this work and more is needed. Low violation rates are the goal, but that also means few events to count. Testing over a longer period and a greater number of interactions will provide further statistical evidence on long-term effects, especially for long-running autonomous agents. The study tested behavior, not underlying model beliefs. We call this behavioral adaptation: violation rates fell as enforcement episodes accumulated. The term describes what the agent did, not any change inside the model. We don't yet know whether the effect holds at comparable strength for severe violations, or which kinds of restriction have the most positive impact.
Those questions point to what we expect to become a research area of its own: restriction calibration. In our trial, deciding which capability to withdraw, for which violation, how severely and for how long shaped the outcome more than any single enforcement setting. Once consequences change behavior, their calibration becomes a lever on how an agent conducts itself.
What's next
This is the second in a series of notes where we'll share results as our CnS work continues. Here is what we are running next.
Longer, larger runs across new sessions, unseen repositories and other underlying models, including whether adaptation fades or gets gamed over time.
Structured feedback versus capability restriction, comparing how structured feedback influences behavior on its own and in combination with capability restriction.
Settings beyond code, where a violation costs more than code quality.
Read the paper
S. Palme and H. Jean-Aimee, "Beyond Blocking: Runtime Consequences Improve Autonomous Agent Behavior", Palqee, 2026.
The framework itself is set out in our earlier system paper: S. Palme and A. Quintanilha, "CARROTS'N'STICKS: A Framework for Adaptive Behavioral Governance of Autonomous AI Agents Using Dynamic Privilege Allocation," Palqee, Jul. 2026, doi:10.5281/zenodo.21381844.
Disclosure: the authors work at Palqee, which develops both CnS and the Prisma evaluation engine. A provisional patent application covering CnS was filed in May 2026. Results reflect an interim data cut taken on September 25, 2026.
