An agent is a bad place to hide an unclear process

A team asks for an agent to review unusual discounts. The request sounds clear until someone asks what “unusual” means. Sales points to the customer relationship, finance points to the margin, and the manager says that exceptions depend on context (which is true, but doesn’t tell the software what to do).

None of this means an agent is the wrong tool. It does mean, however, that the team is asking a model to settle an operating question that the business hasn’t settled itself. If we give the agent access to the CRM, the order system, policy documents, and email at this point, the demo may still look good because a language model is excellent at choosing a plausible interpretation. Production is where one plausible interpretation collides with another.

We’ve delegated the disagreement, not resolved it.

This discount example is illustrative rather than a Notus client case, but the design problem is common (the details will differ, while the ambiguity is familiar). Before choosing an agent framework, we’d want to know which part of the route is genuinely different each time and which part only feels open-ended because nobody has written the rule down.

What are we actually delegating?

The word “agent” now covers almost any software that calls a model, so it helps to be more precise. Anthropic distinguishes workflows from agents by asking who chooses the next step: a workflow follows paths encoded in software, while an agent uses the model to decide what to do next and which tool to use.

That freedom is valuable when the path really does depend on what the system discovers. A coding agent may inspect an unfamiliar repository, choose the relevant files, make an edit, run the tests, and then change direction because of the result. We couldn’t sensibly prescribe every step before the task began. You can feel the difference when the first tool result changes what you need to do next. A purchase request that always needs the same three checks is different, even if one of those checks involves language.

The route is open in the first case and merely undocumented in the second.

In the discount example, one case may require old account notes while another requires a pricing agreement, so letting a model choose which read-only source to inspect could be useful. The authority to approve the discount shouldn’t change from case to case. Neither should the fields that may be written, the evidence that must be retained, or the state that counts as complete.

Something useful happens once those distinctions are visible: the proposed agent often becomes smaller. We may therefore end up with a short workflow in which code loads approved customer fields and calculates policy thresholds, a model summarizes the unstructured notes, and a manager approves any exception before a narrow tool writes it. There is still AI in the system, but the model isn’t inventing policy as it goes.

Write down “done” before building the loop

An agent reports that the discount was approved, but we still need to know whether the order actually changed, whether the approval was recorded, and whether the notification went to the right salesperson. A confident final message can disagree with the database, just as an agent can say it stopped after a tool has already sent an email (the tool call, unlike the sentence, has consequences).

This is why we’d write a few completion checks before adding autonomy. You don’t need many at first. For one normal discount, one case just above the manager threshold, one request with missing notes, and one repeated request, we would record what must be true outside the conversation: the policy check has a result, the required evidence is present, any approval is stored, the final order state is correct, and a retry hasn’t created a second write. None of these checks depends on how confidently the agent describes its work.

That last detail matters more than it may seem.

The cases don’t need to become an elaborate benchmark on day one. Their first job, moreover, is to make the process owner and the technical team agree on what happened. Anthropic’s guide to agent evals makes the same useful distinction between grading a transcript and checking the outcome in the environment. We need both, but the changed record is harder to bluff (and easier for you to inspect after something goes wrong).

There is another benefit (and perhaps the more valuable one). If nobody can decide whether a borderline discount should pass, the eval has found a policy question before the software turns it into inconsistent customer treatment. That isn’t a failed technical exercise. It’s discovery doing its job (before the mistake becomes much more expensive).

The prompt is not a permission system

The risk changes, nevertheless, once a model can take action. We can tell it never to alter an order without approval, but the downstream service should reject that write anyway. A persuasive prompt isn’t an authorization boundary, particularly when the agent is reading documents or messages that may contain instructions of their own.

In other words, the model can propose an action; it can’t grant itself permission (and a document it reads certainly can’t grant that permission either).

OWASP describes excessive agency in terms of excessive functionality, permissions, and autonomy. In the discount workflow, the agent doesn’t need arbitrary database access. It needs a read-only view of specific customer fields, a specific approval request, and a write operation that accepts only the fields this process may change. Those controls belong in code, behind the model.

Repeated calls deserve the same attention. If the agent calls a tool twice after a timeout, an idempotency key should turn that into one business action, not two updates or two emails. The loop also needs a step limit, a budget, an audit trail, and a route to a person when the state is unfamiliar. “Human in the loop” only becomes a safeguard when a named person knows what to review and has the authority to stop it, a point also reflected in NIST’s guidance on human and AI roles.

When an agent has earned its place

After this work, we may still choose an agent. Perhaps the strongest cases have a route that can’t be known in advance, an environment that returns useful evidence, a result that someone or something can check, and a failure whose cost is bounded. Coding and open-ended research often have this shape. Many fixed approval chains don’t, and knowing which situation you’re in before choosing the architecture will save a great deal of avoidable complexity.

The surprising part is that clarifying the process doesn’t make an agent less capable. It gives the agent a smaller area in which judgment is genuinely useful, while the business keeps control of the rules, permissions, and final state. That is usually the difference between a loop that looks intelligent and a system you can inspect, understand, and responsibly allow to act.

However, if the path remains genuinely open after all of those boundaries are clear, the agent now has a much better job to do.

Moreover, the team can explain why that freedom exists instead of treating autonomy as the default.