An agent is a bad place to hide an unclear process

If a team cannot agree who may approve a refund, what counts as a valid exception, or how completion will be checked, an AI agent makes that disagreement executable.

Use an agent when the route genuinely cannot be known in advance and the result can still be verified. For everything else, a rule, one model call, or an explicit workflow will usually be easier to test, cheaper to run, and safer to change.

If the first question is which mechanism belongs at each step, begin with the broader workflow architecture decision. This article starts after an agent appears plausible and asks whether the process is clear enough to support one.

“Agent” describes who chooses the next step

The word has become loose enough to mean almost any software that calls a language model. That is not useful when designing a production system.

Anthropic draws a clean distinction: a workflow orchestrates models and tools through code paths defined in advance; an agent lets the model dynamically direct its own steps and tool use.

Both can produce impressive demos. The operational difference is who chooses the next step: the software engineer who encoded the route or the model responding to what it finds.

That delegation is valuable when a coding system must inspect an unfamiliar repository, decide which files matter, edit them, and respond to test results. The next step depends on what the environment reveals. It is less valuable when a purchase request always needs the same checks in the same order.

Before choosing a framework, list the decisions that genuinely cannot be specified before the task begins. If the list is empty, there is no reason to delegate the route to a model.

An illustrative order exception

Consider an order that has been flagged because the requested discount is unusual. This is an illustration, not a Notus client case.

A vague process sounds like this:

Review the customer, decide whether the discount makes sense, update the order, and tell the sales representative.

It is tempting to give an agent access to the CRM, order system, policy documents, and email. The agent can investigate and take care of the whole thing.

But the sentence hides the important requirements:

• Which customer facts may influence the decision?
• Is the discount policy a hard rule or guidance?
• Who can approve an exception, and above what value?
• Can the system change the order before approval?
• What happens if customer data is missing or contradictory?
• Is success an email saying “done,” or an approved value stored in the order system?

An agent loop can route around this ambiguity during a demo. Each plausible answer looks like progress. In production, one customer gets an exception, another does not, and neither decision can be explained from an agreed policy.

The clearer design might be:

order flagged
  -> load approved customer and order fields
  -> calculate deterministic policy thresholds
  -> ask model to summarize unstructured account notes
  -> validate the summary against required evidence
  -> request manager approval when a threshold is crossed
  -> write the approved change with an idempotency key
  -> verify the final order state
  -> notify the owner

There is still AI in the system. It handles the step for which language is useful. It does not get to invent the policy or quietly redefine completion.

Make the process legible before the route becomes dynamic

An agent does not remove the need for a process. It changes which part of the process can be chosen at runtime. The fixed parts still need to be explicit: the starting state, available evidence, allowed tools, approval points, stopping conditions, and definition of completion.

Return to the order exception. The investigation route may vary because one case needs account notes while another needs a pricing agreement. The authority to approve a discount should not vary with the route. Nor should the fields that may be changed, the evidence that must be retained, or the final state that counts as complete.

This is where an apparently agentic task often becomes smaller. The model may choose which read-only source to inspect next, while code owns thresholds, approval, and the write. The open-ended part earns autonomy; the business rule does not.

Frameworks do not change that boundary. They may accelerate a prototype, but abstractions that hide prompts, tool responses, or control flow make a production failure harder to inspect.

Write the completion check before adding autonomy

An eval is a repeatable test that runs the AI system on a known task and grades what happened. Writing the first cases before implementation forces the process owner and technical team to agree on what “done” means.

For the illustrative discount process, do not begin with a broad score such as “decision quality.” Collect representative tasks:

• a discount within policy;
• an exception just above a manager threshold;
• missing account notes;
• contradictory notes and structured data;
• a repeated request after the order has already been updated;
• a malicious instruction inside an uploaded document.

Then check what matters outside the model:

{
  "policy_check": "pass | review | block",
  "evidence_complete": true,
  "approval_required": true,
  "approval_recorded": true,
  "final_order_state": "unchanged | approved_update",
  "duplicate_write_count": 0
}

The model’s final message is not the final state. An agent can say the discount was approved while the database was never updated, or say it stopped while a tool already sent the email.

Anthropic’s guide to agent evals makes this distinction explicit. It recommends grading both transcripts and outcomes, using the right mix of deterministic checks, model-based rubrics, and human judgment. It also advises reading traces rather than taking a score at face value. A failed eval can reveal a broken grader as easily as a broken agent.

The people who own the process should write or approve the cases. Otherwise the technical team will encode its own guess about good operations and call the resulting score objective.

For a wider release decision that also measures review effort, adoption, speed, and cost, use the end-to-end AI workflow evaluation method.

Permissions should come from the process, not the prompt

Once a model output can trigger an action, the prompt is no longer the main security boundary.

OWASP calls out excessive agency as excessive functionality, permissions, or autonomy. Its mitigations are concrete:

• expose only the tools required for the task;
• make each tool narrow instead of offering open-ended system access;
• run downstream actions in the user’s security context;
• enforce authorization outside the model;
• require human approval for high-impact actions; and
• log and rate-limit tool activity.

For the order example, the model does not need arbitrary database access. It needs a read-only tool for approved customer fields, a specific request-approval operation, and a write operation that accepts only the fields the workflow may change. The downstream service should reject an unapproved write even if the model argues persuasively.

Idempotency matters too. An idempotency key lets a repeated tool call produce one business action, not two refunds or two customer emails. Give loops a maximum number of steps and a budget. Route unknown states to a person rather than asking the agent to improvise indefinitely.

NIST’s guidance on human-AI interaction adds an important organizational point: human roles and responsibilities in AI decisions should be deliberately defined. “Human in the loop” is not a safeguard until a named person knows what to review and has the authority to stop the action.

Those permissions should sit behind the company’s operational AI policy and approval path, not depend on each team inventing its own rules.

Four conditions make autonomy defensible

An agent becomes a credible option when all four are present:

1. The path is open-ended. The next useful step depends on information discovered during the task.
2. The environment talks back. Search results, tool responses, tests, or state changes give the agent ground truth.
3. Success is checkable. A grader or person can distinguish completion from a confident claim.
4. Failure is bounded. Tools, permissions, approvals, budgets, and rollback limit the damage of a wrong step.

Coding agents often fit because tests provide feedback and verification. Open-ended research can fit when sources can be inspected and the output is reviewed. A fixed approval chain usually does not.

Before building the loop, write five representative tasks, the allowed actions, the approval points, and the final-state checks. That exercise often exposes a short explicit workflow. Use it when it does. Autonomy is justified only by the decisions that remain genuinely open after the process is clear.