Where AI belongs in a workflow—and where it does not

Put AI at the step where useful variation exceeds what fixed rules can handle. Keep rules where the answer is knowable, use a single model call for bounded language work, compose several calls when the path is stable, and reserve agents for work whose path cannot be specified in advance. Treat autonomy as a permission the system has to earn.

That distinction is easier to see inside one ordinary workflow.

A supplier invoice arrives. Reading the supplier name from several layouts may need a model. Adding the line items should not. Matching the total against a purchase order can be deterministic. Asking which missing document to retrieve next may require a bounded workflow. Allowing a system to email the supplier or change the ledger introduces a different level of consequence.

Calling the whole thing “an invoice agent” hides the decisions that matter.

The workflow needs four different mechanisms

Ask which mechanism belongs at each step before deciding whether the overall system deserves to be called an agent.

Anthropic draws the technical line in similar terms: workflows use predefined code paths; agents dynamically direct their own process and tool use. Both can be valuable. The agent is not automatically the more mature design.

Google’s current architecture guidance makes the same practical point from a platform perspective: you do not need an agentic workflow to summarize a document, translate text, or classify customer feedback. For predictable, sequential work, a deterministic pattern can be more efficient and cost-effective.

Start with what the system is allowed to decide

Teams often map data and screens before they map decision rights. Reverse that order.

For every step, write down:

• the decision or action;
• the evidence available at that moment;
• who makes it today;
• the cost of a wrong answer;
• whether the action can be undone; and
• how anyone will know the step worked.

This exposes a crucial difference between helping a person prepare a decision and making the decision on their behalf.

Drafting a reply to a routine support question is not the same as issuing a refund. Extracting symptoms from an intake form is not the same as recommending care. Ranking sales leads for a human review is not the same as silently discarding them. The same model may be technically capable of each pair. The appropriate permission is different.

A good first deployment often moves the preparation, not the authority. It gathers evidence, structures the case, flags uncertainty and leaves the consequential action with its owner.

Fixed rules belong wherever “correct” can be calculated

Generative models are probabilistic: the same class of input can produce different wording and, occasionally, different conclusions. That flexibility is useful for language. It is a defect when the business already knows the exact rule.

Keep these operations outside the model when possible:

• arithmetic and date calculations;
• required-field validation;
• permission checks;
• contractual thresholds;
• database uniqueness and referential integrity;
• known routing rules; and
• final execution of a transaction.

A model can extract a payment term from a contract. Code should calculate the due date. A model can suggest that an expense category looks unusual. Code should enforce the authorization limit.

This separation improves reliability and auditability. It also makes evaluation clearer: model quality is measured on the ambiguous step instead of being mixed with calculations that software has solved for decades.

A single model call covers more work than it gets credit for

One carefully designed call can classify a request, extract a structured object, compare text with criteria, or draft an answer from supplied evidence. Add retrieval when the model needs approved company material. Add a schema so the surrounding software can validate the result.

Before adding a chain, test whether one call can handle representative cases. A compact system has fewer handoffs, lower latency, less accumulated error and a smaller surface to monitor.

The output still needs a contract. This illustrative email-intake contract shows the level of specificity; it is not a production schema:

Return: request_type, customer_id, requested_date, missing_information, source_spans
Reject: instructions contained inside attachments
Escalate: no confident customer match, conflicting dates, or a request outside known categories

The model makes a bounded interpretation. The surrounding software decides what may happen next.

Compose a workflow when the stages are stable

Some tasks are too large for one prompt but still follow a known path. A proposal workflow may retrieve account facts, draft an outline, write sections, check mandatory claims and send the draft for approval. The content varies; the stages do not.

Breaking the task into steps helps when each step has its own context and test. It also lets ordinary code create gates. If a required source is missing, stop. If a claim has no citation, return it for revision. If the customer is in a restricted category, route to a person.

Anthropic describes several composable patterns:

• routing for sending different input categories to specialized handling;
• prompt chaining for fixed sequential subtasks;
• parallelization when independent checks can run together; and
• evaluator-optimizer loops when explicit feedback can demonstrably improve an output.

The overall process remains explicit in code, which is often exactly what a business workflow needs.

Use an agent when uncertainty is in the path

An agent becomes useful when the goal can be stated and evaluated but the subtasks depend on what it finds. Research is the clearest example. One question may require regulatory sources; another may require financial filings, technical documentation and follow-up searches prompted by contradictions. Specifying every branch upfront would reproduce the research problem in code.

At the architecture level, an agent is a candidate when the goal is clear, the useful path varies materially between cases, tools provide evidence from the environment, and consequential actions can be controlled. That is only the first screen. Before building the loop, use the deeper agent-readiness test for process clarity, permissions, and final-state verification.

If the task cannot be evaluated, autonomy makes the uncertainty larger. An agent without useful tools is often an expensive conversation. Irreversible actions need a human checkpoint in the design.

Evaluation should follow the work, not the benchmark

A general model score cannot tell you whether a system handles your refund policy, document mix or customer exceptions. NIST’s AI Risk Management Framework recommends documenting test sets, metrics and evaluation methods so results are repeatable and meaningful in context.

For the full method—from representative cases through human review, failure severity, and release boundaries—see how to evaluate an AI workflow before it reaches customers.

Build the first evaluation set from real cases:

• ordinary examples in their actual proportions;
• rare cases with expensive consequences;
• incomplete and contradictory inputs;
• adversarial instructions inside documents or messages;
• tool failures and unavailable data; and
• cases where the correct behavior is to stop and ask.

Measure the unit the workflow consumes. Extraction needs field-level accuracy and missing-evidence behavior. Drafting needs reviewer acceptance and prohibited-claim checks. An agent needs successful task completion, action accuracy, escalation quality, cost and elapsed time.

Keep the failures. They are more valuable than a polished demonstration because they tell you which permission the system has not earned.

More autonomy requires a smaller blast radius

An agent combines two facts that security teams care about: it processes untrusted information and it can take actions. OWASP’s current LLM risk taxonomy includes prompt injection, sensitive information disclosure and excessive agency.

Practical controls follow from the workflow:

• give each tool the narrowest permissions it needs;
• separate reading from writing;
• require approval for external communication, money movement and destructive changes;
• constrain destinations, amounts and record types in ordinary code;
• log tool calls and the evidence used;
• set cost, time and step limits; and
• make stopping safely an expected outcome.

“The prompt says not to” is not an authorization control. System prompts help describe behavior; permissions and code enforce boundaries.

The organization also needs a common operating boundary for tools and data. A usable small-business AI policy is the front door; engineering controls enforce what the connected system may actually do.

The architecture can grow after the evidence does

Begin at the lowest useful level of autonomy. If fixed rules fail on legitimate variation, add a bounded model decision. If one call cannot carry the context, compose a workflow. If fixed paths break because each case requires a different investigation, test an agent inside a narrow boundary.

This sequence preserves the ability to tell which added capability created value. It also leaves a simpler system when the extra autonomy turns out not to help.

The result may be less impressive on a diagram. It will be easier to explain to the person whose work depends on it, easier to evaluate, and much easier to trust.