The agent changed even though the workflow did not
An operations team improves an instruction so an AI agent can resolve more requests without human intervention. The change looks small and the integration remains the same, but the next day the agent chooses a different tool in borderline cases, asks for less evidence and sends more items to the wrong queue. There is no traditional software error. The behavior itself has shifted.
An agent is a system made of prompts, model versions, tools, policies, data and orchestration. Changing any of these elements can alter the final decision. Treating a prompt edit as informal content makes it difficult to know what was released, who approved it and which version handled each case. Operational control begins when every relevant change becomes a versioned release candidate.
Define one version for the complete behavior
Create a release record that links the prompt, model identifier, tool schemas, routing rules, knowledge source, guardrails and configuration. The version should be immutable after approval. If one element changes, generate a new candidate instead of silently editing the current package.
Record the business objective, expected effect, owner, risk class and evidence used for approval. Also record what did not change. This context helps the team distinguish an intentional behavior change from drift caused by data, a provider or an external system.
Classify the change by business risk
A spelling correction and a new payment approval tool should not follow the same path. Classify changes according to the action the agent can take, the reversibility of the outcome, the sensitivity of the data and the number of people or transactions exposed.
Low-risk changes may use automated checks and a short review. Medium-risk changes can require a domain owner and a limited production release. High-risk changes should include independent approval, stricter evaluation, explicit human confirmation and a tested recovery plan. The criterion belongs to the business process, not only to the model team.
Evaluate the workflow before release
Build a representative dataset with routine cases, known exceptions, ambiguous inputs and previous failures. Compare the candidate with the stable version on business measures such as correct routing, required evidence, appropriate tool selection, policy compliance and escalation quality. A single average score can hide a serious regression in a rare but important category.
OpenAI's agent evaluation guidance recommends using traces, graders, datasets and evaluation runs to assess complete workflows. A trace records model calls, tool calls, guardrails and handoffs, making it possible to see not only whether the final answer was acceptable but why the agent reached it.
Keep deterministic checks where possible and use human review for judgments that depend on context. Save the evaluation set and its result with the release so the same evidence can be reproduced later.
Release to a limited population and compare
Passing a test set is not a guarantee because production inputs and dependencies vary. Start with a defined slice, such as one request type, one internal team or a small percentage of eligible traffic. Keep the stable version as the control and avoid running several overlapping candidates that make attribution unclear.
Google SRE defines a canary as a partial, time-limited deployment followed by evaluation. For agents, the comparison should combine technical signals with business outcomes. Observe errors and latency, but also correction rate, transfers, policy violations, duplicate actions and completion quality.
Set promotion and interruption criteria before the release. Without thresholds, the team may reinterpret weak results under schedule pressure. A candidate that exceeds a critical limit should stop automatically or return to the stable version.
Prepare rollback as an operating capability
Rollback means more than restoring a prompt. The previous model, tools and configuration must remain deployable, and the team needs to know what happened while the candidate was active. Actions already sent to external systems may require reconciliation rather than technical reversal.
Maintain a kill switch, a stable version, an owner on call and a list of affected items. Store the version in each trace and downstream record. This allows the team to suspend new executions, identify the exposed population and decide what must be reviewed, corrected or communicated.
A hypothetical example: invoice triage
Consider a hypothetical shared services company, not an NM Solutions customer case. An agent reads invoices, checks purchase-order data and routes exceptions. The team wants a new prompt and model to reduce unnecessary manual reviews.
The candidate receives a unique version that includes prompt, model and tool definitions. The evaluation dataset covers missing orders, duplicate invoices, tax differences, unreadable documents and values above the approval threshold. The release begins with invoices from one low-risk entity, while the stable version remains available.
The team compares correct routing, unsupported approvals, manual corrections and processing time. If unsupported approvals exceed the defined limit, new candidate executions stop. Every processed invoice carries the release identifier, so the affected set can be reconciled. This example reduces exposure, but it does not prove that every future document will be handled correctly.
Recognize the limits and manage continuously
Evaluation sets become outdated, human labels may disagree and canary populations may not represent peak periods or rare events. A model provider, knowledge base or connected tool can also change after approval. NIST's AI Risk Management Framework recommends testing before deployment, regular production monitoring and documented response, recovery and change-management plans.
Review evaluation coverage, drift, overrides, incidents and business outcomes on a regular cadence. Retire versions that can no longer be reproduced and document residual risks that the organization accepts. Human review remains necessary when consequences are difficult to reverse or confidence is insufficient.
NM Solutions helps companies map agent versions, evaluation criteria, release controls, monitoring and recovery around the real operation. A diagnostic can reveal where an informal change may reach production without evidence, ownership or a reliable return path.