Before giving an AI agent permission to act, ask a practical question: if this action is wrong, how would we know, and how would we fix it? The answer should influence how much authority the agent receives.

Some mistakes are annoying but easy to correct. Others create cost, customer harm or disclosure risk. A preference change can usually be reversed. A shipped order, canceled reservation, issued refund or disclosed personal information may not be so simple.

This article uses customer-service examples to show how correction and recovery planning can shape a permission plan. The examples are illustrative, not legal or compliance advice.

Start With the Mistake Scenario

For each action, imagine the agent did it incorrectly. It changed the wrong address. It canceled the wrong reservation. It sent account details to the wrong email address. It refunded too much money. It promised a replacement item that is out of stock.

Then ask what recovery would require. Can the action be undone automatically? Does a person need to intervene? Would the customer need to be notified? Would the business need to contact a carrier, payment processor, marketplace, hotel, bank or regulator? Would the customer still experience harm even after the record is corrected?

NIST’s AI Risk Management Framework encourages organizations to consider potential costs from AI errors and to define controls, measurement and monitoring activities in context. In permission planning, that means looking at the likely correction path before deciding the agent may act.

Reversible Actions Can Often Have More Routine Authority

A reversible action is one the business can correct quickly with limited customer impact. Examples include changing a notification preference, adding a note to a support ticket or switching a display setting.

Even reversible actions need limits. The agent should act on the correct account, use verified channels and record what changed. But if the action is easy to reverse, it may be reasonable to let the agent act within defined limits.

For example, an agent might update a customer’s email preference from “weekly” to “monthly” after the customer asks. If the customer later says that was wrong, the preference can be changed back. The recovery plan is straightforward: retain the request, record the preference change and provide a simple correction path.

Hard-to-Reverse Actions Need Stronger Checkpoints

Other actions become difficult to correct as soon as they happen. A warehouse may ship an order within minutes. A hotel may release a reservation. A refund may be sent to a payment network. A replacement may consume limited inventory. A message may disclose information that cannot be undisclosed.

These actions should usually require stronger checkpoints. The checkpoint might be required evidence, customer confirmation, employee approval, a system rule that pauses execution until required evidence exists or a narrower tool that only performs the safest version of the action.

Consider a delivery-address change. Before shipment, the agent might be allowed to update the address after the customer confirms the exact new address, required evidence is verified and account verification succeeds. After shipment, customer confirmation is not enough; the same request may need a carrier workflow or employee approval because the wrong change could redirect an expensive package.

Some Mistakes Cannot Be Fully Repaired

Disclosure mistakes are different. If an agent sends personal information, internal notes or order details to the wrong recipient, the business may correct the account record, but it cannot make the disclosure disappear. That possibility should shape the agent’s messaging permissions.

For example, an agent that can read order history and send outbound email should not automatically include every field it can see. The message tool can limit available fields, hide internal notes and require review when a message includes sensitive account information.

The FTC has reminded companies to uphold privacy and confidentiality commitments when using AI, including commitments about how customer data is used. A permission plan should account for what information the agent may reveal, not only what records it may read.

Build Detection Into the Workflow

Recovery starts with detection. The business needs a way to know what the agent requested, what the system allowed, what evidence was checked and what action was completed. Without that record, a correction process becomes a search through transcripts and guesses.

For a refund action, useful records include order ID, return scan, item condition code, refund amount, payment destination, approval ID if any, transaction ID and final customer message. For an account change, useful records include verification result, changed field, previous value where appropriate, new value, timestamp and channel.

OWASP’s authorization guidance recommends logging authorization events and enforcing access control checks server-side. For customer-facing AI agents, logs are not just technical artifacts. They are the evidence needed to understand and correct what happened.

Detection should also include checks that compare the completed action with the rule that allowed it. A refund workflow can compare the completed refund with the approved amount and original payment method. An address-change workflow can compare the resulting address with the customer-confirmed address before shipment. A mismatch, missing approval record or customer report should route to a named recovery owner rather than waiting for someone to search through transcripts later.

Action records also need their own permission boundary. The agent should not be able to alter the record that proves what it requested, what the system allowed, what evidence was checked and what action was completed. The system should preserve the evidence needed for review while restricting access to the record, excluding unnecessary sensitive information and defining how long the record is retained. Otherwise, the same agent that made or attempted the action could also change the evidence needed to understand it.

Recovery record diagram starting with a mistake or customer report such as a wrong address, excess refund, mistaken cancellation or information disclosure. The record preserves the request, completed action, evidence checked, pause on further changes, route to the right owner and close-the-loop record. The expected result is system evidence of the request, decision, recovery outcome and customer message.
A useful recovery record preserves the customer report, completed action, evidence checked, pause decision, recovery owner, recovery outcome and final customer message. The agent should not simply produce its own account of success.

Match Authority to the Recovery Plan

The permission decision should reflect the recovery path. If a wrong action is easy to detect and reverse, the agent may be able to act within limits. If a wrong action is hard to detect, expensive to reverse or impossible to fully repair, the agent should pause before making the commitment.

Here are practical examples:

  • Preference change: The agent may act within limits after customer request because the change is visible and reversible.
  • Routine refund: The agent may act only when return scan, item condition, amount limit and original payment method all match the rule.
  • Shipped order change: The agent should not treat customer confirmation as a substitute for a required carrier process or employee approval.
  • Canceled reservation: The agent should pause before canceling because inventory may be released and not recoverable.
  • Disclosure of personal information: The agent should have strict message controls because disclosure cannot be fully undone.
Authority and recoverability comparison with three levels. Easy to detect and reverse actions such as preference changes may allow the agent to act within limits when the request is clear and recorded. Correctable but consequential actions such as refunds or address changes should verify required evidence and customer confirmation, plus employee approval when the action exceeds routine authority. Hard or impossible to fully repair actions such as shipped orders, canceled reservations or personal information disclosure should pause, use narrower tools or be marked not permitted.
Authority should follow recoverability. Actions that are easy to detect and reverse may fit within routine limits; consequential or hard-to-repair actions need evidence checks, customer confirmation, employee approval when authority is exceeded, narrower tools or a Not permitted boundary.

Design the Customer Recovery Experience

Correction is not only an internal process. The customer needs a clear path when something goes wrong. The agent should be able to acknowledge the issue, preserve the customer’s report, avoid making unsupported promises and route the case to the right owner.

Finding a mistake does not automatically give the agent permission to reverse a payment, restore a reservation or change an account. The recovery action needs its own permission boundary: what correction is allowed, what evidence must be checked and when employee approval is required. Otherwise, an attempted fix can create a second mistake.

A useful recovery message might say: “I see the delivery address was changed on this order. I’m pausing further changes and routing this to our support team for review. I’ve included the original address, the requested address and the timestamp of the change.” That message explains what is happening without pretending the agent can fix every consequence on its own.

The recovery plan should also say when an agent is no longer the right actor. If the issue involves suspected account takeover, payment misuse or disclosure of sensitive information, the agent may need to stop normal handling and route to a specialized team.

Test Mistakes, Not Only Successes

Teams often test whether the agent can complete the happy path. Permission testing should also include mistake paths. Try the refund request without a return scan. Try the address change after shipment. Try the cancellation when the customer’s wording is ambiguous. Try an outbound message that would include internal-only information.

The expected result is not always “the agent fixes it.” Sometimes the expected result is that the agent pauses, asks for confirmation, requests approval, blocks execution or routes the case to a person. Those outcomes are part of the design.

Plan Recovery Before Granting Authority

Permission planning should include recovery planning. Before granting authority, decide how a wrong action would be detected, corrected and explained. Then use that answer to choose whether the action belongs under Act within limits, Request approval or Not permitted. The permission planning worksheet and completed refund example can help turn those decisions into a concrete plan.

Use the Permission Plan Builder to turn these decisions into a working plan.

For a worked refund scenario that separates reversal, correction and consequences that cannot be fully undone, see Can Your AI Agent Undo Its Mistake? A Worked Refund Example.

Sources


2 responses to “When an AI Agent Makes a Mistake: Design for Correction and Recovery”

  1. […] tool returns a timeout, an error, an unavailable status check or a conflicting record. The guide to correction and recovery can help decide when a failed or uncertain action should pause. The guide to building a complete […]

  2. […] An agent saying “your refund is complete” is not sufficient evidence without the relevant business-system confirmation. A timeout is especially important because it can leave the outcome unknown. The team should check the transaction record before retrying; another attempt could create a duplicate refund. For more detail, see When an AI Agent’s Tool Fails: Pause, Verify and Avoid Duplicate Actions and When an AI Agent Makes a Mistake: Design for Correction and Recovery. […]