Making Agent Retries Safe After the Tool Already Sent the Email

An agent calls your send_invoice_email tool. The email provider accepts the message, but the response takes 31 seconds and your HTTP client gives up at 30. The tool returns a timeout error to the agent. The agent, doing exactly what a reasonable agent does, tries again. The customer now has two invoices in their inbox, and your support queue has a new ticket that says "why did I get this twice?"

Nothing in that story is a model failure. The model read an error and chose to retry. The failure is that the tool reported "I do not know what happened" as if it meant "it did not happen." Ordinary services have dealt with this for decades. Agents make it worse because the retry decision is made by a component that cannot see your infrastructure, and because it may also retry by rephrasing the request, which no HTTP-level dedupe will catch.

The fix is not a better prompt that says "never retry." It is making the tool layer safe to call twice, and being explicit about the few tools where that is impossible.

Sort your tools by what a second call does

Before writing any code, put every tool into one of three classes. This takes an hour and settles most of the design.

• Read-only. Search, lookup, fetch a record. Retry freely. The only cost is latency and tokens.
• Naturally idempotent. "Set the ticket status to closed," "assign this order to warehouse 3." Calling it twice leaves the same end state. Retry freely, but make sure the tool does not append a second audit entry or fire a second webhook on every call.
• Effectful and not idempotent. Send an email, charge a card, create a record, post to a channel. A second call produces a second effect. These need a deduplication mechanism, or they need a human.

Most agent incidents come from the third class being treated as the second. "Create a task" feels like a state change, but two calls create two tasks.

Put the idempotency key in the tool, not in the prompt

The standard mechanism is an idempotency key: a stable identifier for one intended action, stored with the result. If the same key arrives again, the tool returns the stored result instead of acting. The important decision is who generates the key. If the model generates it, a retry that rephrases the request produces a new key and you are back to two emails. Generate it from the agent run and the step instead, in code the model cannot influence.

function intentKey(runId: string, stepId: string, tool: string, args: unknown) {
  // Same run + same step + same logical arguments => same key,
  // no matter how many times the model re-issues the call.
  return sha256([runId, stepId, tool, canonicalJson(args)].join(':'));
}

async function callEffectful(tool, args, ctx) {
  const key = intentKey(ctx.runId, ctx.stepId, tool.name, args);
  const prior = await db.intents.get(key);
  if (prior?.status === 'done') return prior.result;      // replay stored result
  if (prior?.status === 'started') return { pending: true }; // outcome unknown

  await db.intents.put(key, { status: 'started', at: Date.now() });
  const result = await tool.run(args, { idempotencyKey: key });
  await db.intents.put(key, { status: 'done', result });
  return result;
}

Two details matter here. First, canonicalJson sorts keys and normalizes whitespace so that equivalent arguments hash the same. Second, pass the key onward to the downstream provider when it supports one. Many payment and email APIs accept an idempotency header, which closes the gap between your record and theirs.

Handle the unknown state honestly

The started status is the uncomfortable one. It means we began the call and never recorded the outcome. The process may have crashed, or the response may have timed out after the provider accepted the request. There are only three honest options:

1. Ask the downstream system. If it exposes a lookup by key or by reference, query it and settle the record to done or failed.
2. Return pending to the agent with an instruction that the outcome is unknown, and let the run park until a reconciler or a person resolves it.
3. Retry, but only when the downstream provider guarantees dedupe on your key.

Do not return a generic error string. An agent reading "request failed" will retry. An agent reading "outcome unknown, do not repeat this action, check status with get_email_status" will usually do the right thing, and it gives you something to assert on in tests.

Write the intent before the effect

Record the intent before you perform the side effect, not after. If you only write the record after success, a crash between the effect and the write leaves no trace and the retry sends again. The write-ahead record is what turns "we do not know" into "we know we tried." This is the same discipline we apply to usage metering and billing events, where a missing record means either a lost charge or a double one. Our piece on usage metering that holds up in a dispute covers the same append-first pattern.

Where this advice does not apply

Idempotency keys cost a table, a lookup on every effectful call and some engineering care. They are not worth it for a tool that writes to a scratch area the user reviews before anything leaves the building. If every effect lands in a draft, the human is your dedupe step, and the simpler design is to let the agent create drafts freely.

They also cannot rescue an effect with no stable identity. "Post a message to this channel" has no natural key unless you invent one, and the recipient sees two messages regardless of what your database says. For those tools, the safer choice is a confirmation step before the call, which is the design we describe in human-in-the-loop review that people actually use.

Test it by injecting the failure

You will not find these bugs by running the happy path. Add a test harness that wraps each effectful tool and, on a configurable fraction of calls, performs the effect and then throws a timeout. Run your agent scenarios against it and assert two things: the downstream mock received exactly one effect per intended action, and the final answer to the user does not claim success when the state is still pending.

A short checklist for review:

• Every effectful tool is listed and classed.
• Keys derive from run, step and canonical arguments, never from model output.
• Intent is recorded before the effect.
• Timeouts map to pending, not to a generic error.
• A fault-injection test exists for each class-three tool.

If your agents run for minutes rather than seconds, the same state machine becomes the backbone of resumability. Platforms that must keep working through partial failure, like the offline point of sale we ship in Retailer OS, rely on the same principle: record intent, replay safely, reconcile later. If you want help applying it to an agent you already run, talk to us.