Chatbot Action Verification: Prove Every Task Completed

OpenAI launched GPT-6 Astra on September 3 with stronger computer use and multi-step work. Three weeks earlier, its enterprise research described a broader shift from AI that answers questions to AI that carries out work. A support chatbot can now update a CRM record, submit a cancellation, create a return, or book an appointment. That raises a blunt question: how does the bot know the task actually finished?

Start with an action receipt. Do not let the chatbot say "done" unless it can fill every required field:

That small record is the core of chatbot action verification. The rest of this guide shows how to produce it, test it, and turn uncertain outcomes into honest customer messages.

A successful tool call is not a completed task

Most chatbot action bugs happen in the gap between an API response and the business outcome the customer asked for.

An HTTP 200 may mean the request was received, not committed. A 202 may mean a job entered a queue. A payment provider can accept a refund and reject it later. A calendar API can create an event while the invitation email fails. A CRM can save a lead under the wrong account. The model sees a pleasant JSON response and writes a confident confirmation.

The customer does not care that a tool ran. They care that the requested state now exists.

Treat completion as a sequence:

1. The chatbot understood the intended outcome and exact parameters.
2. The customer approved any consequential action.
3. The tool submitted one uniquely identified request.
4. The system of record accepted or committed the change.
5. A fresh check found the expected state.
6. Required side effects occurred, or the chatbot named what remains pending.

Every step needs its own evidence. A later step may fail after an earlier one succeeds, so one generic success: true flag cannot describe the result.

This is where verification differs from an AI agent audit trail. The audit trail reconstructs what the agent tried. The action receipt states whether the customer's outcome exists now.

Define completion before connecting the action

Write a completion contract for each action. Keep it beside the tool schema so product, support, and engineering review the same promise.

A useful contract names seven things:

• Preconditions. What must be true before execution, such as an authenticated customer, a cancellable subscription, or an available appointment slot.
• Approved parameters. The values the customer confirmed, including amount, date, account, destination, and scope.
• Submission result. What the first API response proves and what it does not prove.
• Postcondition. The exact state a fresh read must return before the action counts as complete.
• Side effects. Emails, credits, inventory changes, notifications, or downstream records that matter to the outcome.
• Uncertain state. The timeout or ambiguous response that must produce pending, never a guessed success or failure.
• Recovery owner. The service, queue, or person responsible for the next check.

For a cancellation, the postcondition might be subscription.status = canceled and renews_at = null. For an appointment, it might be an event with the approved time, attendee, and timezone plus an invitation status. For a support ticket, a durable ticket ID may be enough because creation itself is the requested outcome.

Do not let a tool choose its own meaning of success. The orchestration layer should compare the returned evidence with the contract.

Worked example: the cancellation that looked finished

Suppose a customer says:

Customer: Cancel my Pro plan at the end of this billing period.

Chatbot: Your Pro plan will end on September 30. You will keep access until then,
and you will not be charged again. Cancellation reference: cnl_8M42.

That answer sounds complete. Now inspect the first tool result:

{
  "status": 200,
  "request_id": "req_74d1",
  "message": "Cancellation request received"
}

The response proves only that the service received a request. It does not prove the end date, access period, renewal state, or reference quoted to the customer. The chatbot invented four facts around one weak signal.

A verified flow uses an idempotency key, waits for a committed result, and reads the subscription again:

{
  "action_id": "act_cnl_8M42",
  "idempotency_key": "cancel_sub_3821_v1",
  "requested": {
    "subscription_id": "sub_3821",
    "effective": "period_end"
  },
  "provider_result": {
    "state": "accepted",
    "request_id": "req_74d1"
  },
  "postcondition": {
    "checked_at": "2026-09-05T09:14:22Z",
    "subscription_status": "active_until_period_end",
    "renews_at": null,
    "access_ends_at": "2026-09-30T23:59:59Z"
  },
  "side_effects": {
    "confirmation_email": "queued"
  },
  "customer_status": "confirmed"
}

Now the message has support. The subscription record confirms the date and renewal state. The email remains queued, so the chatbot can say the cancellation is confirmed while treating the email as a separate pending side effect.

If the read-back times out, the receipt should say pending. The correct answer is then: "Your cancellation request was accepted, but I could not confirm the subscription update yet. I will not submit it again. Support reference: req_74d1." That sentence is less satisfying than "done," but it prevents duplicate cancellations and false promises.

Verify against the system that owns the state

Read-after-write verification is the simplest reliable pattern. After the action returns, query the authoritative record through a separate read operation. Compare only the fields in the completion contract.

Use the billing system to verify a refund, the calendar to verify a booking, the CRM to verify an account update, and the ticketing system to verify a case. A copy in the chatbot's conversation history is not authoritative. Neither is the action service's own optimistic cache.

Some systems need other evidence:

• Webhook confirmation. The provider sends a signed event when an asynchronous job commits. Store the event beside the action ID.
• Polling with a deadline. Check a stable status endpoint until it reaches a terminal state. Stop at a fixed deadline and return pending.
• Reconciliation query. Search by idempotency key when the original request timed out. This tells you whether a retry would duplicate the action.
• Side-effect check. Verify the email, inventory reservation, shipment, or credit when that effect is part of the customer promise.

Never ask the model to infer completion from prose such as "processing normally" or "should be updated shortly." Give it a small set of explicit states and the evidence allowed for each one.

Make retries safe before you automate them

Network failures create the most dangerous uncertainty. The service may commit the action and lose the response on its way back. A blind retry can issue two refunds, create two tickets, or book two appointments.

Assign an idempotency key before the first attempt. Reuse that same key for every retry of the same approved action. The receiving service should return the original result instead of applying the change twice.

Bind the key to the approved parameters. If the amount, destination, account, date, or action type changes, require a new approval and a new key. The chatbot approval workflow guide explains how to keep confirmation tied to the exact operation rather than a vague permission to proceed.

Set a retry budget too. One timeout should trigger a status check before another write. Repeated timeouts should move the receipt to pending and create a review item. Ten frantic retries do not produce more certainty.

Give customers three honest outcomes

Customer-facing copy should map directly to the receipt state.

Confirmed. The required postcondition is present. State what changed, name the important parameters, and provide the durable reference.

Pending. The request was accepted but the postcondition is not available yet. Say what is waiting, when the next check will happen, and how the customer can refer to the action. Avoid "probably," "should," and made-up completion times.

Failed. The system returned a terminal failure or the postcondition contradicts the requested result. State that the action did not complete, explain any safe retry or handoff, and make clear whether anything changed.

Partial completion deserves its own wording. If an appointment exists but the invitation failed, keep the booking and report the invitation problem. Do not collapse the whole action into success or failure. The receipt should identify each side effect so the recovery step targets the broken part.

Test the evidence path, not only the happy path

A demo proves little because the API, queue, webhook, and read-back all cooperate. Build tests around the awkward boundaries:

• The write commits, but the response times out.
• The provider returns 202 and never sends the webhook.
• The postcondition appears after the polling deadline.
• A retry arrives with the same idempotency key.
• A retry changes one approved parameter.
• The main record updates, but a required side effect fails.
• The action lands in the wrong tenant or customer account.
• The read-back returns stale cached data.

For every test, assert both the external state and the customer message. A system can recover correctly while the chatbot still says the wrong thing.

The trajectory monitoring guide helps locate the step where a longer workflow went off course. Action verification adds the release gate: no terminal success message without matching postcondition evidence.

Review uncertainty as a product metric

Track more than raw action success. Measure confirmed, pending, failed, partial, duplicate-prevented, and manually corrected outcomes by action type.

The most useful rate is:

verified completion rate = confirmed outcomes / approved action attempts

Keep pending outcomes visible until they reach a terminal state. If a dashboard drops them after the conversation ends, it will make unreliable workflows look healthy. Also compare the chatbot's final message with the eventual system state. A "confirmed" message followed by a failed action is a false-confirmation incident and should trigger review.

OpenAI's recent enterprise report says agents are spreading from engineering into legal, sales, recruiting, and marketing. Simon Willison argues that verification, rather than line-by-line inspection, is the core skill for productive agent use. Customer-facing actions need the same discipline, compressed into evidence that software can check every time.

Make "done" a claim you can defend

Better models will attempt more work, across more tools, with fewer pauses. That does not make completion self-evident. Each action still needs a contract, a unique identity, a check against the system of record, and a customer message tied to the evidence found.

If you connect actions through Agentkit's custom API calls or webhooks, design those endpoints to return stable IDs and explicit states. Conversation logs can then preserve the action reference and customer-facing result for review.

Build your chatbot for free →

No credit card required.