OpenAI's Agents API launch this month gives developers a managed agent runtime while leaving application tools in their hands. Google's Gemini 3.8 Live developer release adds live conversations that can keep moving while tools run. In both cases, a tool call still crosses into your application, where a malformed argument or an overbroad action can affect a customer.
Before enabling any chatbot API action, run this release gate: validate the tool declaration with the provider, validate every returned argument locally, bind the action to the authenticated customer, test a duplicate call, and record what the downstream API actually did. A successful model response proves none of those steps by itself. The test matrix below gives each check a concrete pass condition.
Define the action before defining the schema
Take a support chatbot that can create a ticket for a damaged order. The tool should do one thing: create a support ticket for an order the current customer owns. It should not issue a refund, edit the order, or choose an account based on text the caller supplied.
Write the application's contract first:
- The customer is authenticated before the tool is available.
- The model may supply an order reference, issue category, summary, and whether the customer agreed to follow-up.
- The server supplies the customer identity from the session. The model cannot choose it.
- The server confirms ownership of the order and applies its own length and format limits.
- A repeated request with the same conversation and confirmed intent creates one ticket.
- The result contains a ticket ID or an explicit failure. A friendly sentence from the chatbot is not a receipt.
That contract decides what belongs in the tool declaration. If the declaration exposes customer_id, refund_amount, or an arbitrary API URL, the model has authority it does not need for ticket creation. Removing those fields is simpler than trying to persuade a model never to use them.
The existing tool-selection guide helps decide which action the chatbot should offer. Once you choose the action, the schema and the application must agree on its exact inputs.
Keep the model's argument surface small
Here is a representative function declaration for the ticket action. The example uses OpenAI's documented strict-function shape; other providers use different wrappers, so translate the declaration to the provider you run. The underlying ticket contract stays the same.
{
"type": "function",
"name": "create_support_ticket",
"description": "Create one support ticket for an order owned by the signed-in customer after the customer asks for follow-up.",
"strict": true,
"parameters": {
"type": "object",
"properties": {
"order_id": { "type": "string", "description": "The order reference the customer confirmed." },
"issue_type": { "type": "string", "enum": ["damage", "missing_item", "delivery"] },
"summary": { "type": "string", "description": "A short account of the customer's issue." },
"contact_permission": { "type": "boolean", "description": "Whether the customer agreed to a follow-up." }
},
"required": ["order_id", "issue_type", "summary", "contact_permission"],
"additionalProperties": false
}
}
OpenAI's function-calling guide says strict mode requires every property to appear in required and every object to set additionalProperties to false. If an input is optional, its strict-schema pattern uses a nullable type while keeping the property required. Do not silently turn strict mode off to get an invalid declaration accepted. A provider rejection in a deployment test is cheaper than finding a broken action during a live conversation.
The schema describes syntax and a few permitted values. It cannot prove that AK-0819 belongs to the signed-in customer, that the customer really consented to contact, or that the summary is accurate. Those checks belong in the application. Google's function declaration guide likewise describes a name, description, parameter types, properties, and required fields. Treat a provider's accepted declaration as the first check, not the full authorization decision.
Validate at the execution boundary
Suppose a customer says, "The glass arrived broken. My order is AK-0819. Please email me about a replacement." The chatbot proposes this call:
{
"order_id": "AK-0819",
"issue_type": "damage",
"summary": "Glass arrived broken; customer requests replacement follow-up.",
"contact_permission": true
}
The application should process it in a fixed order. Parse the arguments and check the local schema. Resolve the order under the authenticated customer's account. Confirm the permission against the conversation or an explicit UI consent signal. Generate an idempotency key for this requested ticket. Call the ticket system. Save the downstream response and return the ticket ID to the chatbot.
Now change one fact. The model emits AK-0891, a different but well-formed order reference. A strict provider schema accepts the string. The local format check also passes. The ownership lookup must still reject it if the signed-in customer does not own that order. The chatbot should ask the customer to repeat the reference, not try a wider lookup and not create a ticket against somebody else's order.
A second failure is subtler: the customer said "do not email me," but the model sets contact_permission to true. A boolean schema cannot detect that contradiction. Use a separately recorded consent event when contact permission matters. If that event is missing, return a targeted failure and ask. Never convert the model's confident wording into evidence of consent.
This is why the execution boundary needs the same checks regardless of model or provider. OpenAI's programmatic tool-calling guidance explicitly calls for argument and permission checks in the application, idempotent functions where possible, and application approval for high-impact actions. The model's JSON is an input to your service, not an authenticated instruction to the downstream API.
Use a test matrix that reaches the real boundary
Run these cases through the same adapter and service used in production. A prompt-only test that asks the model what it would do cannot prove the service rejected a bad call. Keep the downstream ticket API in a test environment or replace only that final external dependency with a fake that records requests.
| Test input or condition | Required result | Evidence to retain |
|---|---|---|
| Declaration has a missing required field or unsupported property | Provider rejects the declaration before release | Provider error and schema version |
Model omits order_id, adds refund_amount, or sends "true" as text | Local validator rejects without calling the ticket API | Validation error; zero downstream calls |
issue_type falls outside the allowed values | Local validator rejects and asks for a supported issue | Rejected enum; zero downstream calls |
| Order reference has the right shape but belongs to another account | Ownership check rejects without exposing order details | Authorization result; zero downstream calls |
| Contact permission conflicts with the recorded customer choice | No ticket with email follow-up is created | Consent record and rejected call |
| Same confirmed ticket request arrives twice after a retry | One ticket exists and both attempts resolve to its ID | Idempotency key and ticket-system receipts |
| Ticket API times out after accepting the write | Service reconciles by idempotency key before retrying | Original request ID and lookup result |
| Tool version changes while an old conversation remains open | Old call is handled by a compatible version or fails clearly | Declared version and execution version |
The pass condition is usually zero unauthorized downstream calls, not merely a polite chatbot reply. A chatbot can apologize after it has already sent a bad request. Instrument the service so the test can see both the returned message and the attempted side effect.
For high-impact actions such as refunds or account changes, add a separate approval gate. The human-confirmation guide explains where an approval belongs. A strict JSON schema does not substitute for it.
Test declaration drift as part of deployment
Tool declarations can fail before the model calls them. A renamed property, a field made optional in code but required by a provider's strict mode, or a new enum value that the backend does not recognize can break the path between a plausible demo and a live customer.
Keep one versioned source for each action contract. Generate or compare the provider declaration, the local runtime validator, and the service input type from that source when your stack allows it. When they are maintained separately, add a test that sends the exact deployed declaration to each provider you support. Record the accepted schema version with every call. The test must use the real provider adapter; a JSON parse alone cannot tell you whether that provider accepts the declaration.
Do not widen the schema to solve a provider incompatibility without reviewing the action. If a provider cannot express a constraint such as an order-reference pattern, keep a simple string in the declaration and enforce the pattern locally. If the provider accepts an optional field differently, translate the wrapper in its adapter and keep the business meaning unchanged. The adapter is allowed to vary. The ticket service should not have one authorization rule for each model.
Also test a deployment with an existing conversation still in progress. It may hold a tool definition from the previous version. A service that suddenly interprets issue_type: "shipping" as "delivery" without an explicit mapping can misroute work. Either accept the old version for a defined window or reject it with a recoverable message. Silent reinterpretation is the dangerous outcome.
Make failures legible without leaking data
When validation fails, give the chatbot a structured result it can use to recover. A response such as {"code":"ORDER_NOT_FOUND_OR_NOT_OWNED","retryable":false} lets it ask the customer to verify the reference without confirming whether another person's order exists. Avoid returning raw database errors, access tokens, or the full API response to the model.
For operators, log enough to answer a different question: did the action run? Record the tool name and version, conversation ID, authenticated account ID, validation outcome, idempotency key, downstream request ID, and final status. Treat the submitted summary and customer identifiers as sensitive data under your normal retention policy. If the downstream API reports a timeout, preserve an unknown state until reconciliation proves success or failure. Marking it failed immediately can create a duplicate on retry.
Track three rates separately: declaration rejection during deployment, argument rejection before execution, and downstream action failure after validation. A single "tool success" percentage hides which boundary broke. The action-verification guide shows why a completed write needs a receipt instead of the model's assertion, while audit-trail guidance covers the record a reviewer needs after the fact.
The release decision
Approve the tool only when the declaration loads on every intended provider, bad arguments produce zero downstream calls, account checks reject cross-customer references, retries produce one effect, and the final ticket ID comes from the ticket system. Keep the test cases alongside the tool version. Rerun them when the model, prompt, schema, adapter, or downstream API changes.
Chatbots become useful when they can move beyond answers. A narrow schema makes the proposed action understandable; the service decides whether it is allowed and proves whether it happened. That boundary is what lets a support team trust an API action after the demo ends.
No credit card required.
