Customer support chatbot: setup, human handoff and testing

Set up a customer support chatbot with approved answers, helpdesk escalation and a launch test matrix. Measure resolution without counting abandoned chats.

Cover Image for Customer support chatbot: setup, human handoff and testing

A customer support chatbot answers routine questions from your help content and sends unresolved issues to your team. A useful setup needs both paths. An accurate answer to "How do I reset my password?" helps; telling a locked-out customer that someone will contact them without creating a support request leaves work unfinished.

In Agentkit, the Escalate to Human action creates a ticket, case or conversation in a connected support tool. It requires a paid Agentkit plan and a valid follow-up email. This action creates a request for human follow-up. It does not connect a live person to the chat.

This guide covers a website support chatbot. Start with approved sources, connect the helpdesk your team already uses, then test the answer and escalation paths before embedding it.

Choose which requests the chatbot should handle

Use documentation questions for your first release. Keep account changes and sensitive requests on an approved support path.

RequestSuitable first-release behaviorWhat to check
Password reset instructionsExplain the documented reset processCorrect link and steps; no request for a password
Refund policyExplain the published policyCurrent conditions; no promise that a refund is approved
Product setupGive steps from the relevant guideCorrect product version and prerequisites
Account access or disputed chargeCollect safe context and create a support requestCorrect destination and follow-up email
Current outage or order statusUse a configured source or action that can supply that informationNo invented live status from an old help article
"I want a human"Use the configured escalation actionRequest reaches the support tool

Knowing a policy does not give the chatbot permission to change an account. A help-center article can explain how refunds work, but processing a refund needs a separate authorized workflow. Define those limits in the instructions and test them.

Add sources that answer actual support questions

Review recurring questions from your support queue. For each question, identify the approved answer, its owner and what should happen when the answer does not help. Remove personal customer information before using ticket material as training content.

Agentkit supports website pages, documents, text and Q&A sources. Crawl the relevant help pages, then add missing policies or answers. Avoid importing an entire marketing site just because it is available. Conflicting pricing, old instructions and announcements can make a support answer harder to trust.

Wait until the sources show Ready. In the Playground, ask the questions in your own words and use Show Sources to inspect the retrieved content. A relevant source is useful evidence, but still read the answer. Check whether it includes the required conditions and links.

Keep a small acceptance sheet with the question, expected answer, source URL and escalation decision. Reuse it after source changes. For the source preparation process, see how to train a chatbot.

Connect human follow-up before launch

The public Escalate to Human setup supports these destinations, as documented in the action guide.

Support toolConnectionCreated item
ZendeskSubdomain and OAuth authorizationTicket
FreshdeskSubdomain and agent API keyTicket
Help ScoutOAuth authorization and inbox selectionConversation in the selected inbox
IntercomOAuth authorizationConversation
HubSpotOAuth authorization; portal needs a ticket pipelineTicket
SalesforceProduction or sandbox OAuth authorizationCase
  1. Open your agent and go to Actions > Integrations.
  2. Connect the support tool that should receive requests. Complete the provider setup shown above.
  3. Confirm the integration card shows Connected.
  4. Go to Actions, choose Add Action, then Escalate to human.
  5. Select the connected ticket platform and review When to use.
  6. Save and enable the action.

Only connected external destinations appear in the picker. If the provider needs reconnecting, fix the connection before enabling the action.

The action sends a subject, a summary from the visitor's perspective, their email, their name when available and recent conversation context. Zendesk receives the escalation as an internal note. Inspect a test request in the receiving tool to check that the context is useful to your team.

Choose who monitors that inbox and how they reply. Set response expectations from your actual support policy. A created ticket does not establish that an agent is available now or that the issue will be solved within a particular time.

Write instructions that match the action

Use the agent's AI settings for answer boundaries and the action's When to use field for escalation triggers. Adapt this example to your policies and contact method.

Answer support questions using the approved training sources.
If a required fact is missing, say what you cannot confirm.
Do not promise refunds, account changes or live service status without
a configured workflow that verifies or performs them.

Create a support request when the visitor asks for a human, reports a
billing or account access issue, or says the suggested answer did not help.
Collect or confirm a valid follow-up email before using the escalation action.
Ask for a name only when useful. Do not ask for passwords or full card details.

Write the subject and summary yourself using the issue and steps already tried.
Only confirm that a request was created after the action succeeds.
Explain that the team will follow up by email. Do not promise live chat.
If creation fails, give the approved support contact method and explain
that the request could not be created.

An email capture form and a helpdesk escalation do different jobs. The Collect Leads action captures contact details. Use Escalate to Human when the required outcome is a request in your support tool.

If cases need different specialist teams, define the receiving helpdesk's routing rules and verify them there. The handoff routing guide covers the design questions. Do not assume that naming a team in the prompt creates a queue or assigns a ticket to it.

Test the complete support path

Use a test inbox and non-sensitive examples. Enabling an action can create real items in the connected support tool, so mark test requests clearly and keep the receiving team informed.

The following matrix is an illustrative acceptance checklist. It is not a report of measured Agentkit results.

Test input or conditionExpected behaviorEvidence to inspect
A documented setup questionAnswer matches the approved guideAnswer text, source and working link
A fact absent from all sourcesAcknowledges the gap without inventing detailsReply and any escalation decision
"I want to speak to support" without an emailAsks for a follow-up email before creating a requestConversation and receiving inbox
A human request with a valid emailCreates the item in the configured tool, then confirmsTool result and actual ticket, case or conversation
An invalid emailRequests a valid email; does not report successConversation and absence of a created item
"That did not solve it" after troubleshootingFollows the configured escalation triggerSummary includes what was already tried
A second escalation in the same conversationUses the existing escalation instead of creating anotherExisting item ID and receiving inbox
Expired or revoked provider connectionReports failure and gives the approved contact methodIntegration state, failure result and visitor reply
A request in each supported customer language you useAnswers and follow-up expectations remain accurateReview by someone who can assess that language

Repeat the important cases in the published widget, including on mobile. Check the actual destination, rather than treating a friendly confirmation as proof that the request arrived. See the deployment guide for embedding options.

Channel settings matter. An action enabled for the website should also be allowed by that channel's action selection. The built-in Escalate to Human action is not supported in Agentkit's email channel. Test other configured channels separately and give email its own approved fallback path.

Measure resolution separately from silence

A customer leaving the chat does not establish that their problem was solved. Keep confirmed answers, unresolved chats and requests handed to humans separate.

For comparison, Intercom's Fin workflow documentation distinguishes confirmed resolution from assumed resolution when a customer abandons the conversation. Your own reporting needs an equally clear definition before you call a number a resolution rate.

MeasureWorking definitionWhat it tells you
Confirmed resolution rateConfirmed resolved conversations divided by the support conversations in your defined cohortWhether customers received useful help
Escalation rateConversations with a human-support request divided by that same cohortHow much work reaches the team
Escalation delivery failure rateFailed creation attempts divided by creation attemptsWhether the integration needs attention
Repeat contactCustomers who contact support again about the same issue within your chosen review windowWhether apparent resolutions held up
Knowledge gapsQuestions whose required answer is absent, stale or contradictoryWhat to add or correct next

Choose the review window and denominator before comparing periods. Escalation rate is not the inverse of resolution rate because unresolved or abandoned conversations can exist too. Do not set a universal deflection target from another company's results.

Use Chat Logs to review answers and escalations, then combine that review with the receiving helpdesk's records. The chatbot KPI guide explains the wider measurement choices. Check the available reports before assuming every metric above is calculated automatically.

Start on one support page with a named owner for unanswered questions and failed handoffs. Fix missing source content, re-run the acceptance cases and expand only when the receiving team can handle the resulting requests.

Create an Agentkit account to prepare and test your support sources. Enable a paid plan and connect a supported helpdesk when you are ready to add Escalate to Human.

Get started freeNo credit card required