Chatbot KPIs are measurements tied to a goal, such as resolving a support request or generating a qualified lead. Message volume is a metric; confirmed resolution against a defined target is a KPI. Choose the outcome first, then specify which conversations count and what evidence establishes success.
For support, start with confirmed resolution, unknown outcomes and completed handoffs. For user experience, add satisfaction, survey response rate and response latency. For sales, measure qualified leads within eligible conversations. A conversation ending without a human does not establish that the visitor's problem was solved.
Chatbot KPI definitions and formulas
The table uses a proposed reporting convention, not a universal vendor standard. Define an eligible conversation as one real visitor conversation in the selected use case and reporting period. Exclude tests and spam using a recorded rule. Keep open conversations separate until their observation window closes. Every percentage below multiplies the fraction by 100. A zero denominator means unavailable, not 0%.
| KPI | Formula | What to record alongside it |
|---|---|---|
| Confirmed resolution rate | Conversations with confirmed bot resolution / eligible conversations | Confirmation method and unknown outcomes |
| Containment rate | Conversations with no recorded human escalation / eligible conversations | Escalation coverage and unresolved contained conversations |
| Handoff request rate | Conversations requesting human help / eligible conversations | Requested, failed and completed handoffs separately |
| Handoff completion rate | Requested handoffs accepted by a human or destination queue / requested handoffs | Acceptance evidence; queue receipt is not issue resolution |
| CSAT | Positive ratings / valid ratings received | Rating scale, positive threshold and number of responses |
| Survey response rate | Valid survey responses / conversations offered the survey | Whether the survey reached representative users |
| Fallback rate | Visitor turns triggering a defined fallback / eligible visitor turns | Scope refusals, missing knowledge and system errors separately |
| Engagement rate | Widget sessions with a visitor message / observed widget-open sessions | Use the same session rule in both counts |
| Lead conversion rate | Eligible conversations producing a deduplicated qualified lead / eligible sales conversations | Qualification rule and later sales outcome |
| Unknown outcome without escalation | Non-escalated conversations without sufficient outcome evidence / eligible conversations | Missing tracking and apparent abandonment |
Use a scorecard you can audit
Download the blank KPI scorecard CSV and open it in a spreadsheet. Each row names its numerator, denominator and required evidence. Fill in the counts, reporting window and a decision. The CSV does not calculate results automatically or connect to your chatbot.
Choose one use case and close its observation window before counting. For conversation-based rates, keep a private review sheet with one row per eligible conversation, its outcome evidence and handoff status. Count each conversation once per applicable conversation-based rate. Keep separate turn, session and rating records for fallback, engagement and CSAT, using each row's stated denominator. Two fallback turns out of ten eligible visitor turns means 20%, regardless of how many conversations contain them. The scorecard stores aggregate counts; do not paste visitor messages or personal details into it.
For each percentage, divide the recorded numerator by its denominator and multiply by 100. A blank count means unknown. A zero denominator means unavailable. If your setup does not record survey offers or handoff acceptance, leave those rates unavailable and name the missing evidence. Do not infer those events from a conversation ending.
Use the result to choose a concrete correction. For example, review unknown outcomes for missing sources before deciding whether to change the answer content. The chatbot optimization guide explains how to turn that review into source changes. This scorecard is usable with manual records and does not imply that Agentkit reports all these events automatically.
Resolution, containment and deflection are different
Confirm resolution with an explicit answer about whether the issue was solved, a verified task result, or a documented transcript-review rubric. A positive rating about tone alone does not prove resolution. If you use an inferred-success rule, report that rate separately.
Silence, closing the widget or not returning within a day can mean success, interruption or giving up. Without further evidence, the outcome is unknown. Match repeat contacts about the same issue over a declared follow-up window where permitted and observable. State which channels you cannot track.
Containment describes the absence of an observed escalation under your rule. It can include unresolved conversations. Deflection asks how many human contacts were avoided compared with what would otherwise have happened. Estimate it with comparable contact rates, ideally a suitable control group. A before/after reduction alone can reflect traffic or issue-mix changes. Neither containment nor estimated deflection directly measures staff hours saved.
Worked example: the same 100 conversations
This is a hypothetical example, not Agentkit customer data or a benchmark. After the observation window closes, suppose 100 eligible support conversations contain:
- 45 confirmed bot resolutions with no human escalation.
- 20 requests for human help, including 15 accepted handoffs and 5 failures.
- 35 conversations with no escalation and no reliable outcome confirmation.
Under this reporting convention, all 20 recorded handoff requests count as escalations, including the 5 failed handoffs. The unknown-outcome-without-escalation rate excludes those requests; their eventual resolution requires separate tracking.
Confirmed resolution is 45/100 = 45%. Containment is 80/100 = 80% under the no-escalation definition. Unknown outcomes without escalation account for 35/100 = 35%; the outcomes of escalated issues still need separate tracking. Handoff completion is 15/20 = 75%, not proof that those 15 issues were resolved.
If all 100 received a survey and 10 responded, with 8 positive ratings, CSAT is 8/10 = 80% and survey response is 10/100 = 10%. That does not establish 80% satisfaction among all visitors. Human contacts avoided and staff time saved remain unknown without additional evidence.
Which user-experience KPIs should a chatbot have?
Use CSAT with its response rate, unknown or abandoned outcomes, latency and handoff completion together. A high score from a small, self-selected sample can hide dissatisfied users who left. Record survey counts and check whether escalation paths also offer feedback.
For latency, define the clock: visitor submission to first visible response, or to the complete answer. Report median and 95th percentile separately, with failed or timed-out turns counted alongside them. Fast but incorrect answers still fail the user.
Message depth is total counted messages divided by conversations. Specify whether it includes both sides. Long conversations may be useful or repetitive; short ones may succeed or be abandoned. Inspect transcripts before interpreting either. Similarly, distinguish returning users with a new need from repeat contacts about an unresolved issue.
Set targets from your own baseline
There is no defensible universal "good" percentage for these definitions across every support task. Compare the same use case, language, channel, eligibility rules and observation window. Show the numerator and denominator, not just the percentage. Small samples and changing question mix limit conclusions.
Start with a complete baseline period, choose one correction, then compare a later complete period. Keep policy-required escalations separate from avoidable failures. More containment is not a useful goal if it makes reaching a person harder. A 60% containment rate does not mean 60% less work; measure human contact volume and handling time separately.
Turn the measurements into improvements
Review a sample of confirmed, unknown, fallback and escalated conversations. A fallback can reflect missing sources, an appropriate refusal or a technical failure. Establish the cause before changing training material. Our chatbot optimization guide covers content improvements; the customer support guide helps define scope.
For low lead conversion, inspect qualification, visitor intent, form friction and the timing of the offer. Test one change rather than assuming every visitor should become a lead. See chatbot lead generation. Keep human-service outcomes visible when combining chatbots and live chat.
Use the conversation records and feedback actually available in your setup. This measurement plan does not imply that Agentkit automatically reports every formula or links visitors across support channels. Where a required event is missing, mark the metric unavailable and specify the instrumentation or manual review needed.
Start with the blank scorecard and a complete review period. If you are evaluating Agentkit, create a free chatbot and test a small question set before deciding which outcomes your production setup needs to measure.
