GPT-Live-1 reached the API on September 10, and Google released Gemini 3.8 Live five days later. Both can keep a conversation moving while other work happens. That makes a voice demo easy to admire and an interruption bug easy to miss.
Before launch, run the same voice chatbot interruption test through the actual call or browser interface. Start with the six cases below. Save the audio, timestamped playback and speech events, final answer, and any tool request. A transcript alone cannot show whether the bot kept talking over someone or sent an abandoned request to a backend.
| Test case | Caller behavior | Pass condition |
|---|---|---|
| Intentional barge-in | Says "wait" while the bot speaks | Playback stops; bot hears the new request; old answer stays stopped |
| Thinking pause | Pauses mid-sentence to find a date | Bot waits or asks a neutral clarification; no premature action |
| Backchannel | Says "mm-hm" while listening | Bot does not discard its answer or treat the sound as a new command |
| Background voice | Another person speaks near the microphone | Bot does not change intent or disclose account details |
| Corrected action | Replaces a date after the bot starts confirming it | Old action is canceled; only the confirmed date reaches the tool |
| Slow tool | Interrupts while a lookup is pending | Pending result cannot revive the canceled answer or trigger a write |
Run each case with a quiet headset, a phone speaker, and realistic background noise. Mark the expected behavior before listening to the result. Otherwise a natural-sounding recovery can hide a wrong state transition.
Decide who has the floor
An interruption test needs two separate judgments. Did the caller regain the floor? Did the application abandon work that belonged to the earlier turn? A bot can stop speaking quickly while its canceled tool call continues in the background. It can also cancel the tool correctly while talking for three more seconds over the caller.
Write a small event vocabulary that your own system can produce. The labels can differ, but the sequence must distinguish incoming speech, outgoing audio, a stable intent, and tool effects:
caller_speech_started agent_audio_started caller_barge_in_detected agent_audio_stopped intent_revised tool_request_queued tool_request_sent tool_result_received
For a web voice interface, record what the browser played, not only what the model generated. Buffered audio may continue after the model has stopped. For a phone line, capture the telephony playback stop as well as the model event. Keep clocks synchronized closely enough to order events; if different services disagree by hundreds of milliseconds, compare sequence IDs and traces before reporting a precise stop time.
OpenAI says GPT-Live-1 reasons over incoming and outgoing audio together and reports fewer interruptions in an early language-tutor evaluation. Google describes Gemini 3.8 Live as able to call tools while continuing dialogue. Those are useful capabilities, but neither announcement tells you whether your microphone, buffering, endpointing, and backend preserve the caller's correction. Test the whole path.
Build clips that distinguish hesitation from interruption
Record a small set of consented, production-shaped clips. Ten carefully labeled calls will teach you more than a hundred random transcripts with no expected outcome. Use the phrases your customers actually say, then vary timing, accent, device, and noise. Keep a clean reference recording so a provider update can be compared against the same input.
A useful starter set has four timing patterns:
- A deliberate takeover. The bot starts explaining a return policy. At a fixed point, the caller says, "Stop. I need the warranty terms." The answer must stop and switch topic. A brief acknowledgment before the new answer is fine; resuming the return-policy sentence is not.
- A natural pause. The caller says, "My delivery should arrive on..." pauses while checking a calendar, then says, "Thursday." The bot must not commit a date or launch a delivery change on the partial phrase.
- A short acknowledgment. The caller says "right" while the bot reads two steps. If "right" means "I am listening" in context, cutting off the second step is a false barge-in. Repeat with "no, stop" so the test also proves that the system can distinguish a real correction.
- A competing speaker. A television or nearby colleague says a command-like phrase. The bot should keep the caller's task intact. If speaker identity is uncertain and a private action is involved, it should pause for verification rather than guessing.
Do not tune only against scripted clean speech. A speakerphone can feed the bot's own voice back into its microphone. A customer may start a sentence, cough, and begin again. A call may briefly lose audio packets. Those conditions change both false-interruption and missed-interruption rates. Add them as variants of the same labeled cases so you can see which condition caused a failure.
The voice architecture guide explains the tradeoffs between native speech, cascaded speech-to-text, and hybrid systems. Keep the same interruption corpus when you change architecture. A different stack should earn its place by passing the same customer task, not by sounding better in a fresh demo.
Work through a canceled confirmation
Suppose a caller asks a retailer to refund order RK-4182. The bot starts checking eligibility, then the caller interrupts with "Wait, don't refund it. What's the exchange policy?" The refund lookup is still running. The safe result is an exchange-policy answer and no refund request.
Here is an illustrative event trace. It is a test fixture, not a measurement from a deployed system:
12:00:00.000 caller: "Refund order RK-4182" 12:00:00.780 intent_candidate: refund, turn=14 12:00:00.850 eligibility_lookup_sent: RK-4182, turn=14 12:00:01.100 agent_audio_started: "I can check that ref..." 12:00:01.520 caller_speech_started: "Wait, don't refund it. What's the exchange policy?" 12:00:01.590 agent_audio_stopped 12:00:01.600 turn_14_invalidated 12:00:01.940 eligibility_lookup_result: eligible, turn=14, ignored 12:00:02.050 intent_candidate: exchange_policy, turn=15 12:00:02.410 policy_lookup_sent: exchange, turn=15 12:00:02.790 agent: "You can exchange an eligible item within 30 days..." 12:00:03.900 refund_write_count: 0
The eligibility lookup may finish after the interruption. Its result must carry the old turn ID so the application can ignore it. The exchange-policy answer cannot claim that a refund happened, and the refund write count must stay at zero. A good final reply is insufficient proof if the backend submitted a refund despite the correction.
This is the place to inspect idempotency and cancellation semantics. If a tool request has already made a change, stopping audio cannot undo it. The bot must describe the actual state, ask for permission to correct it if appropriate, and log both effects. The chatbot action verification guide explains why a spoken "done" needs a system receipt. For the interruption test, the receipt also has to identify which caller turn authorized the action.
Score the failure, not the charm of the voice
Have two reviewers label the clips without seeing the model's self-assessment. One listens for conversational behavior. The other inspects events and downstream effects. Disagreements reveal ambiguous expectations that should be settled before a release decision.
Track these measures separately:
missed_barge_in_rate = intentional_interruptions_not_handled / intentional_interruptions false_barge_in_rate = pauses_or_backchannels_treated_as_takeovers / pause_and_backchannel_cases stop_delay_ms = agent_audio_stopped - caller_speech_started stale_action_rate = canceled_turns_with_effective_writes / canceled_turns
Stop delay is a distribution, not one average. Report median and the slowest five percent, and keep a clip for each outlier. State how you detected caller speech. Voice activity detection can mark a cough or background conversation as a start, so compare timing against a human-labeled reference for release testing.
Do not merge the rates into a single "voice quality" score. A low false-barge-in rate achieved by ignoring callers is a poor result. A short stop delay is no comfort if a stale refund or booking request still executes. Show the counts beside percentages, especially in a small test set.
Set the gate according to the task. For a public FAQ, a rare false interruption may mean repeating a sentence. For a booking or account action, any effective write from a canceled turn should block release until the cause is fixed. A team might set a provisional target such as "all six core cases pass on every supported device, no stale writes in 30 corrected-action runs, and review every stop delay above one second." Those numbers are an example policy. Choose your own threshold from user harm, call volume, and measured baseline; do not present it as a model benchmark.
The critical-field transcription test checks whether the system heard the right identifier or date. This test asks whether the right value survived the timing of a correction. Run both when a voice workflow can change customer records.
Diagnose the layer that failed
When a case fails, replay the audio next to the event timeline. A caller hears one awkward moment, but your team needs a narrower diagnosis.
If incoming speech never appeared in the trace, inspect microphone permissions, echo cancellation, codec behavior, and transport gaps. If the model detected speech but playback continued, inspect client audio buffering and the telephony stop command. If playback stopped but the old answer reappeared, check whether a late model or tool result was still allowed to write into the active conversation. If the new answer used the wrong date, compare the captured words, intent revision, and confirmation payload.
Make one change at a time, then rerun the same affected clips and a short unaffected baseline. Changing endpointing sensitivity can fix a missed takeover while creating false interruptions on pauses. Changing a prompt can improve the spoken acknowledgment while leaving stale tool execution untouched. Event evidence tells you whether the fix reached the failure layer.
For a phone deployment, the voice AI support checklist covers job scope and human handoff. Add the interruption cases to its launch rehearsal. Include a real transfer during a slow lookup so the bot cannot finish speaking into a call a human has taken over.
Keep the release corpus alive
Save the exact clips, expected decisions, model and provider versions, endpointing settings, device, network condition, and event traces. Add one redacted case when a customer reports a new interruption pattern. Retain raw audio only under your consent and deletion rules; structured timing events and edited transcripts may be enough for routine regression checks.
Rerun the corpus after changes to the voice model, prompt, audio transport, telephony provider, tool routing, or client playback code. A release can improve answer quality while making the bot harder to interrupt. The operational question is whether the caller can reclaim the floor and whether the old turn has truly lost authority. Prove both before you put the voice chatbot in front of customers.
No credit card required.
