Google introduced Gemini 3.8 Flash TTS on September 23 with controls for voices, accents, pace, and line-by-line delivery. Earlier this month, OpenAI released GPT-Live-1 with prompt controls for tone, pace, and style. Voice teams can shape speech more precisely, which also means they own more ways for it to go wrong.
Start with a pronunciation sheet. Do not wait for a general voice demo to expose the words your customers actually need.
| Test class | Include | Pass condition |
|---|---|---|
| Brand and product names | Company name, plans, products, founders, partners | Approved pronunciation on the first attempt |
| Numbers and money | Prices, dates, percentages, version numbers | Correct units, separators, and currency |
| Customer directions | URLs, email addresses, phone numbers, menu labels | Listener can repeat or follow the instruction |
| Support vocabulary | SKUs, order prefixes, acronyms, technical terms | Correct expansion or letter-by-letter reading |
| Multilingual terms | Local place names, borrowed words, code switches | Native reviewer accepts pronunciation and stress |
Record the expected spoken form, the exact input text, the voice, locale, model version, and rendered audio for every case. That sheet becomes the regression set for voice changes.
Test speech output, not the script
A text review cannot approve a spoken answer. The sentence may be accurate on screen while the audio changes its meaning.
Consider Agentkit Pro costs $119.99 per month. A voice can read the price as one hundred nineteen dollars and ninety-nine cents, one-one-nine point nine-nine dollars, or even eleven thousand nine hundred ninety-nine dollars if normalization fails. All three outputs came from the same characters. Only one belongs in a customer call.
The same problem appears with names. A speech model may turn an unfamiliar product name into a familiar word, pronounce an acronym as a word when the business spells it out, or change stress when the sentence switches languages. Listeners cannot inspect the spelling while the bot keeps talking.
The W3C's Speech Synthesis Markup Language specification separates text normalization, text-to-phoneme conversion, and prosody because each stage can change the final audio. It also defines controls for phonemes, substitutions, language, emphasis, breaks, and speaking rate. Your provider may expose those controls through SSML, a pronunciation dictionary, a prompt, or a vendor-specific API. The release test should stay the same even when the control changes.
Build the lexicon from real support language
The best test terms rarely come from the marketing homepage alone. Pull them from the places where customers and support agents already stumble.
Review recent conversations for product names that visitors misspell, acronyms that agents expand, and URLs that need to be read over the phone. Add every public plan name, frequent partner name, location, medication, material, and technical unit. Include words with more than one valid reading, such as read, lead, live, and Polish, only when your support content uses them.
Give each entry one owner-approved spoken form. Plain-English respelling is useful for reviewers, while International Phonetic Alphabet notation or a provider lexicon may be needed by the speech engine.
display_text: AER-7 is available at agentkit.ai/aer7 for €129.99. expected_audio: "A E R seven is available at agent kit dot A I slash A E R seven for one hundred twenty-nine euros and ninety-nine cents." locale: en-IE must_spell: AER must_say_currency: euro allowed_variation: "one hundred and twenty-nine" disallowed: "air seven", "one twenty-nine point nine-nine", "dollars"
This contract is stricter than a script and looser than a byte-for-byte audio comparison. It names the parts that carry meaning and permits harmless regional variation.
Keep the source lexicon under version control or in another system that records an owner, review date, and change history. A pronunciation fix without a saved test is likely to disappear during the next model or voice migration.
Render the combinations customers will hear
Do not test one voice in one locale and assume the rest will follow. A pronunciation override can work in US English and fail in Irish English. The same voice can handle a product name correctly in a short sentence, then change it after a number or language switch.
Use a small matrix first:
| Dimension | Minimum release sample |
|---|---|
| Voice | Every production voice plus the fallback |
| Locale | Every supported locale with meaningful traffic |
| Context | Term alone, mid-sentence, after a number, before punctuation |
| Delivery | Neutral answer, confirmation, warning, fast list |
| Channel | Browser audio, mobile speaker, phone codec if used |
Generate all combinations automatically when the provider permits it, but keep human listening in the gate. Audio similarity can catch a large change. Speech recognition can flag whether another system heard the right words. Neither proves that a customer heard the intended brand term clearly.
Use at least two listeners for priority terms. One should know the product, and one should not. The product expert catches violations of the approved form. The unfamiliar listener reveals whether the audio communicates enough information without prior knowledge.
If you are still choosing between native speech and a separate text-to-speech layer, the voice chatbot architecture scorecard explains which boundaries you can inspect and replace. The pronunciation corpus should survive either choice.
Score meaning before pleasantness
Naturalness scores are tempting because they produce one tidy number. They can hide the worst support errors. A beautiful voice that says the wrong price has failed.
Score each clip in this order:
- Did the listener identify the exact term or value?
- Did the pronunciation match the approved form?
- Was the phrase understandable on the first play?
- Did stress, pace, or emotion distort the support meaning?
- Did the voice remain consistent with nearby turns?
For critical items, use exact match. Ten listeners hearing a plan name correctly and nine hearing its price correctly gives the price a 90% comprehension rate, not a vague average for the clip. If the price is a purchase condition, 90% should block release.
Pronunciation accuracy and voice style belong on separate lines. A warm, expressive performance can still misread an acronym. A flat but correct reading may pass a policy announcement and fail a consumer onboarding experience. Combining those judgments too early makes fixes harder to assign.
Work through one failed clip
Suppose the test sentence is:
"Your AER-7 renewal is €129.99. Review it at agentkit.ai/aer7."
The first render sounds like this:
"Your air seven renewal is one twenty-nine point ninety-nine dollars. Review it at agent kit aye slash air seven."
The voice sounds polished, but the clip has four defects. It treats AER as a word, changes euros to dollars, uses an ambiguous decimal reading, and drops .ai from the URL.
Fix each layer separately. Add an approved pronunciation for AER, mark or rewrite the currency into spoken text, make the URL speakable, and slow the confirmation sentence without slowing the whole call. Then render the case again with the production locale and phone codec.
"Your A E R seven renewal is one hundred twenty-nine euros and ninety-nine cents. Review it at agent kit dot A I slash A E R seven."
Now ask an unfamiliar listener to write down the product, price, currency, and URL. Four exact fields produce a useful result. "Sounds better" does not.
When the input comes from generated chatbot text, save both the raw answer and the speech-ready form. That boundary shows whether the language model wrote an ambiguous value or the speech layer normalized a clear one incorrectly.
Test code switching and local names with native reviewers
Google says its new TTS models cover more than 100 languages and dialects, while its production voice library includes regional varieties. Broad coverage does not approve your particular mix of names, locales, and sentences.
A French support answer may contain an English product name. A Spanish sentence may include a Polish surname. A customer may ask for a street whose local pronunciation conflicts with the spelling rules of the selected voice. Build these switches into complete phrases instead of testing isolated words.
Use native reviewers for the sentence's main language and, when possible, for the borrowed term. Ask whether the output is understandable, respectful, and locally plausible. Record accepted variants so reviewers do not repeatedly debate a harmless accent difference.
The multilingual chatbot support guide covers routing, source parity, and handoffs across languages. Pronunciation testing adds the audio evidence that translated text alone cannot provide.
Keep text and speech contracts separate
Your written chatbot brand voice still defines vocabulary, sentence length, warmth, and support boundaries. Spoken delivery adds pronunciation, pace, pauses, emphasis, and turn timing. Store the shared rules once, then maintain channel-specific tests for what only speech can express.
Do not solve every pronunciation problem by rewriting the visible answer. A phonetic spelling that helps the voice may look wrong in a transcript, email, or chat widget. Prefer a speech-only representation when the platform supports one. If it does not, document the compromise and test both the audio and displayed transcript.
Input accuracy also needs its own gate. The voice transcription test plan checks whether the system heard the customer's names, IDs, dates, and corrections. This pronunciation test checks what the system says back. A voice workflow needs both directions to pass.
Make every voice update rerun the corpus
Rerun the pronunciation set when you change the speech model, voice, locale, prompt, lexicon, text normalization, audio codec, or response template. Add any production correction to the corpus after removing personal data.
Track the model and voice version with each audio file. A provider can improve general speech while changing a term your business already approved. The saved clip makes that regression audible and gives the team a concrete comparison.
Voice quality becomes manageable when the test uses the words that carry customer meaning. Approve names, numbers, directions, and local language before debating whether a voice sounds charming. A clean script is only the input. The audio is the product customers hear.
No credit card required.
