The question.
Recognition errors and unnatural pronunciation can compound inside a voice product, especially when speakers switch languages within a sentence.
The study separates recognition, synthesis, and conversation evaluation, using consented audio and held-out speaker groups to avoid misleading results.
The proposed approach.
- Create a consent-aware audio catalog with speaker and recording metadata
- Evaluate recognition by accent, noise, and code-switching pattern
- Compare adaptation recipes with an unchanged baseline
- Review synthesis intelligibility and pronunciation with native speakers
Target architecture.
Consented audioAnnotation and splitsASR / TTS adaptationSpeech metricsHuman listening tests
The difficult parts.
- Scarce diverse labeled speech
- Speaker leakage across train and test splits
- Measuring synthesis quality beyond a single score
How to evaluate it.
- WER and CER by speaker group and acoustic condition
- Pronunciation and intelligibility judgments
- Streaming latency and robustness under mixed-language speech
Planned deliverables.
- A documented speech evaluation protocol
- A bilingual listening-test set
- A reproducible adaptation comparison
Design principles.
Hold out speakers, not only audio clips.
Measure how speech works for the people who will use it.