Consider an ordinary Urdu voice message. A speaker begins a sentence, pauses to find a word, switches briefly to English, then starts again. A fan turns overhead. The phone is held a little farther from the mouth than it was a moment ago. Someone else speaks from the next room.
There is nothing remarkable about this imagined recording. Its ordinariness is the point. A speech system intended for everyday use must encounter people speaking in circumstances that researchers cannot entirely arrange.
Artificially generated speech offers researchers a useful shortcut. Written sentences can become recordings without asking someone to read every line. The resulting collection can support experiments that would otherwise be difficult to begin. But a model doing well on generated voices leaves a question unanswered: how will it respond when the speaker is a person it has never heard?
Why researchers generate voices
A speech dataset needs more than audio. Depending on the task, each recording may require a checked transcript, a description of its contents or labels marking particular words. Collecting recordings, securing permission, and checking those labels take work. Speech synthesis can supply additional examples, especially when researchers already have useful text.
For Urdu research, this is a practical opportunity. UrHiOdSynth, presented at a low-resource language workshop in 2026, describes approximately 75 hours of parallel speech across Urdu, Hindi, and Odia. Its authors propose uses ranging from speech translation to recognition and synthesis. The resource illustrates how researchers are trying to make limited data go further. [1]
Synthetic training data can also improve measured performance. In a 2025 study, Sewade Ogun and colleagues varied characteristics of generated speech and evaluated speech recognisers on English datasets. Their experiments found benefits from carefully combined augmentation, while some individual changes offered little help. Generating additional audio was useful, but the way it was generated mattered. [2]
That distinction is easy to lose when a dataset is described mainly by its size. Ten thousand recordings may contain many sentences while drawing on a narrow range of voices or speaking styles. Counting files does not tell us how much of a language’s spoken variation they represent.
A convincing voice and a useful test
Two questions often sit close together in speech research. Does generated audio sound natural? And does it provide a useful basis for learning how to handle human speech? A listener can help answer the first. The second requires a different kind of evidence.
A study by Dareen Alharthi and colleagues explored this distinction by training speech recognisers on synthetic recordings and evaluating them on real speech. The authors proposed using recognition errors on human recordings to assess differences between the generated and real data, extending evaluation beyond intelligibility alone. [3]
A school analogy helps. A student may perform well on practice questions written in a familiar style. That result is encouraging, but a different examination is needed to discover whether the student can apply the same knowledge to unfamiliar questions. Synthetic recordings can be valuable practice material. Testing with human speech examines what carries over.
Nor does every human recording establish everyday readiness. People reading prepared sentences in a quiet room still differ from people composing spontaneous voice messages. Both collections may be useful; they support different claims. A careful article, like a careful research paper, needs to describe which one was actually tested.
What Urdu safety research can reveal
My involvement in MUTEX-M, a text-and-speech extension of Urdu safety research, brings this question into a particular setting. The work asks how language technology can identify harmful content using text and speech. Its results need to be interpreted within that task.
Our earlier MUTEX preprint concerns toxic span detection in written Urdu. Instead of assigning only one label to an entire passage, it seeks to identify the specific words responsible for toxicity. The later MUTEX-M preprint, Advancing Urdu Toxicity Detection: Improved Span Models and a Multimodal Audio Extension, extends this work to speech. [4, 5]
This matters because speech technology covers several different problems. A recogniser attempts to transcribe words. A safety classifier attempts to judge content. A span detector attempts to locate particular harmful words. A result for one of these tasks cannot be treated as a general score for understanding Urdu.
In that September preprint, we report 83.2% weighted F1 on synthetic speech and 77.1% on 2,000 human-recorded clips. The 6.1-percentage-point difference concerns whole-recording toxicity classification, not exact toxic-word boundaries. F1 balances identifying harmful content with avoiding incorrect flags; the weighted average accounts for class frequency. [5]
The model combines an audio classifier with a text module applied to speech transcripts. The human evaluation includes four regional speech varieties, with its configuration fixed using synthetic validation data. This tests transfer beyond the generated recordings, while remaining evidence about one system and task. The findings are reported in a preprint. [5]
The difference does not isolate accent as its cause. Content and recording conditions also differ between the datasets. The comparison shows a gap between these evaluation settings; more controlled experiments are needed to separate the contributing factors.
The variation beyond the script
Take a simple illustrative request: “Mera balance check kar dein”, meaning “Please check my balance.” It combines Urdu with English words. One person might say it slowly; another might shorten a sound, hesitate or emphasise a different word. A prepared script fixes the sentence, but it does not fix every way that sentence can be spoken.
An Urdu evaluation should therefore ask which speaking patterns it includes and which remain untested. Speakers’ other languages, their regional backgrounds and the circumstances of recording are relevant questions for data collection. They should not become explanations for a failure merely because they seem plausible.
Recording conditions deserve the same care. The 2025 augmentation study examined environmental variation alongside linguistic and acoustic characteristics. Its broader lesson for this feature is methodological: evaluate the effects of changes rather than assuming that all additional recordings contribute equally. The study’s numerical improvements should remain attached to its English recognition experiments. [2]
Speakers can also tell us what a technical failure costs in practice. Did they repeat a request, type it instead or ask another person to help? Did they change language? Those accounts can identify consequences that a benchmark score leaves out. They provide individual experiences rather than a population estimate.
A test that deserves our confidence
A stronger evaluation begins with a precise claim. If the claim concerns new speakers, those speakers should be kept separate from the material used to train and tune the model. If it concerns phone recordings, the evaluation should include documented phone recordings. Broader claims require broader tests.
Where feasible, researchers can ask people to record the same sentences used for synthesis. Holding the wording constant helps investigate differences in the audio, although reading a script still leaves spontaneous conversation to be tested separately. Each experiment answers a narrower question and makes the remaining uncertainty easier to see.
Results also need enough detail to be interpreted. An overall score should be accompanied by the task definition, the number of examples and the evaluation procedure. Comparisons across speaker groups need adequate samples and uncertainty estimates. A small group should not acquire a sweeping reputation because of a handful of difficult recordings.
For safety research, it is useful to examine the errors themselves. Missing harmful content and flagging harmless speech have different consequences. A system intended to assist moderation needs evidence about both, along with a clear account of when a person should review its output. A headline score alone cannot settle that decision.
Listening beyond the demonstration
Synthetic speech has a place in building Urdu language technology. It can help researchers develop experiments and explore questions that scarce recordings make difficult to study. The responsibility comes when the evidence is translated into a claim about people.
Return to that imagined voice message, with its hesitation and background fan. It is easy to prepare a cleaner version for a demonstration. The harder, more useful question is what the system can do with the message as it is.
For the speaker, the relevant outcome is simple: can they finish the task in their own words? Evaluations grounded in real recordings bring that outcome closer to the centre of research. The promise of synthetic speech becomes more credible when we are willing to listen beyond it.
References
- Jamaluddin, Subhankar Panda, Aditya Narendra, Kamanksha Prasad Dubey and Mohammad Nadeem. 2026. UrHiOdSynth: A Multilingual Synthetic Corpus for Speech-to-Speech Translation in Low-Resource Indic Languages. Proceedings of LoResLM 2026, pp. 584–594. https://aclanthology.org/2026.loreslm-1.50/
- Sewade Ogun, Vincent Colotte and Emmanuel Vincent. 2025. An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASR. arXiv version reviewed: 2503.08954v1. https://arxiv.org/html/2503.08954v1
- Dareen Alharthi, Roshan Sharma, Hira Dhamyal, Soumi Maiti, Bhiksha Raj and Rita Singh. 2023. Evaluating Speech Synthesis by Training Recognizers on Synthetic Speech. arXiv version reviewed: 2310.00706v1. https://arxiv.org/html/2310.00706v1
- Inayat Arshad, Fajar Saleem and Ijaz Hussain. 2026. MUTEX: Leveraging Multilingual Transformers and Conditional Random Fields for Enhanced Urdu Toxic Span Detection. Public text-based preprint, 2603.05057v1. https://arxiv.org/abs/2603.05057
- Fajar Saleem, Inayat Arshad and Ijaz Hussain. 2026. Advancing Urdu Toxicity Detection: Improved Span Models and a Multimodal Audio Extension. September preprint. https://doi.org/10.13140/RG.2.2.31745.62567
Also Read: Smart Machines, Stronger Teams: AI at the FIFA World Cup
Inayat Arshad is an AI researcher and engineer based in Islamabad and Head of Applied AI at TechGIS. She holds a BS in Computer and Information Sciences from PIEAS. Her work spans Urdu language resources, multilingual AI safety, and explainable machine learning, with a focus on making AI systems more trustworthy. She writes to help people beyond the technology industry understand how AI works, where it falls short and how it affects their everyday lives.
