Getting Arabic speech translation under 500 milliseconds: what fine-tuning Whisper actually took

Speech translation that arrives two seconds late is not translation. It is subtitling, and the listener is always a beat behind the room.

Qudwah translates Arabic sermons into more than 80 languages and delivers the result as audio in under half a second. Below is where that budget goes, what we had to change to fit inside it, and what still breaks.

Why half a second is the number

In a room, latency is not an abstract metric. The congregation reacts to the imam, and a listener on a delayed feed reacts visibly later. At two seconds that gap is obvious to everyone around them. Below roughly half a second it stops being noticeable, which is why the target was set there rather than at whatever the pipeline happened to produce.

That target then has to be divided. Transcription, translation, speech synthesis and delivery each own a share, and each has to give its share up when it overruns rather than borrowing from the next stage.

Why off-the-shelf transcription was not enough

General speech models handle Modern Standard Arabic acceptably in clean conditions. A sermon is not clean conditions. The delivery has its own cadence, the vocabulary is religious rather than conversational, and the room has the acoustics of a large hall with hundreds of people in it.

Fine-tuning Whisper on that specific material was the step that moved accuracy on the terms that matter. Not on general speech, where the base model was already fine, but on the domain vocabulary where a near-miss changes meaning.

The part that is not a machine-learning problem

Religious terminology tolerates no drift. A general translation model renders Islamic terms loosely, and a loose rendering can change what a passage means. No amount of latency engineering fixes that, and no benchmark score catches it either, because the failure looks fluent.

So the models were trained on Islamic vocabulary specifically, and the output was reviewed by religious scholars before the system went into any mosque. That review is not a launch gate that was passed once. It is the reason we can quote a terminology accuracy figure at all.

One speaker, thousands of listeners, many languages at once

The engineering problem people underestimate is not the translation. It is the fan-out. A single audio source has to reach thousands of concurrent listeners, each on a different language track, over consumer mobile networks, inside buildings that were not designed with connectivity in mind.

That is why the delivery layer is WebRTC rather than anything that buffers for reliability, why Redis carries the state that cannot wait for a database round trip, and why the whole thing runs on Kubernetes: sermon traffic is not evenly distributed. It is a wall of concurrent listeners arriving at a fixed time, then nothing.

What still breaks

Heavy background noise degrades transcription, and a hall full of people is noisy. Overlapping speech is handled poorly, though sermons contain less of it than meetings do. Accents far from the training distribution cost accuracy. And network conditions inside older buildings remain the single most common reason a listener has a worse experience than our own measurements suggest.

We publish those limits because a system deployed in a real room will meet all of them, and a supplier who has not named them in advance has usually not measured them.