Arabic Speech-to-Text: Accurate Transcription for Code-Switched, Multi-Dialect Speech

When natural code-switching is processed through a standard Arabic speech recognition engine, the error rate routinely exceeds 30%, sometimes going above 35% if you take into account the combination of dialect and English words in a single sentence. This is no mere rounding error; it shows that the system is unable to reliably transcribe the way people in the Middle East actually speak. 

Bilingual speakers tend to mix languages within a single breath. Traditional speech recognition systems separate the different languages and break down at the point where they meet. Hence Alif was designed to eliminate that problem.

The Problem with Transcribing Arabic Speech

In the Middle East, it is customary for people to switch between Arabic and English during a sentence when mixing in vocabulary, grammar and expressions, and to correctly identify this practice, one must:

• Handling dynamic vocabulary boundaries that shift language mid-utterance
• Tracking fluid phonetic shifts as a speaker crosses from Arabic to English sounds
• Resolving contextual ambiguity across two distinct linguistic systems at once

Most Arabic voice-to-text systems were not designed for this kind of usage. They assume that each utterance should be in one language, follow one grammar, and have one acoustic profile, and code-switched speech breaches all of these assumptions at the same time.

Why Most Speech Recognition Models Fail on Arabic

The real problem is data fragmentation. The Arab world includes over twenty-five countries, among them dialects such as Gulf (Khaleeji), Levantine, Egyptian, and others, which differ from Modern Standard Arabic (MSA) in terms of phonology, vocabulary, and syntax. The majority of current speech models are based on one of two very limited types of data:

• Pristine news broadcasts, such as the MGB-2 corpus, which reflect formal MSA, not street-level dialect
• Isolated monolingual corpora, which fail the moment real-world audio mixes in English technical terms

Very little industry or academic work has been done to combine these various sources into one model. As a result, the ASR market performs well in demonstrations but fails considerably when tested on actual operational audio.

Alif: One Model, Every Dialect, Both Languages

Alif v1 is a unified, end-to-end speech-to-text model which has been specifically designed to transcribe speech that code-switches between Arabic and English and, at the same time, supports a variety of regional dialects, ranging from Gulf and Levantine to Egyptian, all within a single shared representation. This model builds on the code-switching method used in Akshara and Ragini, Convozen’s speech models for Indian languages, since it has been shown that in order to capture mixed-language speech it is necessary to model the transitions between languages directly within one neural representation rather than assembling separate language pipelines.

Architecturally, Alif uses a unified encoder-decoder with a shared multilingual vocabulary tokenization system:

Instead of switching between separate language models halfway through each utterance, Alif decodes both languages in a common latent space and thus tracks the transitions between languages as joint acoustic-textual probabilities, which reduces the boundary errors that occur whenever a conventional pipeline passes from one language to another.

Built from Real, Fragmented Arabic Speech Data

Alif overcomes MENA’s data fragmentation through a multi-source acoustic and textual normalization pipeline, drawing on:

• Broadcast media (MGB-2)
• Studio recordings (MASC Clean)
• Noisy operational environments (MASC Noisy)
• Gulf colloquial speech (SADA)
• North African dialects (Casablanca)

It is only by sourcing and curating data from these various corporations, rather than by training on one single clean dataset, that Alif is able to cope with the noisy, dialect-mixed, code-switched audio that enterprise voice systems in fact come across.

Arabic Speech-to-Text Benchmark Results

The Alif v1 was tested on custom code-switched speech test sets together with the leading public and commercial speech engines.

Alif achieves the lowest overall average WER in the group at 23.88%, representing a relative improvement of 35.2% over Whisper Large v3, and performs particularly well on both studio and noisy operational audio (12.23% on MASC Clean and 24.43% on MASC Noisy). Scribe v2 from ElevenLabs does better on the Casablanca and CommonVoice datasets. Alif also shows good stability in the regional dialect tests, for example with a WER of 27.89% on Gulf SADA speech.

Real-Time Transcription for Voice Agents and Live Calls

Only if it is fast enough to be suitable for a live conversation will high accuracy across dialects be useful. Alif is designed for real-time streaming with a time-to-first-token (TTFT) of less than 120ms, achieving this by using chunked streaming attention and dynamic prefix decoding so that it can continuously process live audio frames without having to wait for complete silence boundaries before transcribing.

Where Arabic Speech-to-Text Is Used

• Interactive voice agents and automated customer support bots
• IVR telephony systems handling Arabic-English customer calls
• Live bilingual meeting transcription
• Contact centre QA and compliance review across Gulf, Levantine, and Egyptian dialects

Alif transcribes Arabic the way it’s actually spoken, across dialects, across languages, in real time. With the lowest WER among tested engines and sub-120ms latency, it’s built for production voice agents, not demos.

Ready to see Alif in action? Book a demo with our team today.