Measuring how AI understands the languages, dialects and voices of the Maghreb.
Benchmarks, datasets and evaluation methodology for Algerian Darija, Maghrebi Arabic, Tamazight varieties and North African French — the languages most models were not trained to handle.
Demonstration material. The benchmark published here is a demonstration edition. Where a figure appears anywhere in the Lab, it carries its sample size and how it was produced — and where no verified run exists, we say so rather than showing a number.
Benchmarks
Structured item sets for Maghrebi varieties, versioned, with the answer keys kept private and the methodology published.
0 published
Open benchmarks →Speech
Read, spontaneous and conversational recordings with regional metadata — and samples you can actually listen to.
0 datasets listed
Open speech →Languages
What each variety is, why it is hard for AI, and what we can collect and evaluate for it.
0 profiles
Open languages →Why this happens
Four reasons systems fail here
“Arabic” is treated as one language
Most pipelines collapse Modern Standard Arabic, Egyptian, Levantine, Gulf and Maghrebi into a single label. Algerian Darija differs from MSA in vocabulary, negation, numerals and time expressions — the exact places where errors become costly.
The Maghreb writes in several scripts at once
The same sentence can appear in Arabic script, Latin-script Arabizi (3, 7, 9 for ع، ح، ق) or French orthography. Tamazight adds Tifinagh. Tokenisers and language detectors rarely handle this gracefully.
Code-switching is the default, not the exception
Urban Algerians mix Darija, French and MSA within a single sentence. Utterance-level language identification cannot represent that, so half the meaning is discarded before the model reasons about it.
Local reality is thinly documented online
Administrative procedures, institutions, prices, place names and everyday practice are under-represented on the web. Where the training data is thin, models produce confident, plausible, incorrect answers.
Evaluate your AI in Algerian Darija
Tell us what you have built and what you need to know about it. We will come back with a proposed evaluation design, the number of native evaluators it needs, and what it would cost.