Retrieval-augmented generation is well understood in English. Point the same pipeline at an Arabic corpus and recall quietly collapses — usually without anybody noticing, because the model still produces fluent answers.
Fluent and wrong is the failure mode
This is what makes Arabic retrieval dangerous rather than merely disappointing. A pipeline with poor recall in English tends to produce visibly thin answers. In Arabic, the generation layer is strong enough to write a confident response from whatever weak context it received. The output reads well. It is simply not grounded in the right documents.
So the first thing to build is not the pipeline. It is the evaluation set.
Where recall actually goes
Orthographic variation
The same word is routinely written several ways depending on the author, the decade and the department: hamza forms that may or may not be written, alif maqsura against ya, ta marbuta against ha. Diacritics are usually absent but occasionally present. A lexical index treats each variant as a different token, and a query written one way misses documents written the other.
Normalisation before indexing handles most of this — but it must be applied identically to queries and to documents, and it must be recorded, because it is lossy.
Morphology
Arabic is richly inflected and clitics attach directly to the word: definite articles, prepositions, conjunctions and pronoun suffixes all fuse onto the stem. A single lemma surfaces in dozens of written forms. Off-the-shelf subword tokenisers trained mostly on English split these badly, producing fragments that carry little meaning and inflating token counts by a factor that shows up directly on the invoice.
Dialect against Modern Standard Arabic
Institutional documents are in Modern Standard Arabic. The people querying them frequently are not — they write in Gulf dialect, or in a mixture, or transliterated into Latin script. Embedding models trained predominantly on MSA handle dialectal queries unevenly. This gap is invisible in testing if the test queries were written by the same people who wrote the documents.
Mixed-language corpora
Real Qatari institutional archives are bilingual, often within a single document: an Arabic circular quoting an English contract clause, or a table with bilingual headers. Chunking on character counts cuts across the language boundary and produces fragments that embed poorly in either space.
What works in practice
- Hybrid retrieval. Dense embeddings alone underperform on named entities, reference numbers and legal citations — exactly what institutional queries centre on. Combine with a lexical index over normalised text.
- Normalise consistently, and log it. One normalisation function, applied at index time and query time, versioned alongside the index.
- Chunk on structure, not character count. Article, clause and section boundaries beat a fixed window, and they keep bilingual passages intact.
- Evaluate with real queries in the real register. Collect them from the people who will use the system, in the language and dialect they actually write.
- Re-rank. A cross-encoder over the top candidates recovers a meaningful share of the recall lost upstream.
Measure recall separately from answer quality
Track retrieval and generation as two numbers. If you only score the final answer, a strong generator masks a weak retriever, and you will spend months tuning prompts to fix an indexing problem.
Build the Arabic evaluation set first. Everything downstream is guesswork without it.