Why we chose retrieval over a bigger model for legal search across three million Gulf documents

General-purpose language models answer questions about Gulf law readily enough. What they cannot do is tell you which article the answer came from, and an answer a lawyer cannot cite is an answer they cannot use.

We built Sylos, a legal research system over more than three million legal documents from the United Arab Emirates, Saudi Arabia, Qatar and Bahrain. Below is why the architecture looks the way it does, including the parts that did not work.

Why retrieval rather than a bigger model

The obvious approach is to take a strong model and fine-tune it on legal text until it knows the law. We did not do that, for a reason that has nothing to do with cost.

A model that has absorbed the law produces answers from memory. Those answers are often right, and when they are wrong they are wrong in the same confident register as when they are right. A lawyer cannot tell the two apart without checking, and if they have to check everything, the system has saved them nothing.

Retrieval inverts that. The system finds the passages first and answers from them, so the citation is not decoration added afterwards, it is the thing the answer was built out of. The user can follow it back to the official text in one click. That property is worth more than a few points of fluency.

Why hybrid retrieval, not vectors alone

Dense vector search finds material that means the same thing in different words, which is what you want when a lawyer describes a situation rather than naming a statute. It also quietly fails on the thing legal researchers do most: looking for an exact instrument.

Ask a vector index for “Federal Law No. 2 of 2015” and it will return things about commercial companies. Useful, but not the article you asked for. Lexical search finds the string. So the system runs both, over FAISS for meaning and Elasticsearch for the exact match, and PostgreSQL holds the structure – jurisdiction, instrument type, date – so filtering happens before ranking rather than after it.

Chunking follows the document’s own structure rather than a fixed token count. A provision cut in half retrieves as two half-wrong answers, and in legal text the second half is often the part that carries the condition.

Why we fine-tuned AraBERT instead of using a multilingual model

This is the decision we tested hardest, because it is the expensive one. General multilingual embeddings handle Modern Standard Arabic reasonably. They handle legal Arabic considerably less well, and the failure is not random.

Legal Arabic uses terms whose meaning is fixed by statute rather than by usage. Two words a general model treats as synonyms can carry different legal effects, and a retrieval model that collapses them returns the wrong provision while looking confident about it. Fine-tuning AraBERT on the legal corpus itself, rather than on the open web, was what closed that gap. The decision came out of measurement on our own evaluation set, not out of preference for the harder path.

The part most systems skip: saying nothing

Every answer is checked against the retrieved passages before it reaches the user. Where no passage supports it, the system says it has nothing rather than producing fluent text.

That is a feature we shipped deliberately, and it is unpopular in demonstrations, because a system that sometimes declines looks weaker than one that always answers. In legal work the opposite is true. A tool that answers everything is worse than one that answers less, because the user cannot tell which kind of answer they are holding. We would rather return three cited paragraphs and an honest gap than five paragraphs and a filing that does not survive contact with the court.

What it still does not do well

Coverage is not uniform across the four jurisdictions, and the system is only as current as the last indexing run, which matters in a region where instruments are amended often. Questions that turn on the interaction between two jurisdictions are handled less well than questions inside one. And no retrieval system removes the need for a lawyer to read the provision it points at.

Sylos is public. The fastest way to judge any of this is to ask it something you already know the answer to, and see whether the citation holds up.