AI & automationJune 20266 min read
Enterprise RAG: where to start
RAG projects that go off the rails almost all share one thing: someone rushed to the model and forgot the data. The quality of the answer is decided first by what you retrieve, not by the model that writes it.
Where do I start a RAG project without getting it wrong?
With the data, not the model. The principle of a RAG: I retrieve the right passages from your documents, then ask the model to answer from them. In 70 to 90% of cases, the quality of the answer is decided by what you retrieve, not by the model that writes it. So you start by preparing the corpus, not by choosing the model.
Why clean the documents before plugging in the model?
Because a dirty corpus produces wrong answers stated with confidence. Poorly scanned PDFs, duplicates, outdated versions: the model does not sort them out, it answers from what it is given. Before anything else, I sort, deduplicate and date. It is thankless, and it is what decides reliability.
How do you chunk the documents well?
By respecting logical units, not cutting every 500 characters. A document split at random loses its meaning: I chunk by section, paragraph or procedure, so each passage stays understandable on its own. That is the difference between a useful passage and a mangled one.
An LLM does not fix bad retrieval. It dresses it up and makes it credible.
Once these foundations are in place, plugging in the model becomes the easy part. I then evaluate on real business questions, and iterate on the indexing, not on the prompt.