RAG or fine-tuning: which one adapts AI to your data?

Two clients asked me the same thing a week apart: 'do we fine-tune, or do we do RAG?'. Both had already picked a side before working out which problem they were trying to solve. That is the real trap in this debate: people treat it as a choice of tool, when it is first a choice of problem.

The starting misunderstanding: these aren't the same problem

RAG and fine-tuning don't compete, they answer two different needs. RAG is about what the model knows: facts, documents, data that moves. Fine-tuning is about how the model behaves: its form, its style, its way of answering. Asking 'RAG or fine-tuning?' without saying which of the two you're missing is like asking 'hammer or screwdriver?' without saying whether you have a nail or a screw.

RAG in one sentence

RAG gives the model memory: I fetch the right passages from your documents at question time, and ask the model to answer from them, with sources. The model doesn't 'know' your data, it consults it on the fly. Done well, the concrete result is a sourced answer in 2 seconds, 70% less searching for your teams. It is the approach I lay out on the RAG page.

Fine-tuning in one sentence

Fine-tuning changes the model's behavior: I retrain it on examples so it adopts a format, a tone or a way of reasoning by default. The model doesn't reliably learn new facts, it learns a way of doing things. It is a habit you give it, not a library you add to it.

The simple test: memory or method?

Before any technical choice, I ask one question: what's missing, memory or method? If the model gives wrong or outdated answers about your facts, that's a memory problem, so RAG. If it knows the facts but answers in the wrong format or the wrong tone, that's a method problem, so possibly fine-tuning.

• Memory (facts that move), RAG: product catalog, internal procedures, pricing, contracts, knowledge base, anything that changes and has to be quoted accurately.
• Method (form, style), fine-tuning: a very precise output format, a constant brand tone, a way of classifying or writing, repeated at very high volume.
• The common confusion: 'the model gets our data wrong' is almost never a method problem. It is memory, so RAG.

RAG fixes what the model knows. Fine-tuning fixes how it behaves. Confusing the two means paying a lot to solve the wrong problem.

Cost, delay, maintenance: the head-to-head

Beyond the principle, the difference shows up on the invoice and the calendar. RAG sets up fast and updates itself: when a document changes, you re-index, and the answer follows. Fine-tuning means preparing a set of examples, running a training job, then redoing it all with every change to the base model. That recurring cost often weighs more than the first.

• RAG: short delay, updates by re-indexing, cost mostly in document preparation. Facts stay fresh with no retraining.
• Fine-tuning: longer delay, an example set to build, training to redo with every model change. Nothing updates itself.
• A runaway bill rarely comes from the RAG-versus-fine-tuning choice itself: it's almost always what you put around it that costs.

Why start with RAG in most cases

Because the most common need is memory: 'the model has to answer accurately about our documents'. RAG answers that faster, cheaper, and fixes itself by changing a document rather than relaunching a training job. It also makes the answer verifiable, since it cites its sources. Before wiring anything up, the real question is the state of your documents: I cover it in where to start with RAG and in is your data ready for AI.

When to combine them (the hybrid)

The two don't exclude each other. The case that justifies it: your facts change all the time (so RAG for memory) and you need a very precise format or tone, at very high volume (so fine-tuning for method). RAG brings the fresh facts, fine-tuning brings the constant form. But it is an optimization, not a starting point: you only add fine-tuning if RAG alone, done well, has shown its limits on form.

And where does prompting fit in?

Often, neither one. Before fine-tuning to get a format or a tone, a good prompt and a few examples are enough nine times out of ten, with no technical debt. My default order: prompt first, add RAG as soon as you need up-to-date facts, and only fine-tune last, if the numbers demand it. I worked through that trade-off in fine-tune or prompt.