RAG vs Fine-Tuning vs Prompt Engineering
Prompt engineering, RAG or fine-tuning? Learn what each one changes, what it costs, and the order to try them when your AI gives poor answers.

If your AI assistant gives poor answers, it's tempting to blame the model. More often, it hasn't been given the right instructions, the right information, or enough examples of what "good" looks like.
There are three main ways to improve an AI model's results: prompt engineering, RAG (retrieval-augmented generation) and fine-tuning. They aren't rivals. Each fixes a different kind of problem, and many products use all three.
Prompt engineering is giving a new employee clear instructions. RAG is handing them the right files. Fine-tuning is sending them on a training course so certain habits become second nature.
The key question is whether you have a knowledge problem (the model doesn't know something) or a behaviour problem (it knows enough, but doesn't respond the way you need).
At a glance
| Prompt engineering | RAG | Fine-tuning | |
|---|---|---|---|
| Fixes | Unclear instructions, format, tone | Missing or out-of-date knowledge | Behaviour a prompt can't reliably produce |
| Changes | The input you send | The information added to the input | The model's weights |
| Effort | Hours | Days to weeks | Weeks, plus a dataset |
| Running cost | Longer prompts cost more tokens | Search infrastructure, extra tokens | Training cost; may then need shorter prompts |
| Updating | Edit the text | Update the documents | Train again |
| Best for | Every project, first | Private, changing or source-backed facts | Consistent style, format or narrow tasks at scale |
Prompt engineering: always start here
Prompt engineering means writing clear, structured instructions: who the AI is acting as, what to do, what to avoid, what format to answer in, and examples of good answers.
Instead of "Write a marketing email", you might give the audience, the goal, the tone, a word limit and two past emails that performed well. Asking without examples is called zero-shot prompting; including a few examples (few-shot prompting) is often the single biggest improvement you can make.
- What it changes: only what you send the model, so it's instant to try and easy to undo.
- Cost: your time, plus slightly longer requests.
- Limits: it can't give the model facts it doesn't have, and very long prompts get expensive and harder to maintain.
Capable modern models handle a large share of everyday business tasks with good prompting alone, so try this before anything else. Read more in What is prompt engineering?
RAG: when the model doesn't know the facts
Suppose your prompt is excellent, but you ask "Summarise the Q3 report we published yesterday" and the model can't, because it has never seen it. That's a knowledge problem.
RAG fixes it by searching your documents for the relevant passages and adding them to the prompt before the model answers. The model then works from your material, and can cite it.
- What it changes: the information in each request, not the model.
- Cost: building and maintaining the search index (usually a vector database), and the extra tokens each request carries.
- Best for: help-centre answers, internal knowledge, policies, product documentation: anything private, changing or needing sources.
- Limits: results are only as good as retrieval. Find the wrong passage and the answer will be wrong.
For live values such as an account balance or stock level, you usually don't search documents at all: the AI calls a system directly through tool calling.
Learn moreWant a deeper dive into RAG?
Read our explainer on how RAG works under the hood, and where it goes wrong.
Fine-tuning: when behaviour must be built in
Now suppose the model has the right facts and a strong prompt, but still doesn't behave the way you need. It drifts from your strict output format, can't hold a specialised house style, or needs a page of instructions on every request to do one narrow task. That's a behaviour problem, and the case for fine-tuning.
Fine-tuning continues training an existing model on your own examples of inputs and ideal outputs. It's a form of transfer learning: the model keeps its general knowledge and adapts it to your task, usually through supervised learning on labelled examples.
What it involves in practice:
- Collect examples: often hundreds to a few thousand high-quality input/output pairs. Teams sometimes add synthetic data generated by a larger model, checked by people.
- Train: run training for a small number of passes over the data (epochs), with settings (hyperparameters) such as the learning rate. Many providers offer this as a managed service.
- Evaluate: compare the tuned model with the original on examples it didn't train on (model evaluation). Watch for overfitting: a model that copies its training examples but handles new cases worse.
- What it changes: the model's weights, so the behaviour sticks without long prompts.
- Cost: preparing data, training compute and ongoing evaluation, and you repeat it whenever you need changes.
- Best for: strict formats, consistent tone, classification or extraction at high volume, or getting a small, cheaper model to do one job well.
Don't fine-tune for factsFine-tuning is a poor way to teach a model facts that change, like prices or policies. It's costly to redo, the model may still mix the facts up, and it can't show where an answer came from. Use RAG for facts and fine-tuning for behaviour.
Which should you try first?
Work through them in order, and only move on when the previous step genuinely isn't enough:
- Improve the prompt. Clear instructions, a defined format, a few examples. Test on real cases.
- Add knowledge with RAG (or tools for live data) if answers fail because information is missing or out of date.
- Fine-tune if, with good prompts and the right information, the model still can't behave consistently, or prompts have become too long and costly at your volume.
At every step, keep a small set of test questions with known good answers, so you can tell whether a change actually helped.
Using them together
A customer support assistant for a bank might use all three:
- Fine-tuning so it reliably follows the bank's tone and its strict format for complaint summaries.
- RAG over policy documents and help articles, so answers about fees and rules are current and cite their source.
- Tool calling to read the customer's live balance or recent transactions from the bank's systems.
- Prompt engineering for the day-to-day instructions: which products to mention, what to escalate to a person.
Frequently asked questions
Is RAG better than fine-tuning?
Neither is better; they solve different problems. RAG adds knowledge that changes; fine-tuning shapes behaviour. Many teams use RAG and never need fine-tuning.
Is fine-tuning expensive?
It costs more than prompting or RAG to set up: you need good examples, training runs and evaluation, and you repeat it when requirements change. Managed services and small models have made it far cheaper than training a model from scratch.
Can prompt engineering replace fine-tuning?
Often, yes, especially with capable models and good examples in the prompt. Fine-tuning earns its place when you need very consistent behaviour at high volume, or want a smaller model to match a bigger one on a narrow task.
Does fine-tuning stop hallucinations?
Not reliably. It can make a model more consistent, but it doesn't give it a source to check. Grounding answers with RAG and asking for citations helps more.
What is the difference between fine-tuning and training a model?
Training builds a model from scratch on huge amounts of data. Fine-tuning starts from an already-trained model and adjusts it with a much smaller, focused dataset.
Summary
- Prompt engineering changes the instructions: start here, always.
- RAG adds missing or changing knowledge from your documents; use tools for live data.
- Fine-tuning changes the model itself to fix behaviour, and needs good examples and evaluation.
- Try them in that order, testing as you go.
- Most real products combine them.

Stephen Gbolagade
Software Engineer and Technical Writer at Ploba
Writes Ploba's guides to AI tools, agents and protocols, explaining how they work in plain language.
Keep learning
Plain-language explanations, technical guides and practical comparisons.

