Cost
OpenAI's accuracy guide gives efficiency as one reason to fine-tune: the same accuracy for fewer tokens, or a smaller model. A retrieval step puts source text in the prompt instead. This card does not add a price.
Map · Ship & Maintain · 09 · Fine-tuning vs prompting vs RAG
Concept card · Ship & Maintain
Fine-tuning vs prompting vs RAG
Fine-tuning versus prompting versus RAG is the choice among changing the prompt, retrieving source text, and training a model update.
Choosing among a prompt change, retrieval, and a trained update.
What changed
Oct 18: First card. The picture switches among a prompt edit, a retrieved source, and a trained update. Sources are OpenAI's accuracy guide and Anthropic's glossary.
1 What it is
OpenAI's guide to optimizing LLM accuracy says prompt engineering, retrieval-augmented generation, and fine-tuning are levers that solve different problems, and that you pick the lever after you see why the eval failed. It describes RAG as retrieving content to put in the prompt before the model answers, so the model can use domain-specific context. It describes fine-tuning as further training, either to perform a task more accurately or to reach the same accuracy with fewer tokens or a smaller model. Anthropic's glossary defines fine-tuning as further training on additional data so the model mimics that dataset, and defines retrieval-augmented generation as passing retrieved documents into the context window at runtime. The glossary also says the Claude API does not currently offer fine-tuning. A prompt change or a retrieval change does not need a training path. A trained update needs a path you actually have, and an eval set either way.
Which failure you have
OpenAI's accuracy guide says to diagnose the failure before you pick a lever. If the fact was not in the request, retrieve it or put the source in the prompt. If the fact was there and the format keeps slipping, change the prompt, and compare a trained update only where you can train and then beat that prompt on the eval set.
2 Why it matters
Quality
OpenAI's accuracy guide says many large deployments used prompt engineering and RAG, and that fine-tuning does not replace retrieval when the failure is missing context. It also says non-representative training examples are a common pitfall: the examples have to look like what production sends.
Risk
OpenAI's accuracy guide says non-representative training data is a common pitfall: the examples have to look like what production sends. It also says to keep a hold-out set and evaluate after training. Ship the prompt or retrieval change the eval set can already grade before you plan a trained update.
3 How to apply · try today
Take five failed examples. Mark each one "the fact was missing" or "the fact was there and the format was wrong." Missing facts go to retrieval or a prompt that includes the source. Format errors stay on the prompt until a training path you can use beats that prompt on the eval set.
Common mistake
Fine-tuning so the model learns the documents. OpenAI's accuracy guide says retrieval is how you give the model context it does not already have. Fine-tuning is for behavior you can show with examples.
Self-check
The close time is wrong because the document was never in the prompt. Which lever comes first?
Coming soon. Five failures marked missing-fact or wrong-format, and the lever you would test first on each.
Coming soon. How OpenAI's accuracy guide separates context you retrieve from behavior you train, and how Anthropic's glossary defines both.
Questions
- What is fine-tuning versus prompting versus RAG?
- Fine-tuning versus prompting versus RAG is the choice among changing the prompt, retrieving source text, and training a model update.
- Does fine-tuning teach the model your documents?
- OpenAI's accuracy guide says retrieval supplies domain context in the prompt. Fine-tuning is further training so a task is more consistent or uses fewer tokens. A missing document is a retrieval problem.
- Does every API offer fine-tuning?
- No. Anthropic's glossary says the Claude API does not currently offer fine-tuning. OpenAI's accuracy guide says a training run depends on the platform or framework you use. Prompt and retrieval changes do not need a training path.