Three techniques get proposed for almost every LLM problem, and they solve genuinely different failure modes. Choosing wrongly is expensive - not because the wrong one fails loudly, but because it half-works, which is worse. The distinction is easier to hold if you frame each one by the deficiency it corrects.
Prompting corrects a model that does not know what you want. If the model has the knowledge and the capability but produces the wrong shape, tone, or reasoning path, the fix is in the instruction. This includes structured output formats, worked examples, and explicit reasoning steps. It is the cheapest option, it is iterable in minutes, and a surprising number of problems presented to us as needing fine-tuning turn out to be prompt problems wearing a disguise. Always exhaust this first, because it costs almost nothing to find out.
Retrieval-augmented generation corrects a model that does not know your facts. No amount of prompting will make a model recall a policy document it has never seen, and no amount of fine-tuning will keep it current with a knowledge base that changes weekly. If the failure looks like the model confidently inventing details about your business, your products, or your customers, the answer is retrieval - fetch the relevant material at request time and give it to the model alongside the question. Crucially, retrieval also gives you citations, which is often the requirement that actually matters to the business.
Fine-tuning corrects a model that does not behave the way you need it to. Not knowledge - behaviour. Consistent adherence to a niche output format, a specialised classification task where you have thousands of labelled examples, a domain vocabulary the model handles awkwardly, or latency and cost pressure that make a smaller tuned model preferable to a large general one. Fine-tuning teaches a pattern; it is a poor and expensive way to teach facts, because every fact change means retraining.
The practical decision procedure, in order. Can a better prompt fix it - have you actually tried few-shot examples and explicit output schemas? If the failure involves information the model could not possibly know, use retrieval. If the failure persists with correct information in context and a well-constructed prompt, and you have hundreds to thousands of good examples of the behaviour you want, consider fine-tuning. Most production systems we build end up as retrieval plus careful prompting; fine-tuning appears in a minority, usually for classification or for cost reasons rather than quality.
One caution about fine-tuning specifically: it locks you to a model version. Providers deprecate models, and a tuned model must be retrained against the successor, revalidated, and redeployed - a maintenance commitment that lasts as long as the feature does. Retrieval and prompting mostly carry across model upgrades and often improve for free when the underlying model does.
Whichever combination you land on, the thing that determines whether it works is not on this list: it is having an evaluation set of real inputs with known-correct outputs, built before you start. Without it you cannot tell whether a change improved the system or merely altered it, and every subsequent decision becomes a matter of impression. Teams that build the eval set first move faster than teams that build it after the first disappointing demo, which is when most teams build it.
