What is fine-tuning of a model?
Further training a pretrained model on your own examples: excellent for style, format and recurring tasks, wrong for facts that change daily.
Fine-tuning is the process of taking an already pretrained language model and continuing to train it on a smaller set of examples specific to your business: already resolved tickets, already classified contracts, answers in your tone of voice. The result is a model that behaves the way you want on that task, without building a new system from scratch. In practice you prepare hundreds or thousands of input-output examples representative of the task and update the model for a few epochs at a low learning rate, so it does not erase the general capabilities it already has; the result is a new artifact to version and evaluate every time the source dataset changes, so it is a recurring cost, not a one-off setup step. It works well when the problem is style, format or classification, but needs care when the facts to know change often, because in that case the model still needs retraining to stay current.
See also: LoRA · A fine-tuning technique that trains small extra matrices instead of the whole model: comparable practical results, a fraction of the cost.
How it works
You prepare a dataset of input-output examples representative of the task (hundreds or thousands of cases, not millions) and update the model for a few epochs on that data, usually with a low learning rate so it does not overwrite the general capabilities it already has. The resulting model is a new artifact, to be versioned, evaluated and redeployed every time the source dataset changes: it is a recurring cost, not a one-off configuration step.
Why it matters for your business
Fine-tuning wins when the problem is style, format or classification: making the model always answer in the same tone, producing output in a fixed schema, teaching it to sort requests into categories you define. It loses when the problem is fresh knowledge: if facts change every week (prices, availability, policy), the model still needs retraining to stay current, while RAG retrieves those facts at question time without touching the weights. In most enterprise cases RAG solves the problem with less maintenance; fine-tuning remains the right choice when you need a stable, repeatable behavior that no prompt can reliably guarantee.
Related terms
- LoRA · A fine-tuning technique that trains small extra matrices instead of the whole model: comparable practical results, a fraction of the cost.
- LLM · An AI model trained on huge amounts of text that understands and generates language: the engine behind ChatGPT, Claude and Gemini.
- SLM · A small, specialized language model: a fraction of an LLM's cost, runs even on-premise, and for focused tasks it is plenty.
- Distillation · Training a small model to imitate a large one's answers: it inherits much of its ability at a fraction of the cost.
- Domain-Specific Language Model (DSLM) · A language model specialized on a vertical domain (legal, healthcare, finance): more accurate and compliant than a general-purpose LLM on its own turf.