Field note · Post-training · July 2026
Fine-tuning in 2026: adapt behaviour, not changing facts
Fine-tuning should make stable behaviour more reliable. Current knowledge, permissions and operational truth should remain outside the weights.
Start with clearer instructions, examples and structured outputs.
Use current facts and authoritative actions without retraining.
Teach repeatable behaviour, format and domain language.
Optimise choices when paired feedback is available.
Train against verifiable environments only when the reward is trustworthy.
Start with the behaviour gap
Fine-tuning should address a stable behaviour that prompting, retrieval or deterministic tooling cannot deliver reliably enough. It is a poor way to inject frequently changing facts, enforce permissions or replace a missing workflow. Those belong in retrieval, APIs, policy and application code.
The first design question is not “Which trainer should we use?” It is “What repeatable failure remains after the prompt, context, tools and evaluator are improved?”
Choose the objective that matches the evidence
Supervised fine-tuning teaches a model to imitate high-quality demonstrations. It is useful for output structure, tone, tool-call syntax and domain-specific task patterns. DPO or KTO can shape preferences when examples show what should be preferred or rejected. GRPO and other online methods become relevant when an environment can score outcomes reliably across repeated attempts.
TRL v1 now exposes SFT, DPO, KTO, GRPO, reward modelling and multi-environment agentic RL in one post-training stack. That breadth is useful, but the method should follow the available evidence. A weak reward function makes reinforcement learning optimise the wrong behaviour more efficiently.
Use parameter-efficient tuning as the default experiment
LoRA and related PEFT methods update a small adapter rather than every base-model parameter. This reduces GPU memory, training time and artifact size, making it practical to compare several datasets or task variants before considering a full fine-tune.
My default sequence is: baseline the untuned model, run a LoRA or quantised LoRA experiment, evaluate the adapter against the same held-out suite, then justify any larger training run with measured evidence. Full-model tuning is reserved for cases where adapters cannot reach the target and the organisation can support the larger training, storage and regression surface.
The dataset is the product
A useful training set contains representative tasks, correct tool use, edge cases, refusals, recovery behaviour and difficult counterexamples. It also separates training, validation and final evaluation data by source and time so that duplicated examples do not create a false sense of progress.
For agentic workloads, store more than the final answer. Capture the accepted plan, tool calls, environment state, verifier results and human corrections. Synthetic examples can expand coverage, but they need deterministic checks or human review before entering the trusted training set.
Evaluation must surround training
Compare the base model and every adapter on task success, tool-call validity, unsafe-action rate, refusal quality, latency, token use and regression outside the target domain. Keep safety and general capability suites separate from the optimisation target so the training job cannot quietly trade them away.
Training is complete only when the adapter, prompt template, tokenizer, runtime configuration and evaluator versions form one reproducible release candidate.
Serve adapters as governed artifacts
vLLM can serve LoRA adapters alongside a base model, but its documentation warns that dynamic load and unload APIs are intended for local development. In production, adapters should come from an approved registry with immutable versions, compatibility metadata, signatures and rollback.
Route by tenant or use case only after checking that adapter selection cannot cross data or authority boundaries. Measure the memory and latency impact of concurrent adapters instead of assuming one base model makes every customised workload cheap.
A practical post-training pipeline
- DefineWrite the behaviour contract, baseline metrics and stop conditions.
- CurateVersion demonstrations, preferences, negative cases and provenance.
- TrainBegin with SFT or PEFT, then escalate only when evidence supports it.
- EvaluateRun task, safety, regression, latency and cost suites against the base and candidate.
- PackagePublish the adapter, tokenizer, prompt contract and evidence bundle together.
- ObserveMonitor drift and route uncertain cases back into review, not directly into training.
Executive takeaway
Fine-tuning is a product and governance decision, not a model trick. Use it to make stable behaviour cheaper and more consistent. Keep current knowledge, policy and operational truth outside the weights.