01
RAG vs fine-tune audit
Week-1 workshop. We benchmark retrieval against the use case before committing a training budget. Often the answer is: skip fine-tuning entirely.
★ Fine-tuningDirect LLM · no framework8-week engagement
FEEDBACK TRAINING · MODEL TUNINGFine-tune when RAG genuinely won't get you there.
LoRA, DPO, or full fine-tune — chosen by data volume and use case, not vendor pitch. Feedback capture pipeline → labeled dataset → eval gate → adapter shipped. We pay back the training cost in weeks, not quarters.
PythonPyTorchLoRAHuggingFace
Cycle
8 weeks · audit-first
Stack
HuggingFace · PyTorch · LoRA · DPO
Output
Adapter + eval suite + feedback pipeline
Default
Audit before training spend
[WHY THIS EXISTS]
Fine-tuning is expensive — and often unnecessary.
Most teams reach for fine-tuning when better retrieval would solve the problem cheaper, faster, and reversibly. Our default is RAG. Fine-tuning is for the cases where the base model genuinely lacks the format, the domain vocabulary, or the latency profile you need.
- First answer is always: 'Is RAG cheaper here?' — and usually it is
- LoRA adapters for domain language without retraining the base model
- DPO for preference alignment when you have human feedback at scale
- Full fine-tune only when the data volume + lock-in is justified
[HOW WE BUILD IT]
Cheap before expensive. Reversible before permanent.
02
Feedback capture pipeline
Thumbs-up/down, edit-deltas, conversion signals — captured at the proxy and routed to a labeled dataset. The training set grows as you ship.
03
LoRA over full fine-tune
LoRA adapters as the default. Hot-swappable per customer or per tenant. The base model stays current; the adapter carries the customisation.
04
Eval-gated promotion
An adapter only replaces the production version after passing the golden eval suite. Same gate as every other prompt change.
[OUTCOMES AT HANDOFF]
What's live at week eight.
1 audit
RAG vs fine-tune decision documented
1 adapter
LoRA or DPO shipped + eval-gated
Continuous
Feedback pipeline feeding the next round
Hot-swap
Per-customer adapters under one base model
[ALSO WORTH READING]
Related LLM engagements.
DIRECT LLM · APPLIED KBring the problem.
Bring the problem.
We’ll bring the build.
Eight weeks, fixed price, eval suite at handoff. Direct LLM engineering on top of the K-Framework. Two Q3 slots remain.