Continued pretraining, SFT, preference optimization, reasoning distillation, model merging. The recipe that produced Llama-Nemotron and the R1 distillation family. We run it end to end and ship signed weights with a model card you can defend in audit.
Eight stages, each with a named output. The first four cover vocabulary and behaviour. The next two add reasoning and consolidation. The last two close the loop with evaluation and a model card.
Base + CPT + SFT + Preference + Distill + Merge + Eval + Sign.
Llama / Qwen / Mistral
Domain corpus
Labelled (prompt, response)
Preference pairs
R1 lineage + Sigstore
Llama 4 Maverick (85.5% MMLU, late 2025), Qwen 3 (leading multilingual), Mistral Small 4 (Apache 2.0, function calling native), Phi-4 (small + strong), DeepSeek V3 (MoE). Pick by license posture, target size, and the language coverage that matches your data. Always start from base, never from Instruct, when you intend to retrain alignment.
1B to 100B tokens of curated domain text. Vocabulary extension if the tokenization is inefficient. Replay 5 to 20% of the original instruction-tuning mix to limit forgetting. Multi-node FSDP or Megatron-LM, checkpoint every 1B tokens. This is the stage that teaches vocabulary the base never saw (legal Latin, ICD codes, chemistry SMILES, regional scripts).
Full SFT for deep re-tasking, LoRA at rank 64 for cheaper iteration. LR 1e-5 to 5e-5, cosine, low warmup. The transition from CPT to SFT is delicate: too high an LR destroys the CPT capabilities, too low and SFT does not stick.
DPO is our default. SimPO claims +6.4 AlpacaEval 2 over DPO and is worth benchmarking. ORPO collapses SFT and preference into one stage if budget is tight. KTO when production feedback is thumbs not pairs. Beta 0.1 to 0.5, 1 to 2 epochs.
DeepSeek-R1 (Jan 2025) used 800k verified reasoning trajectories to SFT smaller students (Qwen 1.5B up to Llama 70B) into frontier-grade reasoning. The 2025 breakout pattern. Distill from R1, GPT-4.1, Claude Sonnet 4.5, or a custom reasoning teacher into the production-target student. Verifier in the loop for math and code.
Combine task-specific fine-tunes into a single deployable with mergekit. TIES for general consolidation, DARE for noisy LoRAs, SLERP for two-model blends. Multi-skill consolidation in minutes, no GPU training needed.
MMLU-Pro, IFEval, MT-Bench, AlpacaEval 2, Arena-Hard, domain golden set. HarmBench + JailbreakBench for safety. Bias audit. LLM-as-judge with calibration. Block the deploy on any regression.
Sigstore + in-toto attestation per checkpoint. Dataset hash maps to base model hash maps to checkpoint hash. Model card per OECD format with intended use, training data summary, evaluation results, known limitations, copyright posture. Required for EU AI Act GPAI providers.
The 2025 R1 lineage made reasoning teachable to a small student. Most enterprise reasoning use cases can now be served by a 7B-14B specialist that costs an order of magnitude less per call than a frontier API. Custom models from scratch remain a labs-and-frontier game; custom models from distillation are a board-room decision we ship from week one.
SFT, LoRA, QLoRA, DoRA, DPO, SimPO, ORPO, KTO, GRPO/RFT, distillation, model merging. Every named technique with when it earns the build.
Read moreSourcing, PII redaction (Presidio), synthetic data (Distilabel, Nemotron), DEITA quality scoring, MinHash + SemDedup, labeling vendors, feedback loops.
Read moreOpenAI RFT, Anthropic on Bedrock, Vertex, Azure Foundry, Databricks Mosaic, Together, Predibase, NeMo Customizer, Modal, Lambda. Side-by-side with our take.
Read moreUnder 1k examples to over 1M, single A10G to 128 B200. Indicative cost, recommended method, hardware tier.
Read moreEU AI Act (Article 25 substantial-modification trap), GDPR, HIPAA, FedRAMP, Colorado AI Act, India DPDP, China GenAI Measures. Region-by-region for tuned LLMs.
Read more