What is Fine-tuning?
Fine-tuning is the process of retraining a pre-trained AI model on a smaller, specialized dataset. The goal is adapting a foundation model to a specific domain (e.g., law, medicine, finance) or task (e.g., classification, data extraction, report generation).
Fine-tuning techniques
Full fine-tuning — retraining all model parameters (expensive, requires GPU). LoRA/QLoRA — low-rank adaptation, training a small parameter subset (10-100x cheaper). Instruction tuning — training on instruction-response pairs. RLHF — learning from human feedback.
When fine-tuning, when RAG?
Fine-tuning: when you want to change model style, format, or specialization. RAG: when you need current data (changing documents). In enterprise, both are usually combined: fine-tuned model + RAG from company knowledge base.