Fine-tuning

Fine-tuning is further training of a pretrained model on a smaller, task-specific dataset. The model's weights are updated so it adopts a style, output format or skill, such as classifying support tickets or writing in your house tone. It teaches behaviour more reliably than it teaches new facts. See retrieval augmented generation for facts.

How it works

You prepare a few hundred to a few thousand input and output examples, train for a few epochs, then evaluate on held-out data. Full fine-tuning updates every weight. For a 7B model with Adam in mixed precision, that is about 16 bytes per parameter, or roughly 112 GB.

LoRA freezes the base weights and trains small adapter matrices. At rank 8 on the query and value projections of a 32-layer model with hidden size 4,096, that is about 4.2M trainable parameters, around 0.06% of 7B.

Why it matters in production

Fine-tune after prompting and retrieval fall short, not before. A tuned model is a new artifact to version, evaluate and redeploy, so it needs LLM evaluation and MLOps discipline. Training data that leaks private records can surface in outputs.

See also

• Retrieval augmented generation

• LLM evaluation

• MLOps

• Embeddings

On the One2N blog

• Understanding MLOps from a software engineers' perspective

• Exploring retrieval augmented generation (RAG): enhancing LLMs with external knowledge

Further reading

• LoRA: low-rank adaptation of large language models (Hu et al., 2021)

• Universal language model fine-tuning for text classification (Howard and Ruder, 2018)