LoRA & Parameter-Efficient Fine-Tuning

Full fine-tuning of large models requires enormous GPU resources. Parameter-efficient fine-tuning (PEFT) methods like LoRA let you customize models using a fraction of the compute.

The Problem with Full Fine-Tuning

A 7B parameter model needs ~28GB of GPU memory just to load — and fine-tuning requires 2-3x more:

Full fine-tuning of Llama 3 8B:
- GPU Memory: ~60-80 GB (requires A100 or multiple GPUs)
- Training Time: Hours to days
- Storage: 16 GB per checkpoint

PEFT methods solve this by only training a small subset of parameters.

What is LoRA?

LoRA (Low-Rank Adaptation) adds small trainable matrices alongside the model's frozen weights. Instead of modifying billions of parameters, you train millions.

Original Model (Frozen)     LoRA Adapters (Trainable)
┌─────────────────────┐     ┌───────────┐
│  Weight Matrix W    │  +  │  A × B    │
│  (4096 × 4096)      │     │  (4096×16  │
│  ~16M parameters    │     │   × 16×4096│
│  FROZEN             │     │  ~131K     │
└─────────────────────┘     │  TRAINABLE │
                            └───────────┘

Total trainable: ~0.8% of original parameters!

How LoRA Works

Freeze all original model weights
Add small decomposed matrices (A and B) to attention layers
Train only these small matrices
At inference: merge LoRA weights with original model

Key parameter: rank (r) — controls the size of LoRA matrices:

r=8: Very small, fast training
r=16: Good balance (most common)
r=32-64: Higher quality, more memory

Fine-Tuning with LoRA (Practical)

Setup

bash

pip install transformers peft trl datasets accelerate bitsandbytes

Training Script

python

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model
from trl import SFTTrainer
from datasets import load_dataset

# 1. Load base model
model_name = "meta-llama/Llama-3.1-8B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
    load_in_4bit=True  # QLoRA: 4-bit quantization
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token

# 2. Configure LoRA
lora_config = LoraConfig(
    r=16,                        # Rank
    lora_alpha=32,               # Scaling factor
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Output: "trainable params: 6,553,600 || all params: 8,030,261,248 || 0.08%"

# 3. Load training data
dataset = load_dataset("json", data_files="training_data.jsonl")

# 4. Training arguments
training_args = TrainingArguments(
    output_dir="./lora-output",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    warmup_steps=10,
    logging_steps=10,
    save_strategy="epoch",
    fp16=True
)

# 5. Train
trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    tokenizer=tokenizer,
    max_seq_length=2048,
)

trainer.train()

# 6. Save LoRA weights (just ~25MB!)
model.save_pretrained("./my-lora-adapter")

Using the Fine-Tuned Model

python

from peft import PeftModel

# Load base model + LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")
model = PeftModel.from_pretrained(base_model, "./my-lora-adapter")

# Generate
inputs = tokenizer("Write a product description for:", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0]))

QLoRA: Quantized LoRA

QLoRA combines quantization (4-bit) with LoRA for even lower memory usage:

Full Fine-Tuning (8B model):   ~60 GB GPU memory
LoRA (8B model):               ~20 GB GPU memory
QLoRA (8B model):              ~6 GB GPU memory  ← Fits on consumer GPU!

The key addition is load_in_4bit=True when loading the model:

python

from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="float16",
    bnb_4bit_use_double_quant=True
)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=quantization_config,
    device_map="auto"
)

Training Data Format

Chat Format (Recommended)

jsonl

{"messages": [{"role": "user", "content": "Classify: 'The product broke after one day'"}, {"role": "assistant", "content": "Category: Complaint\nSentiment: Negative\nPriority: High"}]}
{"messages": [{"role": "user", "content": "Classify: 'Love the new update!'"}, {"role": "assistant", "content": "Category: Feedback\nSentiment: Positive\nPriority: Low"}]}

Instruction Format

jsonl

{"instruction": "Translate to French", "input": "Hello, how are you?", "output": "Bonjour, comment allez-vous ?"}
{"instruction": "Summarize", "input": "Long article text...", "output": "Brief summary..."}

Other PEFT Methods

Method	Description	Best For
LoRA	Low-rank adaptation matrices	Most use cases
QLoRA	LoRA + 4-bit quantization	Limited GPU memory
Prefix Tuning	Trainable prefix tokens	Text generation
Prompt Tuning	Soft prompt optimization	Classification
IA3	Learned vectors scaling activations	Minimal parameter changes

Cloud Fine-Tuning Platforms

If you don't have local GPUs:

Platform	Models Available	GPU Provided
Google Colab	Any HuggingFace model	Free T4, paid A100
Lambda Labs	Any	A100, H100 (hourly)
RunPod	Any	Various GPUs (hourly)
Together AI	Popular open-source	Managed fine-tuning
Hugging Face	Any HF model	AutoTrain service

LoRA Hyperparameter Guide

Parameter	Description	Recommended
`r` (rank)	LoRA matrix rank	8-32
`lora_alpha`	Scaling factor	2× rank
`target_modules`	Which layers to adapt	q_proj, v_proj, k_proj, o_proj
`lora_dropout`	Regularization	0.05-0.1
`learning_rate`	Step size	1e-4 to 3e-4
`epochs`	Training passes	1-5
`batch_size`	Examples per step	4-16

Merging LoRA Weights

For production, merge LoRA into the base model to eliminate adapter overhead:

python

from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(model_name)
model = PeftModel.from_pretrained(base, "./my-lora-adapter")

# Merge LoRA weights into base model
merged_model = model.merge_and_unload()

# Save the merged model
merged_model.save_pretrained("./merged-model")
tokenizer.save_pretrained("./merged-model")

Key Takeaways

LoRA trains <1% of parameters — massive resource savings
QLoRA + consumer GPU — Fine-tune 8B models on a single GPU with 6GB VRAM
LoRA adapters are tiny (~25MB) — easy to store and swap
Start with rank 16 and adjust based on quality
Merge for production — eliminates adapter overhead at inference time

Next, we'll cover dataset preparation and model evaluation.