Dataset Preparation & Model Evaluation

The quality of your fine-tuned model depends entirely on the quality of your training data. This topic covers preparing datasets and evaluating results.

Dataset Preparation Pipeline

Raw Data → Cleaning → Formatting → Splitting → Validation → Training

Step 1: Collecting Training Data

Sources of Training Data

Source	Pros	Cons
Existing logs (support tickets, emails)	Real-world, diverse	May contain errors
Manual creation	High quality, controlled	Expensive, slow
LLM-generated (synthetic)	Fast, scalable	May lack diversity
Public datasets	Free, large	May not match your domain

Generating Synthetic Data

Use a powerful model to generate training data for a smaller model:

python

from openai import OpenAI

client = OpenAI()

def generate_training_examples(task_description, n_examples=50):
    examples = []

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": f"""Generate {n_examples} diverse training examples for this task:
            {task_description}

            Format each example as:
            INPUT: [user input]
            OUTPUT: [ideal model response]

            Make examples diverse — vary length, complexity, edge cases, and topics.
            Include at least 5 edge cases."""
        }],
        temperature=0.9,
        max_tokens=4000
    )

    # Parse the generated examples
    raw = response.choices[0].message.content
    # ... parse into structured format
    return examples

Step 2: Cleaning Data

Common Issues to Fix

python

def clean_example(example):
    """Clean a single training example."""
    # Remove extra whitespace
    example["input"] = " ".join(example["input"].split())
    example["output"] = " ".join(example["output"].split())

    # Remove PII (if not needed)
    example["input"] = redact_pii(example["input"])

    # Ensure consistent format
    if not example["output"].endswith((".", "!", "?")):
        example["output"] += "."

    return example

def validate_example(example):
    """Check if an example meets quality standards."""
    checks = {
        "input_not_empty": len(example["input"].strip()) > 0,
        "output_not_empty": len(example["output"].strip()) > 0,
        "input_not_too_short": len(example["input"]) > 10,
        "output_not_too_long": len(example["output"]) < 5000,
        "no_placeholder_text": "[TODO]" not in example["output"],
        "consistent_format": True  # Add your format checks
    }

    return all(checks.values()), checks

Deduplication

python

def deduplicate(examples, threshold=0.95):
    """Remove near-duplicate examples."""
    from difflib import SequenceMatcher

    unique = []
    for example in examples:
        is_duplicate = False
        for existing in unique:
            similarity = SequenceMatcher(
                None,
                example["input"],
                existing["input"]
            ).ratio()
            if similarity > threshold:
                is_duplicate = True
                break
        if not is_duplicate:
            unique.append(example)

    print(f"Removed {len(examples) - len(unique)} duplicates")
    return unique

Step 3: Formatting for Training

OpenAI Format (JSONL)

python

import json

def format_for_openai(examples, system_prompt):
    """Convert to OpenAI fine-tuning format."""
    formatted = []
    for ex in examples:
        entry = {
            "messages": [
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": ex["input"]},
                {"role": "assistant", "content": ex["output"]}
            ]
        }
        formatted.append(entry)
    return formatted

def save_jsonl(data, filepath):
    with open(filepath, "w") as f:
        for entry in data:
            f.write(json.dumps(entry) + "\n")

# Usage
examples = format_for_openai(
    cleaned_examples,
    "You are a helpful customer support agent for BIGXSTAR."
)
save_jsonl(examples, "training_data.jsonl")

Alpaca Format (Open-Source Models)

python

def format_for_alpaca(examples):
    formatted = []
    for ex in examples:
        formatted.append({
            "instruction": ex.get("instruction", ""),
            "input": ex["input"],
            "output": ex["output"]
        })
    return formatted

Step 4: Train/Validation Split

python

import random

def split_dataset(examples, train_ratio=0.85):
    """Split into training and validation sets."""
    random.shuffle(examples)
    split_point = int(len(examples) * train_ratio)

    train = examples[:split_point]
    val = examples[split_point:]

    print(f"Training: {len(train)} examples")
    print(f"Validation: {len(val)} examples")

    return train, val

Step 5: Data Validation

OpenAI Data Validation

python

def validate_training_file(filepath):
    """Validate a JSONL file for OpenAI fine-tuning."""
    errors = []
    with open(filepath) as f:
        for i, line in enumerate(f):
            try:
                entry = json.loads(line)

                # Check structure
                if "messages" not in entry:
                    errors.append(f"Line {i}: Missing 'messages'")
                    continue

                messages = entry["messages"]

                # Check roles
                roles = [m["role"] for m in messages]
                if roles[-1] != "assistant":
                    errors.append(f"Line {i}: Last message must be 'assistant'")

                # Check content
                for m in messages:
                    if not m.get("content", "").strip():
                        errors.append(f"Line {i}: Empty content for {m['role']}")

            except json.JSONDecodeError:
                errors.append(f"Line {i}: Invalid JSON")

    if errors:
        print(f"Found {len(errors)} errors:")
        for e in errors[:10]:
            print(f"  {e}")
    else:
        print("Validation passed!")

    return len(errors) == 0

Model Evaluation

Evaluation Metrics

Metric	What It Measures	How to Compute
Accuracy	Correct answers	Compare to ground truth
BLEU	Text similarity	N-gram overlap with references
ROUGE	Summarization quality	Recall of reference n-grams
Perplexity	Model confidence	Lower = more confident
Human eval	Overall quality	Human raters score outputs

Automated Evaluation

python

def evaluate_model(model_id, test_examples):
    """Evaluate a fine-tuned model against test data."""
    results = {
        "correct": 0,
        "total": len(test_examples),
        "scores": []
    }

    for example in test_examples:
        # Generate response
        response = client.chat.completions.create(
            model=model_id,
            messages=[
                {"role": "system", "content": example.get("system", "")},
                {"role": "user", "content": example["input"]}
            ],
            temperature=0
        )
        predicted = response.choices[0].message.content
        expected = example["output"]

        # Score with LLM-as-judge
        score = llm_judge(example["input"], predicted, expected)
        results["scores"].append(score)
        if score >= 4:
            results["correct"] += 1

    results["accuracy"] = results["correct"] / results["total"]
    results["avg_score"] = sum(results["scores"]) / len(results["scores"])

    return results

def llm_judge(question, predicted, expected):
    """Use GPT-4 to score a response on a 1-5 scale."""
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{
            "role": "user",
            "content": f"""Rate this AI response on a 1-5 scale.

Question: {question}
Expected answer: {expected}
Actual answer: {predicted}

Score (1=terrible, 5=perfect):
Reasoning:"""
        }],
        temperature=0,
        max_tokens=100
    )
    # Parse score from response
    text = response.choices[0].message.content
    import re
    match = re.search(r'Score[:\s]*(\d)', text)
    return int(match.group(1)) if match else 3

A/B Comparison

Compare fine-tuned model against the base model:

python

def ab_compare(base_model, finetuned_model, test_examples):
    """Compare two models side by side."""
    wins = {"base": 0, "finetuned": 0, "tie": 0}

    for example in test_examples:
        base_response = generate(base_model, example["input"])
        ft_response = generate(finetuned_model, example["input"])

        # Use GPT-4 to judge which is better
        judgment = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": f"""Which response is better for this question?

Question: {example['input']}

Response A: {base_response}
Response B: {ft_response}

Answer: A, B, or TIE"""
            }],
            temperature=0
        )

        verdict = judgment.choices[0].message.content.strip().upper()
        if "A" in verdict:
            wins["base"] += 1
        elif "B" in verdict:
            wins["finetuned"] += 1
        else:
            wins["tie"] += 1

    return wins

Dataset Size Guidelines

Task Type	Minimum Examples	Recommended	Notes
Classification	20 per class	100+ per class	Balance classes
Text generation	50	200-500	Diverse outputs
Chat/conversation	100	500-1000	Multi-turn examples
Code generation	50	200+	Cover various patterns
Summarization	50	200+	Vary lengths

Complete Evaluation Checklist

Validated training data format (no errors)
Checked for duplicates and removed them
Split data into train/val sets
Training loss decreased consistently
Validation loss didn't diverge from training loss
Tested on held-out examples
Compared against base model (A/B test)
Tested edge cases and adversarial inputs
Measured latency and cost per request
Human evaluation on sample outputs

Key Takeaways

Data quality is everything — 100 perfect examples beat 1000 noisy ones
Validate before training — Catch formatting errors early
Synthetic data is viable — Use GPT-4 to generate training data for smaller models
Always hold out test data — Never evaluate on training examples
Use LLM-as-judge — Scalable automated evaluation
Compare against baseline — Prove your fine-tuning actually improved things

With fine-tuning covered, the next module explores AI Agents — systems that autonomously use tools to accomplish complex tasks.