How to Evaluate a Fine-Tuned Model
Evaluation is the step that separates useful fine-tuned models from expensive failures. A model can achieve low training loss by memorizing your examples without learning the underlying patterns. It can score well on your test set but fail on inputs that differ even slightly from your training distribution. And it can perform perfectly today but degrade silently over weeks as the input distribution shifts. Each of these failure modes requires a different evaluation approach.
Step 1: Build Your Evaluation Dataset
The evaluation dataset must be completely separate from your training and validation data. If any example appears in both the training set and the evaluation set, your metrics will be artificially inflated because the model has memorized those specific examples.
Create a test set of 100-500 examples that covers the full range of inputs your model will encounter in production. Include: typical requests that represent 80% of your traffic, edge cases that are rare but important, adversarial inputs that might confuse the model (ambiguous queries, requests outside the model's domain, malformed inputs), and examples at various difficulty levels from simple to complex.
For each test example, create a reference answer that represents the ideal output. These reference answers serve as the ground truth for automated metrics. Have at least two domain experts independently create reference answers for a subset (20-50 examples) to measure inter-annotator agreement. If your experts disagree significantly, your task definition may be too ambiguous for reliable evaluation.
Stratify your test set by category. If your model handles 5 types of requests, include at least 20 examples per type. This lets you measure performance per category rather than just an aggregate score. A model that achieves 95% overall accuracy but only 60% on one important category is not ready for production.
Step 2: Run Task-Specific Metrics
The right metric depends on your task type. Choose metrics that directly measure what matters for your application, not generic scores that sound impressive but do not predict production quality.
Classification tasks (sentiment, intent, category): Measure accuracy, precision, recall, and F1 score. Report per-class metrics, not just averages, because aggregate scores hide poor performance on minority classes. A model with 95% accuracy overall but 40% recall on the "urgent" class is dangerous if missing urgent cases has real consequences.
Text generation tasks (summaries, responses, content): ROUGE scores measure overlap with reference answers (ROUGE-1 for unigrams, ROUGE-L for longest common subsequence). BLEU measures n-gram precision. BERTScore uses semantic similarity rather than exact word matching. None of these metrics perfectly capture quality, so use them as filters (models below a threshold are definitely bad) rather than as definitive rankings (a ROUGE score 2 points higher does not guarantee a better model).
Structured output tasks (JSON, code, templates): Measure parse success rate (percentage of outputs that are syntactically valid), schema compliance (percentage that include all required fields with correct types), and semantic accuracy (percentage where the values are correct). For agent tool calling, also measure tool selection accuracy and argument correctness separately.
Always compare against the base model. Run the same test set through both the fine-tuned and base model (with your best prompt). Report the improvement or regression on each metric. This comparison answers the critical question: did fine-tuning actually help, or would better prompt engineering have achieved the same result?
Step 3: Test for Regression
Fine-tuning specializes the model, which can come at the cost of general capability. Run your fine-tuned model against a standard benchmark to quantify any regression.
MMLU (Massive Multitask Language Understanding): Tests general knowledge across 57 academic subjects. A drop of 1-3 points from the base model is normal and acceptable for most fine-tuning projects. A drop of 5+ points suggests overfitting.
HumanEval: Tests code generation ability. Important if your fine-tuned model will encounter any code-related requests. Even a model fine-tuned for customer support might need basic formatting capabilities that rely on the same skills as code generation.
MT-Bench: Tests multi-turn conversation quality with GPT-4 as a judge. This is particularly relevant for conversational fine-tuned models because it measures whether the model can maintain coherence across multiple turns.
Some regression on benchmarks unrelated to your fine-tuning domain is expected and acceptable. A model fine-tuned for medical text summarization does not need to maintain its pre-training score on poetry analysis. But if the model loses basic capabilities like following simple instructions, holding a coherent conversation, or performing arithmetic, something went wrong during training. The most common causes are: too many training epochs (overfitting), too high a learning rate (destabilizing the base weights), or training data that is too narrow in scope.
If regression is severe, reduce epochs from 3 to 1-2, lower the learning rate, increase training data diversity, or reduce the LoRA rank. A lower rank constrains the adaptation to a smaller space, which limits both the fine-tuning gains and the potential for regression.
Step 4: Conduct Human Evaluation
Automated metrics miss quality dimensions that humans detect instantly: tone, helpfulness, factual accuracy in context, and whether the response actually solves the user's problem. Human evaluation is more expensive and slower, but it is the only way to measure what your users will actually experience.
Set up a blind A/B comparison. Present evaluators with the input and two outputs, one from the fine-tuned model and one from the base model, without labeling which is which. Ask evaluators to rate each output on a 1-5 scale for task-specific dimensions: correctness, completeness, formatting, tone, and overall quality. Also ask them to choose which output they prefer overall.
Use at least 3 evaluators per example and at least 50 examples. Calculate inter-rater agreement using Cohen's kappa or Krippendorff's alpha. If agreement is low (kappa below 0.4), your evaluation criteria may need to be more specific, or the quality difference between models is too small to measure reliably.
Common pitfall: evaluators who know they are judging a fine-tuned model tend to rate it more favorably (confirmation bias). The blind setup is essential for honest evaluation. If possible, include a third option in the comparison, outputs from a completely different model, to prevent evaluators from developing a pattern of always choosing the "better" one.
Step 5: Set Up Production Monitoring
Evaluation does not end at deployment. Production traffic differs from test sets because real users send inputs that no test set fully anticipates. Set up automated monitoring that continuously measures your model's quality on live traffic.
Sample-based evaluation: Route 1-5% of production traffic through an automated evaluation pipeline. Compare the fine-tuned model's output against a reference (either a gold-standard answer if available, or the output of a stronger model like GPT-4o). Track aggregate scores daily and alert when they drop below a threshold.
User feedback signals: Track user actions that indicate quality: thumbs up/down ratings, message editing (user modifies the model's output), task abandonment (user gives up and restarts), and escalation rates (user requests human assistance). These behavioral signals are noisier than expert evaluation but are available at scale and reflect real user satisfaction.
Drift detection: Compare the distribution of inputs your model receives this week against the distribution from when you collected training data. If the input distribution has shifted significantly (new topics, different complexity levels, changed user expectations), your model's training data no longer represents its operating conditions. This is the signal to collect new training examples from the shifted distribution and retrain.
Retraining cadence: Plan for retraining before you need it. Set a schedule based on your domain's rate of change: monthly for fast-moving domains (technology, current events), quarterly for moderate domains (product documentation, customer support), and biannually for stable domains (legal procedures, medical protocols). Treat each retraining cycle as an opportunity to improve: add examples that address the specific failure modes you observed since the last training run.
Evaluation requires three layers: automated metrics on a held-out test set to measure task performance, regression benchmarks to confirm general capability is intact, and human judgment to catch quality issues that metrics miss. Then monitor continuously in production because real-world performance always diverges from test set performance over time.