How to Prepare Training Data for Fine-Tuning
The majority of failed fine-tuning projects fail at the data stage, not the training stage. Teams invest hours configuring LoRA parameters and experimenting with learning rates when the real problem is that their training examples contain contradictions, errors, or insufficient diversity. Getting the data right is the single highest-leverage investment in any fine-tuning project.
Step 1: Define Your Task and Output Format
Before collecting any data, write a precise specification of what the model should do. This is not a system prompt, it is a document that describes the task clearly enough that a new hire could produce correct examples after reading it. Include: what information the model receives as input, what output it should produce, what format the output should follow, and what edge cases exist.
Create 10 gold-standard examples by hand. These should be your best possible outputs, the ones you would show to a client or put in your documentation. These examples serve three purposes: they validate that your task specification is implementable, they become the first entries in your training set, and they establish a quality benchmark for all future examples.
The specification should be specific enough to remove ambiguity. "Write a helpful response" is too vague. "Write a response of 2-4 sentences that directly answers the user's technical question, includes a code example if applicable, and ends with a suggestion for the next step" is specific enough that two different people would produce similar outputs from the same input.
Common mistake: defining the task too broadly. If you try to train one model to handle customer support, content writing, and data analysis, you will get mediocre results on all three. Fine-tune one model per task, or at minimum per task family. A customer support model that also handles billing questions and account changes is fine. A model that handles customer support and also writes marketing copy is too broad.
Step 2: Collect Raw Examples
Production logs are your best source. If you have an existing AI application running with prompt engineering, every successful interaction is a potential training example. Filter for interactions where the output met your quality bar (confirmed by user feedback, manual review, or downstream metrics). These examples capture real-world input distributions, including the messy, ambiguous, and poorly-formatted queries your model actually receives.
Expert work product is your second-best source. If you are building a medical summarization model, collect summaries written by experienced clinicians. If you are building a code review agent, collect code reviews from your senior engineers. The value of expert data is that it encodes judgment calls that are difficult to specify in rules but easy to learn from examples.
Synthetic data fills gaps. Use a strong model like GPT-4o or Claude to generate examples for scenarios you lack real data for. This is useful for rare edge cases, error handling examples, and expanding the diversity of your input distribution. But synthetic data has a critical limitation: it can only reproduce patterns that the generating model has seen. It will not capture the quirks, exceptions, and domain-specific nuances that real data contains.
A good dataset combines all three sources: 60-70% production logs, 20-30% expert examples, and 10-20% synthetic data for edge cases. This gives you the realism of production data, the quality ceiling of expert examples, and coverage of rare scenarios through synthetic generation.
How many examples? For most fine-tuning tasks, 500 to 2,000 high-quality examples produce noticeable improvements. For complex tasks requiring nuanced judgment, aim for 5,000 to 10,000. Going beyond 10,000 shows diminishing returns unless you are doing continued pre-training on a new domain. Start with 500, evaluate the model, and add more examples only for the specific input types where performance is lacking.
Step 3: Clean and Standardize
Remove personally identifiable information. Names, email addresses, phone numbers, account numbers, IP addresses, and any other PII must be removed or replaced with realistic but fake values. This is both a legal requirement (GDPR, CCPA) and a practical one: you do not want your model memorizing and reproducing real customer data.
Fix factual errors. Every incorrect example in your training set teaches the model to make that same mistake. Have a domain expert review at least a random sample of your examples. If the error rate in your sample exceeds 3%, clean the entire dataset. Common errors include outdated information, copy-paste mistakes, and edge cases where the original system gave a plausible but wrong answer.
Standardize formatting. If some examples use bullet points and others use numbered lists for the same type of content, pick one and apply it consistently. If some examples use "USD" and others use "$", standardize. The model learns from every detail, including inconsistencies. Consistent formatting in the training data produces consistent formatting in the model's outputs.
Remove duplicates and near-duplicates. Exact duplicates teach the model to overweight certain patterns. Near-duplicates (same question phrased slightly differently with the same answer) have a similar effect. Keep the best version of each unique example and remove the rest. A good deduplication check uses both exact string matching and semantic similarity to catch paraphrased duplicates.
Discard the bottom 10%. Rank your examples by quality and remove the worst 10%. It is better to have 900 excellent examples than 1,000 examples where 100 are mediocre. The model cannot distinguish between "this is a good example to learn from" and "this is a borderline example you should not quite follow." It learns all patterns equally.
Step 4: Format for Training
Modern LLMs expect training data in a conversational format. Each example is an array of messages with roles and content. The standard roles are "system" (task context that does not change), "user" (the input), and "assistant" (the desired output). Your training framework applies the model's specific chat template to convert these roles into the token sequences the model expects.
A properly formatted JSONL example:
{"messages": [{"role": "system", "content": "You are a product recommendation engine for an outdoor gear retailer."}, {"role": "user", "content": "I need a tent for winter camping in Colorado."}, {"role": "assistant", "content": "For winter camping in Colorado, you need a 4-season tent rated for heavy snow loads and sub-zero temperatures. The MSR Access 2 ($500) handles winds up to 60 mph and sheds snow effectively with its steep sidewalls. For extreme conditions, the Hilleberg Soulo ($800) is the gold standard for solo winter camping with a proven track record above 14,000 feet."}]}
Keep system messages consistent across your dataset. If every example has a different system message, the model has to learn to adapt to varying instructions, which dilutes the fine-tuning signal. Use one system message for all examples, or at most 2-3 variants if your model needs to handle different personas or modes.
For multi-turn conversations, include the full conversation history in each example. The model learns from the complete context, including how to handle follow-up questions and maintain consistency across turns. A multi-turn example might have 6-8 messages alternating between user and assistant.
Watch your token lengths. If any single example exceeds your max_seq_length setting during training, it will be truncated, which can cut off the assistant response mid-sentence. This teaches the model to produce incomplete outputs. Calculate the token count of each example and either shorten the long ones or increase your max_seq_length (which increases VRAM usage).
Step 5: Validate and Split
Quality audit: Randomly select 50-100 examples and review each one against your task specification from Step 1. Check that the assistant response is correct, well-formatted, and matches your quality standard. If more than 5% of the sample fails your quality check, go back to Step 3 and clean the full dataset. This audit catches systematic issues that individual reviews miss.
Distribution check: Analyze the distribution of your training data across relevant dimensions. If you are building a customer support model, check that you have examples for all product lines, issue types, and customer segments. A model trained on 90% billing questions and 10% technical questions will handle billing well and technical questions poorly. Balance the distribution by adding more examples in underrepresented categories.
Train/validation split: Split your dataset into 90% training and 10% validation. The split should be random but stratified: ensure the validation set contains examples from all categories in roughly the same proportions as the training set. Never put related examples (like different turns from the same conversation) in both sets, as this creates data leakage.
Final sanity checks: Verify there are no empty messages, no messages with only whitespace, no examples where the assistant response is identical to the user message, and no examples that exceed your token limit. These issues cause training errors or teach the model degenerate behavior. A simple script that validates each JSONL line catches most formatting problems before you waste compute on a failed training run.
Save your final datasets as train.jsonl and validation.jsonl. Include a metadata file that records the dataset version, creation date, number of examples, source breakdown (production/expert/synthetic), and any cleaning steps applied. This documentation is essential when you retrain months later and need to understand what data the current model was trained on.
Training data preparation is the most important step in fine-tuning. Invest the time to define your task precisely, collect real-world examples, clean them ruthlessly, format them consistently, and validate the final dataset before training. One hour spent on data quality saves many hours of debugging model behavior later.