A training run fails on the first malformed record, and it usually finds it after you have gone home. The checks worth doing beforehand are unglamorous and quick: every record an object with a messages array, every message carrying a role and some content, the conversation starting where it should, and — the one people miss — ending on an assistant message, because that is the turn the record is teaching. Two user messages in a row are flagged too, since they usually mean two records got merged. Alongside the errors, the size distribution across the file shows the tail that will not fit before it becomes the reason the run stops.
Features
Every finding reported against the line it came from
Checks each record parses, and carries on past the ones that do not
Roles checked against the known set, unusual ones flagged
Catches a record that does not end on an assistant message
Catches two consecutive turns from the same side
Catches empty content, which validates and teaches nothing
Size distribution across the file: median, 90th percentile and largest
A configurable messages key for formats that call it something else
How to use the Fine-Tuning Dataset Validator
Drop in the .jsonl dataset, or paste it
Adjust the messages key if your format differs
Fix the errors, then look at the warnings
Check the size distribution against your context limit
Example
Input
line 3: two user messages in a row
Output
Records: 3 · Errors: 0 · Warnings: 1
Usually a sign that two records were merged by accident.
Common errors & troubleshooting
A record does not end with an assistant message. — That record has nothing to learn from — the final assistant turn is the target. It is an error rather than a warning for that reason.
It cannot find the messages array. — Set the messages key to whatever your format calls it. Conversation and turns are both common.
Two user messages in a row are flagged. — Usually two records concatenated by mistake. If the doubled turn is deliberate, it is a warning rather than an error.
Some records are enormous. — The percentiles show the tail. Anything beyond the context limit will fail, and it is far cheaper to find that here than partway through a run.
Frequently asked questions
What format does this expect?
The chat format: one JSON object per line, each with a messages array of role and content pairs. The key name is configurable, since conventions differ, but the shape inside is the same everywhere.
Why must a record end on an assistant message?
Because that turn is what the record teaches. A conversation ending on a user message has a question and no answer, so there is nothing for the run to learn from it.
Are the token counts exact?
No, they are character-based estimates. They are for spotting the outlier that will not fit, which does not need precision — the exact count comes from the provider's own tokeniser.
Does a clean result mean the data is good?
It means the file is well-formed. Whether the examples teach the right thing is a question about your data, and no validator answers it.
Is my dataset uploaded?
No. Training data is usually the most sensitive file a team has, and all of this runs in the browser.