AI test tool
AI Output Benchmark
Compare two sets of structured AI answers against expected JSON and an optional schema, case by case.
Tool Area
Everything runs in the browser.
Small evaluations reveal regressions that headline model scores miss. Upload or paste JSONL rows with expected, actual_a and actual_b. The report separates an exact answer from a merely schema-valid answer and exports a CSV for review. It never calls a model or needs an API key.
- Exact match compares parsed JSON, ignoring object key order but not array order.
- A schema-valid answer may still be factually wrong.
- Files are read in the browser; no prompts are sent to a model.
How to use
- Prepare one JSON object per line with id, expected, actual_a and optionally actual_b.
- Add a JSON Schema if structure matters.
- Run the comparison and inspect failed rows before downloading the CSV.
Common use cases
- Compare two model versions on the same extraction task.
- Check whether a prompt change breaks known examples.
- Review format errors separately from wrong values.
FAQ
Does it benchmark model speed or reasoning?
No. It compares supplied structured outputs against expected examples.
How many cases can I test?
Up to 200 JSONL rows and a 500 KB local file per run.