Evaluating an LLM application should feel like running a test suite.
01 · Context
Evaluating an LLM application should feel like running a test suite.
The problem
Many conventional tests compare an output with a known expected result. An AI agent can give different answers to the same input, so a single successful run tells you less. I wanted a simple way to run evaluations, inspect what happened, and compare results as an agent changes.
My role
I’m building Evalstand in Python to make it easier for me and other developers to evaluate the AI agents we work on. The aim is a local workflow that feels familiar to someone who already writes tests.
02 · Engineering decisions
How it works.
- 01Use stable case IDs to compare runs without silently pairing different cases.
- 02Share an execution runner between the CLI and pytest plugin.
- 03Persist results in SQLite and cache model responses locally.
- 04Capture nested calls, latency, tokens, and cost alongside scores.
- Cases + task
- Runner · pytest
- Scorers + traces
- SQLite + comparison
03 · Quality strategy
How I tested it.
Tests cover case matching, timeouts, caching, nested traces, scorer protocols, and what reports say about results. An integration test replays responses offline; tests that call live providers run only when enabled.
04 · Result & reflection
Results so far.
The local workflow runs evaluations, shows traces, and compares saved results. The terminal UI also has a watch mode. Evalstand is still in development and hasn’t had a public release.
Scope & limitations
Score differences describe what changed between runs; they don’t establish statistical significance. The LLM judges have not yet been calibrated against human labels.
What I learned
A lower score needs context before I can call it a regression. Being able to inspect the case and its execution matters as much as seeing the comparison.
