Archived · Developer tooling

AI Test Harness

A small Rust harness for running model evaluations and getting the same number twice.

Anyone comparing models who has noticed their benchmark keeps moving.

Animated schematic of a representative session. Command names are real; timings are compressed for the page.

What it is

A minimal harness for running repeatable evaluations against a model endpoint. Small, early, and largely superseded by the evaluation tooling now living inside rev-local.

Capabilities

Anyone comparing models who has noticed their benchmark keeps moving.

Repeatable prompt/response evaluation
Repeatable prompt/response evaluation runs from a single Rust binary

How it works

This is how AI Test Harness works, not a set of steps you can follow yet — it is archived and not distributed.

Drop it in
AI Test Harness attaches to the repository or project you already have.
Give it the job
It takes a concrete task with a defined output, not an open-ended prompt.
Review the result
Everything it produces is a diff, a file or a report you approve before it counts.

Specification

Read from the repository at build time rather than typed onto a marketing page.

Related work

rev-local Autonomous code review that never uploads your repository

Ask about integrating it with the repositories and agents you already run.