Does the AI still work on your patients?

What a validation pipeline is

A validation pipeline is a fixed set of clinical cases with known-good answers, run against your AI workflow the same way every time. It deliberately stresses the workflow — missing documents, reworded inputs, swapped model versions — and measures what breaks. The result is a validation report you can hand to your quality manager or an auditor, regenerated on demand.

Two worked examples, on open data

Both pipelines below run on public or synthetic cases, so we can show you everything: the cases, the failures, and the report.

How it works

• Case bank. A fixed set of clinical cases with answers agreed before any model runs.
• Perturbations. Each case is re-run with documents withheld, reordered, or degraded, to expose what the AI actually relies on.
• Calibrated AI judge. A second model grades every output against a rubric, and the judge itself is checked against human grading before its scores count.
• Pinned run. Every run is frozen under a content hash, so each published number traces to archived evidence you can audit.
• Validation report. The pipeline ends in a dossier written for your QMS, generated straight from the pinned run.

Read the full methodology →

Open source

The pipelines run on validrig, our use-case-agnostic evaluation engine (AGPL-3.0). The engine and both example packs open as source repositories at launch. Clinic case banks and client data stay private — only the machinery and the open-data examples are public.

For clinical-AI tool manufacturers

Your customers will be asked for validation evidence. Two hooks in your product make your tool validatable by pipelines like these — and easier to buy:

• A callable endpoint. An OpenAI-compatible or documented inference API, so a pipeline can drive your tool the way a clinic does.
• Machine-readable run traces. Logs of what your tool received and produced per case, so results can be verified instead of trusted.

If you build clinical-AI tooling and want your product validatable, talk to us.

Get started

• Clinics: pick one AI workflow you run or plan to adopt. We scope a validation pipeline on your cases, inside your QMS — starting with a retrospective case bank, expanding to regression checks on every model change. Reach out.
• Tool manufacturers: expose the two hooks above and your product ships with validation evidence a hospital can file. Talk to us about a joint worked example with your name on it.
• Clinicians: the reference answers and the judge trust-check on these examples are waiting for clinical reviewers. If grading a sample of cases in your specialty interests you, we would like to hear from you — contributions are credited by name.

Start with the piece that fits and expand over time; every stage of the pipeline stands on its own.