Does the AI still work on your patients?

What a validation pipeline is

A fixed set of clinical cases with known-correct answers, run against your AI the same way every time. The pipeline deliberately breaks things: removes documents, reorders inputs, swaps the model version. It measures what stops working. The result is a report you can hand to your quality manager, regenerated whenever something changes.

Two worked examples, on open data

Both pipelines below run on public or synthetic cases. That means we can show you everything: the cases, the failures, the full report.

How it works

• Case bank. A fixed set of clinical cases with answers agreed before any model runs.
• Stress test. Each case runs again with documents removed, reordered, or degraded. If the output does not change, the model was not using what you fed it.
• AI judge, checked by humans. A second model grades every output against a rubric. Before those grades count, clinicians grade a sample and the engine checks whether the judge agrees with them.
• Frozen run. Every run is saved under a content hash. Each number we publish links to archived evidence you can check.
• Validation report. The pipeline ends in a dossier written for your QMS, produced straight from the frozen run.

Read the full methodology →

Open source

The pipelines run on validrig, our evaluation engine (AGPL-3.0). The engine and both example packs open as source repositories at launch. Clinic case banks and client data stay private. Only the machinery and the open-data examples are public.

For clinical-AI tool manufacturers

Your customers will be asked for validation evidence. Two hooks in your product make your tool validatable by pipelines like these, and easier to buy:

• A callable endpoint. An inference API a pipeline can drive, the way a clinic does.
• Run traces. Logs of what your tool received and produced per case, so results can be checked instead of trusted on faith.

If you build clinical-AI tooling and want your product validatable, talk to us.

Get started

• Clinics: pick one AI workflow you run or plan to adopt. We scope a validation pipeline on your cases, inside your QMS. Start with a retrospective case bank, expand to regression checks on every model change. Reach out.
• Tool manufacturers: expose the two hooks above and your product ships with validation evidence a hospital can file. Talk to us about a joint worked example with your name on it.
• Clinicians: the reference answers and the judge trust-check on these examples are waiting for clinical reviewers. If grading a sample of cases in your specialty interests you, see what it involves. Contributions are credited by name.

Start with the piece that fits. Every stage of the pipeline stands on its own.