How HealthAdminBench works
Kinetic Systems · 135 admin tasks · updated August 17, 2026
HealthAdminBench measures agents on the paperwork side of healthcare. Kinetic Systems' research team built it with domain experts from Stanford Hospital and published it in April 2026. It covers prior authorizations, denial appeals, and durable medical equipment ordering. It evaluates computer-use agents, meaning agents that drive the software interfaces a person would use rather than calling a clean API.
| publisher | Kinetic Systems (with Stanford Hospital domain experts) |
|---|---|
| released | 2026-04 |
| size | 135 tasks / 1,698 rubric-scored subtasks |
| scale | percentage end-to-end task success 0-100, higher better |
Tasks, subtasks, and all-or-nothing scoring
The suite holds 135 tasks broken into 1,698 rubric-scored subtasks, roughly a dozen subtasks per task. The headline metric is end-to-end task success as a percentage, and it is binary at the task level: an agent gets credit only if it completes every subtask in that task.
That rule is why end-to-end numbers read lower than per-step accuracy would suggest. Reliability compounds across steps, so an agent that handles almost every subtask correctly can still fail most tasks, because one missed field or one skipped step sinks the whole thing. A version of the same benchmark that awarded partial credit would report a much higher number for identical behavior.
Why the subtask rubric is still there
A binary task score tells you whether the job got done, not where it broke. The 1,698 subtask scores carry that detail: which step of a prior authorization the agent skipped, which field it filled in wrong, which appeal it assembled without the supporting evidence. When you read the board, the task percentage is the deployment question and the subtask detail is the diagnostic question.
How it sits next to other admin agent boards
CHI-Bench covers similar administrative workflows and scores them with deterministic unit tests plus a judge model for evidence grounding and consent. HealthAdminBench differs in the interface it tests: computer use rather than tool calls. The same model can score differently under the two, because the harness and the interface are part of what is being measured.
Limits
The board is published by a company that sells healthcare administration automation. The research team is independent of the model vendors, which is what matters for whether the rows were fairly run, but the choice of workflows and the framing come from a party with a commercial interest, so read the write-up with that in view. There are no GPT-5.6 or Claude 5 rows and no stated refresh cadence, so the table ages between releases. Coverage is three workflow types out of a much larger administrative surface.
For current rows and the other agentic boards beside it, see clinicalbenchmarks.ai.
The grading machinery behind pages like this one is covered in the rubric grading and model graders guides.