How OpenAI's dynamic mental health evaluations work
OpenAI · 3 safety metrics · updated August 17, 2026
OpenAI's dynamic mental health evaluations are a vendor safety evaluation, published in August 2026 with the company's GPT-5.6 August updates. They cover mental health, emotional reliance, and the quality of responses to self-harm. What sets them apart mechanically is that the test conversations are simulated and adaptive rather than fixed scripts. They cover OpenAI models only, so they are a vendor safety eval and not a cross-vendor benchmark.
| publisher | OpenAI |
|---|---|
| released | 2026-08 |
| size | dynamic simulated conversations (counts not disclosed) |
| scale | compliance rate per metric, 0 to 1, higher better; headline number is the mental-health metric |
Simulated users that adapt
In a static evaluation, a test case is a fixed transcript. The model answers once, and a grader scores that answer. A dynamic evaluation replaces the transcript with a simulated user that reads the model's reply and responds to it, so the conversation branches on what the model actually said.
That matters for this subject. Many failure modes only appear after several turns: a user who pushes back on a refusal, who reframes the request, or who returns to the topic once the conversation has settled. Adversarial simulation is built to reach those turns.
Compliance rates
Each metric is reported as a compliance rate from 0 to 1: the share of simulated conversations in which the model's behavior met OpenAI's written definition of the desired response. The headline figure is the mental-health metric.
Two things follow. The rate is only as stable as the behavior definitions and the simulator behind it, so a number produced under one set of definitions is not comparable to a number produced under another. And the conversation counts are not disclosed, so you cannot put a confidence interval around any of the rates.
What the rate is measured against
OpenAI states that the error rates here are not representative of average production traffic. The simulations concentrate on hard and adversarial cases by design, so the failure rate is higher than a general population of conversations would produce. Read the numbers as a comparison between OpenAI model versions on a fixed hard set, not as an estimate of how often a user meets a bad response.
Limits
This is an internal evaluation. It is not independently runnable, the conversation counts are not published, the simulator and the behavior definitions belong to OpenAI, and only OpenAI models appear. None of that makes the numbers wrong, but it means you cannot place another vendor's model on the same scale and you cannot reproduce a figure to check it. A vendor safety evaluation is most useful as a record of what a lab chose to measure and how its own number moved between releases.
clinicalbenchmarks.ai tracks this evaluation as a vendor safety eval, labeled apart from the cross-vendor benchmarks.
The grading machinery behind pages like this one is covered in the rubric grading and model graders guides.