
by OpenAI
HealthBench is an OpenAI benchmark of 5,000 realistic health conversations, each graded with a physician-written rubric. It measures how well AI models respond in health settings.
DatasetFree
Step 1: Read the paper and the code linked on the announcement page.
Step 2: Run your model on the conversations and collect its last-turn responses.
Step 3: Grade each response against the conversation's rubric criteria with a model-based grader.
Step 4: Sum the points of the criteria met and divide by the maximum possible.
Step 5: Break results down by theme and axis.
A model-based grader checks each physician-written rubric criterion, and the overall score is the points earned over the maximum possible.
They are multi-turn and multilingual, with layperson and clinician personas across many specialties, and were selected for difficulty.
A group of 262 physicians who have practiced in 60 countries, proficient in 49 languages and trained in 26 specialties.
Themes group conversations, for example emergency situations, while axes describe the behavior a criterion grades, such as accuracy.
ChatGPT WorkChatGPT Work is an agent in ChatGPT that takes action across your apps and files and turns a goal into finished work such as sheets, slides, docs and web apps. It can stay with a project for hours.
OpenAI Realtime APIOpenAI's Realtime API lets developers build speech-to-speech voice agents that work directly with audio, keep conversation state and call tools. It is for developers building voice assistants and other low-latency conversational apps.
OpenAI DotsDots are always-on agents in ChatGPT that take on ongoing responsibilities and keep making progress between conversations. You choose which apps they use and when they need your approval.
Our World in Data HealthOur World in Data's global health collection brings together charts, data and writing on life expectancy, child and infant mortality, and healthcare spending. Charts are open access under a Creative Commons BY license.