The health and medicine subset of a broad expert-level multimodal reasoning benchmark.
Audited by MedCheckYue et al., 2024
Key facts
| What it measures | The health and medicine subset of a broad expert-level multimodal reasoning benchmark. |
| Who should care | Teams comparing medical performance against general multimodal capability. |
| Use it when | Placing medical performance in the context of general reasoning. |
| Do not use it for | Deep clinical evaluation. It is a subset of a general benchmark. |
| Score scale | Percent accuracy on the health and medicine subset. |
| Grader method | unknown |
| First released | 2024 |
| Languages | English |
| Use cases | imaging, multimodal, exam knowledge |
| Citation | Yue et al., 2024 |
| Status | active |
Frequently asked questions
What does MMMU (Health and Medicine) measure?
The health and medicine subset of a broad expert-level multimodal reasoning benchmark.
Who should use MMMU (Health and Medicine)?
Teams comparing medical performance against general multimodal capability. Placing medical performance in the context of general reasoning.
What should MMMU (Health and Medicine) not be used for?
Deep clinical evaluation. It is a subset of a general benchmark.
How are MMMU (Health and Medicine) scores reported?
Percent accuracy on the health and medicine subset. Scores from different benchmarks are not comparable to each other.
Is MMMU (Health and Medicine) independent?
No conflicts of interest have been verified for MMMU (Health and Medicine) in this registry as of 2026-08-05. Absence of a recorded conflict means none has been verified, not that none exists.
Compare with
Spectrum evaluation for multimodal medical models across many imaging modalities and difficulty levels.
A large chest radiograph dataset with uncertainty labels and expert comparison. Predates language models and i
Multimodal evaluation for endoscopy image and video analysis.
Large multimodal benchmark for general medical AI, spanning many imaging modalities, departments and task type
Very large medical visual question answering set spanning many modalities and anatomical regions.
Expert-level pathology understanding and reasoning from slide images.
