Whether a model computes clinical scores and formulas correctly from a patient note.
Audited by MedCheckKhandekar et al., NeurIPS 2024
Key facts
| What it measures | Whether a model computes clinical scores and formulas correctly from a patient note. |
| Who should care | Anyone deploying a model anywhere near dosing or risk scoring. |
| Use it when | You need arithmetic and formula reliability, which is where models quietly fail. |
| Do not use it for | Conversational or reasoning quality. |
| Score scale | Percent accuracy on computed values. |
| Grader method | unknown |
| First released | 2024 |
| Languages | English |
| Use cases | clinical decision support, safety |
| Citation | Khandekar et al., NeurIPS 2024 |
| Status | active |
Frequently asked questions
What does MedCalc-Bench measure?
Whether a model computes clinical scores and formulas correctly from a patient note.
Who should use MedCalc-Bench?
Anyone deploying a model anywhere near dosing or risk scoring. You need arithmetic and formula reliability, which is where models quietly fail.
What should MedCalc-Bench not be used for?
Conversational or reasoning quality.
How are MedCalc-Bench scores reported?
Percent accuracy on computed values. Scores from different benchmarks are not comparable to each other.
Is MedCalc-Bench independent?
No conflicts of interest have been verified for MedCalc-Bench in this registry as of 2026-08-05. Absence of a recorded conflict means none has been verified, not that none exists.
Compare with
Simulated clinical encounters where the model must gather information over several turns rather than answer on
Real clinical practice text, drawn from actual notes rather than exam material, across many languages and task
Large-scale Chinese benchmark built around real clinical scenarios rather than exam questions.
Broad clinical benchmark covering summarization, diagnosis and treatment planning in one suite.
Comprehensive Chinese medical benchmark spanning exam knowledge and clinical case work.
Questions answered from real discharge summaries, written by clinicians for real-world practice.
