Clinical language model evaluation

CLExEval

A Human-in-the-Loop Framework for Qualitative Evaluation of LLM Clinical Reasoning

Ajmal M.1,2, Abin R.3,*, Afthab S. K.3,*, Jawadh A. K.3,*, Jerin J.3,*, Preslav Nakov1, Zhuohan Xie1

1 MBZUAI  ·  2 IIT Madras  ·  3 Calicut Medical College

* Abin R., Afthab S. K., Jawadh A. K., and Jerin J. contributed equally.

Paper Abstract Dataset Code
5,600expert annotations
200reasoning traces
7clinical dimensions
4physician experts

01 · Abstract

Fluency can conceal clinical failure.

While Large Language Models reportedly ace medical exams, their clinical deployment remains precarious due to an Evaluation Illusion where benchmarks prioritize linguistic fluency over causal reasoning. To test this, we introduce CLExEval, a forensic audit of 5,600 expert-physician annotations across 200 clinical reasoning traces using Progressive Information Masking. Our analysis exposes three critical failure modes: a Verbosity Bias (GPT-4o-mini's accuracy collapses from 95.0% to 32.5% under information scarcity), a Hidden Knowledge Paradox (specialized models possess 92.5% latent knowledge, but fail in verbose contexts), and a 68.6% Reasoning-Output Mismatch from self-censoring correct internal reasoning. Crucially, evaluating the LLM-as-a-Judge paradigm on a human-verified failure dataset (n=142) reveals a severe Hallucination Approval Rate. GPT-4o-mini approved 47.9% of human-verified fatal errors, while HuatuoGPT-o1 approved 100% and showed a positive self-preference bias, suggesting that standalone automated clinical leaderboards may substantially overestimate clinical reliability.

32.5% Accuracy under scarcity GPT-4o-mini falls from 95.0% as information is masked.
92.5% Latent knowledge Specialized models may know the answer but fail to commit.
68.6% Reasoning-output mismatch Correct internal reasoning is lost before the final answer.
100% Approval of fatal errors HuatuoGPT-o1 approved every verified failure in the audit.

02 · Failure anatomy

Inside the mind of a clinical AI.

A representative reasoning-output mismatch: the correct diagnosis appears in the internal trace, the final answer diverges, and automated judges reward the fluent but incorrect output.

03 · Method

The CLExEval pipeline.

Expert-curated cases are progressively masked, evaluated by two clinical language models, and assessed by medical residents across a seven-dimensional rubric.

04 · Release

Research materials.

The project page is available now. Dataset documentation, evaluation code, and reproducibility materials are being prepared for public release.

Paper Read the CLExEval preprint on arXiv. Read ↗
Project This research page and visual overview. View ↑
RARECASE-2000 Dataset card active; documented release in preparation. Open ↗
Code Evaluation and reproducibility materials in preparation. Open ↗

05 · Citation

Cite CLExEval.

If this work supports your research, please cite.

BibTeX 
@misc{m2026clexevalhumanintheloopframeworkqualitative,
      title={CLExEval: A Human-in-the-Loop Framework for Qualitative Evaluation of LLM Clinical Reasoning},
      author={Ajmal M. and Abin Roy and Afthab Salam Kanniyan and Jawadh Abdul Kabeer and Jerin James and Preslav Nakov and Zhuohan Xie},
      year={2026},
      eprint={2606.31608},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.31608},
}