ConsultMind: Automating Diagnostic Consultations with Uncertainty Awareness

Paper: ConsultMind:Towards Automated Diagnostic Consultation via Uncertainty-Aware Reasoning

Authors: Xiao Sun, Yuming Yang, Yun Chen, Jiang Zhong, Junnan Zhu, Xinyi Jiang, Haoyang Zeng, Ruirui Chen, Yining Wang, Xinyu Zhou, Rong Tang, Kaiwen Wei

Executive Summary

This research presents advancements in automating the diagnostic consultation process in healthcare, specifically focusing on how clinicians gather evidence from patients to reach a diagnosis. The study introduces two key innovations: AutoDisym and ConsultMind, which leverage Bayesian networks to enhance decision-making in clinical settings.

AutoDisym is a tool designed to create a Disorder–Symptom Bayesian Network (DSBN) by integrating diagnostic knowledge with varied clinical narratives that are labeled with diagnosis information. This automated pipeline aims to streamline the construction of high-quality DSBNs, which serve as the foundation for clinical decision-making. In tests involving multiple medical fields—psychiatry, respiratory medicine, and fever clinics—AutoDisym demonstrated its ability to generate effective DSBNs. It achieved a macro-averaged F1 score of 81.37 for recognized symptoms and 72.19 for related manifestations when evaluated using a specific model, GPT-5.6-Sol. These scores indicate a robust performance in linking symptoms to potential diagnoses.

Building on the capabilities of AutoDisym, ConsultMind is an uncertainty-aware framework that updates the disorder probabilities after each patient interaction. This means that as clinicians gather more information, the system refines its understanding of the patient's condition. ConsultMind not only enhances the accuracy of diagnoses but also improves the clarity of explanations provided to clinicians. It showed significant improvements in diagnostic performance, with increases in Top-1 and Top-3 accuracy by as much as 22.15 percentage points and 37.89 percentage points, respectively.

The research also highlights that physician evaluations found ConsultMind to enhance the quality of explanations for diagnosis rankings and rationales across various large language models (LLMs). This suggests that the framework not only improves decision accuracy but also aids in the communication of those decisions, which is crucial for clinician understanding and patient care.

These findings indicate that the integration of AutoDisym and ConsultMind could potentially transform diagnostic processes by making them more efficient and reliable. The automated nature of these tools may reduce the cognitive load on healthcare providers, allowing them to focus more on patient care rather than the intricacies of diagnosis.

While the study's results are promising, they stem from benchmarks and simulations, suggesting that real-world applications may require further validation. Nonetheless, the research illustrates a significant step toward enhancing automated diagnostic consultations, with implications for improving patient outcomes in various medical fields.

Academic Abstract

Diagnostic consultation is an online sequential decision-making process in which clinicians gather evidence through patient interaction until a diagnosis is sufficiently supported. Automating this process requires adaptive inquiry and interpretable decisions. Bayesian networks offer a natural foundation by updating diagnostic posteriors as evidence accumulates, but their use in open-ended consultation raises two challenges: linking diagnostic hypotheses to potential inquiries and translating evolving posteriors into consultation decisions. We introduce AutoDisym, an automated pipeline that integrates diagnostic knowledge with heterogeneous diagnosis-labeled clinical narratives to construct a Disorder--Symptom Bayesian Network (DSBN). Building on the DSBN, we propose ConsultMind, an uncertainty-aware framework that updates disorder posteriors after each response and uses posterior uncertainty to guide inquiry and diagnosis. We evaluate both methods across psychiatry, respiratory medicine, fever clinics, and three public datasets. The results show that AutoDisym can automatically construct high-quality DSBNs and that ConsultMind consistently improves diagnostic performance and explanation soundness. For example, AutoDisym achieves macro-averaged F1 scores of 81.37 for canonical symptoms and 72.19 for manifestations using GPT-5.6-Sol. ConsultMind improves Top-1 and Top-3 diagnostic accuracy by up to 22.15 and 37.89 percentage points, respectively. Physician evaluation further shows that ConsultMind improves the quality of ranking explanations, differential diagnoses, and diagnosis rationales across LLMs of different scales. This work offers a promising approach to automatic diagnostic consultation.