“In science, progress is possible. In fact, if one believes in Bayes’ theorem, scientific progress is inevitable as predictions are made and as beliefs are tested and refined. Under Bayes’ theorem, no theory is perfect. Rather, it is a work in progress, always subject to further refinement and testing. ”

Nate Silver, American statistician and author/editor

 

The diagnosis of diseases is dependent on patients’ history and physical examination as well as diagnostic test results, with the latter dimensions becoming higher in volume and increased in complexity than ever before. The area of diagnostic reasoning is a potential weak area for clinical practitioners, and any study on the understanding of diagnostic reasoning in human clinicians is welcomed and timely given the growing presence of artificial intelligence in clinical decision making.

The authors studied 553 practitioners (resident and attending physicians as well as nurse practitioners and physician assistants) in outpatient clinics from 8 US states. Four common clinical scenarios (in adult medicine) were studied: pneumonia, cardiac ischemia, breast cancer screening, and urinary tract infection) were studied with the association of positive and negative test results with probabilities of these diseases. An expert panel determined the correct responses for these questions. To calculate the adjustments in the probabilities of diseases after either a positive or negative test result, imputed likelihood ratios were calculated (posttest odds by pretest odds were calculated as probability divided by 1 minus probability).

The results showed that pretest probabilities were overestimated in all four scenarios. In addition, the probabilities of diseases after positive results were also overestimated with pneumonia being the highest at 95%, and others being lower (urinary tract infection at 80%, cardiac ischemia at 70%, and breast cancer at 50%). Overestimates of disease probabilities with negative test results were found to be 50% for pneumonias and 5% for the others. This study then suggested that practitioners overestimated the probabilities of common diseases both before and after testing by a large margin. While pretest probabilities for all these diseases were overestimated, adjustments in probabilities after a positive or a negative result did vary by the test.

In short, overdiagnosis and overtreatment are very common in the practice of medicine due to a myriad of cognitive biases (such as base rate neglect, anchoring bias, and confirmation bias), and perhaps these biases can be neutralized by appropriate use of artificial intelligence tools in order to improve the clinicians’ performance in diagnostic reasoning. As practitioners often do not think in terms of probabilities, perhaps this deficit can be much more of a focus in early stages of professional education in healthcare.

Of note, it is interesting that in this entire manuscript, Bayes’ theorem was barely mentioned (only one time with Bayesian in small case: “There is also a need to improve bayesian adjustment in probability from test results, …”) .

 

The full article can be read here