AI spots dozens of missed incidental pulmonary embolism diagnoses at one hospital

Artificial intelligence spotted dozens of missed incidental pulmonary embolism diagnoses over the course of a year at one medical center, according to an analysis published Monday in JACR.

The investigation was retrospective, utilizing nearly 12,000 CT scans gathered at Duke University Medical Center in 2014. But scientists believe their proprietary algorithm could be prepped for pinpointing these near-misses in real-time clinical work.

“Radiologists must play a role in the design of future care algorithms to manage the balance between sensitivity and specificity in these scenarios,” Benjamin Wildman-Tobriner, MD, with Duke’s Department of Radiology in Durham, North Carolina, and co-authors wrote Feb. 15. “Nonetheless, our data suggest that prospective implementation may identify clinically significant missed [incidental pulmonary embolism] and should be considered.”

Previous studies have indicated that such unsuspected PE prevalence is about 2.6%, with radiologists likely encountering this indication often in a busy clinical practice. Yet, another analysis estimated that more than half of incidental cases went unreported during the initial interpretation, which is critical as most acute PE requires treatment, experts noted.

To address this concern, scientists created an AI algorithm trained on more than 30,000 computed tomography scans gathered from 2,000-plus U.S. hospital sites. Wildman-Tobriner and colleagues took the tool for a test drive, using it to analyze 11,913 CT exams of the chest, abdomen and pelvis for indications other than pulmonary embolism. Natural language processing and a trained abdominal rad helped to further refine the results.

All told, investigators discovered 49 (0.41%) missed incidental pulmonary embolism diagnoses in a single year. That compares to 79 (0.66%) spotted during the initial clinical interpretation, “demonstrating that incidental emboli may be missed nearly as often as they are identified,” the authors said.

“It is important to recognize that algorithms are not perfect, and that most (85.8% [289 of 337]) of the studies identified by our algorithm as possibly containing IPEs were ultimately found to be negative,” Wildman-Tobriner cautioned. “Although a ‘hit rate’ of about 15% may be considered reasonable to catch missed IPE, alarm fatigue is an important consideration, particularly if multiple AI algorithms are implemented, compounding the number of [false positive] cases to be assessed.”

Read more about the investigation in the Journal of the American College of Radiology here.