Artificial intelligence (AI) can assess thoracic disease findings in chest x-rays with higher accuracy than radiologists, according to research published in JAMA Network Open. The same algorithm can also improve reader performance when utilized as a second reader.
The authors developed a deep learning-based automatic detection algorithm (DLAD) for differentiating “major thoracic diseases that are common, clinically important and detectable” on chest x-rays. This included pulmonary malignant neoplasms, active pulmonary tuberculosis, pneumonia and pneumothorax.
The DLAD was trained with more than 53,000 chest x-rays with normal findings and more than 34,000 abnormal chest x-rays showing one of the identified thoracic diseases. Another 300 normal chest x-rays and 750 abnormal chest x-rays were used as a “tuning data set,” and 300 normal chest x-rays and 789 abnormal chest x-rays were used for in-house validation.
For external validation, the researchers used 486 normal chest x-rays and 529 abnormal chest x-rays. The performance of three physician groups—nonradiology physicians, radiologists and thoracic radiologists—was also examined.
Overall, the DLAD had a median area under the ROC curve of 0.979 for image-wise classification and 0.972 for lesion-wise localization. This was a stronger performance than all three physician groups in terms of both image-wise classification and lesion-wise localization. In addition, when used as a second reader, the DLAD resulted in “significant improvements” in both areas for the physician groups.
“Our algorithm consistently demonstrated high performance across independent data sets, even outperforming physicians, including thoracic radiologists,” wrote Eui Jin Hwang, MD, department of radiology at Seoul National University College of Medicine in South Korea, and colleagues. “Furthermore, we demonstrated improved physician performance with the assistance of the DLAD.”
This shows that their DLAD could help improve radiologist workflow by prioritizing x-rays with abnormal findings, the authors explained.
The team also noted one important strength of their research was that the development data set for the DLAD underwent “extensive data curation by radiologists.” In addition, radiologists “meticulously reviewed” all chest x-rays used in the study. External validation sets were also collected from independent institutions, the authors added, showing that their algorithm’s performance “may be generalized to various populations.”
Michael has more than 16 years of experience as a professional writer and editor. He has written at length about cardiology, radiology, artificial intelligence and other key healthcare topics.