Competing to classify thyroid nodules on ultrasound images as either malignant or benign, three deep learning models have essentially drawn a tie with four radiologists.
Based on the finding, the authors of the study have judged the algorithms as potential augmenters of human readers while hoping AI grows more proficient at classifying subtypes of thyroid malignancies.
The research was carried out at Seoul St. Mary’s Hospital in South Korea and published online April 30 in European Radiology. Dr. Yangsean Choi and colleagues used 15,400 images from 7,300 patients who underwent ultrasound-guided fine-needle aspiration biopsy at their institution over a 10-year period ending in 2020.
Two of the participating radiologists were experts subspecialized in the task at hand and two were residents in their second and fourth year of training. The team randomly grouped the images into training (n = 12,327) and validation (n = 3,082) sets, as well as independent internal (n = 432; 197 patients) and external (n = 168; 59 patients) test sets. The external data was from Yeoeuido St. Mary’s, a sister hospital in the Catholic University of Korea health system.
The deep learning models bested the radiologists in the internal test set but came up short in the external.
There were no statistical differences in AUCs among the models. The best-performing of them had an AUC of 0.86 with sensitivity of 91.8% and specificity of 73.2% with the internal test set, alongside an AUC of 0.83 with sensitivity of 78.6% and specificity of 76.8% with the external set.
The radiologists’ best AUC scores included 0.76 with the internal set and 0.80 with the external.
In their discussion, Choi et al. comment that their deep learning models demonstrated “comparable diagnostic performance to the four radiologists, suggesting their potential role in augmenting the radiologists’ diagnosis of thyroid nodules. Future studies will have to continue to explore DL models’ performance in classifying subtypes of thyroid malignancies.”
More:
One of the strengths of our study is the large number of pathologically confirmed thyroid nodules collected over a long period of time and the application of three well-known deep learning algorithms for image classification in search of optimal performance. Furthermore, performance was directly compared with those of four radiologists, which allowed the approximation of deep learning models relative to practicing radiologists. Our results demonstrated an efficient pipeline for the detection and classification of thyroid nodules on ultrasound images.”
More coverage of AI with ultrasound imaging:
AI looking handy with 3D abdominal ultrasound
AI boosts performance of novice nurses when assessing pregnant patients via ultrasound
Pairing ultrasound with artificial intelligence reduces unnecessary breast biopsies
AI classifies breast masses found on ultrasound as well as radiologists
AI makes biopsy recommendations comparable to expert radiologists
Reference:
Yeon-Jae Kim, Yangsean Choi, Chan Kwon Jung, et al., “Deep convolutional neural network for classification of thyroid nodules on ultrasound: comparison of the diagnostic performance with that of radiologists.” European Radiology, April 30, 2022. DOI: https://doi.org/10.1016/j.ejrad.2022.110335
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.