Researchers have demonstrated the use of natural language processing (NLP) to identify urinary-tract stones in positive radiology reports on CT scans of the kidneys, ureter and bladder. While the tool had lackluster sensitivity, it achieved high specificity, which may point to worthwhile utility for extracting data from reams of unstructured text when researchers are building cohorts for epidemiological studies.
Physicians Andrew Yu Li and Nikki Elliot of the Canterbury District Health Board in New Zealand had their results published online in the Journal of Medical Imaging and Radiation Oncology.
They reviewed all CT radiology reports on kidney, ureter and bladder exams conducted over a single calendar year at 833-bed Christchurch Hospital (n = 1,874 reports) using a “locally available” NLP tool to automatically classify reports based on findings for stones. They checked the tool’s performance by comparing it with manual report review.
Li and Elliot found the manual classification beat NLP, 36 percent to 27 percent.
Still, the accuracy of NLP was 85 percent, with a specificity of 95 percent. Sensitivity was 66 percent.
Where the NLP fell short, the stumbles were caused by misspellings, variable syntax, terminology, pluralization and the tool’s inability to exclude clinical request details from the search algorithm.
“Our NLP tool demonstrated high specificity but low sensitivity at identifying CT kidney-ureter-bladder reports that are positive for ureteric stones,” the authors concluded. “This was attributable to the lack of feature extraction tools tailored for analyzing radiology text, incompleteness of the medical lexicon database and heterogeneity of unstructured reports. Improvements in these areas will help improve data extraction accuracy.”
Dave P. has worked in journalism, marketing and public relations for more than 30 years, frequently concentrating on hospitals, healthcare technology and Catholic communications. He has also specialized in fundraising communications, ghostwriting for CEOs of local, national and global charities, nonprofits and foundations.