Retrieval of Radiology Reports Citing Critical Findings with Disease-Specific Customization

Ronilda Lacson*, Nathanael Sugarbaker, Luciano M Prevedello, IP Ivan, Wendy Mar, Katherine P Andriole, Ramin Khorasani
Brigham and Women’s Hospital/Harvard Medical School, 20 Kent St., 2nd Floor, Brookline MA 02445, USA

Article Metrics

CrossRef Citations:
Total Statistics:

Full-Text HTML Views: 3627
Abstract HTML Views: 2341
PDF Downloads: 249
Total Views/Downloads: 6217
Unique Statistics:

Full-Text HTML Views: 1853
Abstract HTML Views: 1268
PDF Downloads: 165
Total Views/Downloads: 3286

© Lacson et al.; Licensee Bentham Open.

open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License ( which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.

* Address correspondence to this author at the Brigham and Women’s Hospital/Harvard Medical School, 20 Kent St., 2nd Floor, Brookline MA 02445, USA; Tel: (617)525-9712; Fax: (617)525-7575; E-mail:



Communication of critical results from diagnostic procedures between caregivers is a Joint Commission national patient safety goal. Evaluating critical result communication often requires manual analysis of voluminous data, especially when reviewing unstructured textual results of radiologic findings. Information retrieval (IR) tools can facilitate this process by enabling automated retrieval of radiology reports that cite critical imaging findings. However, IR tools that have been developed for one disease or imaging modality often need substantial reconfiguration before they can be utilized for another disease entity.


This paper: 1) describes the process of customizing two Natural Language Processing (NLP) and Information Retrieval/Extraction applications – an open-source toolkit, A Nearly New Information Extraction system (ANNIE); and an application developed in-house, Information for Searching Content with an Ontology-Utilizing Toolkit (iSCOUT) – to illustrate the varying levels of customization required for different disease entities and; 2) evaluates each application’s performance in identifying and retrieving radiology reports citing critical imaging findings for three distinct diseases, pulmonary nodule, pneumothorax, and pulmonary embolus.


Both applications can be utilized for retrieval. iSCOUT and ANNIE had precision values between 0.90-0.98 and recall values between 0.79 and 0.94. ANNIE had consistently higher precision but required more customization.


Understanding the customizations involved in utilizing NLP applications for various diseases will enable users to select the most suitable tool for specific tasks.

Keywords:: Critical imaging findings, critical test results, document retrieval, radiology report retrieval.