Identifying incidental findings from radiology reports of trauma patients: An evaluation of automated feature representation methods

Abstract Background Radiologic imaging of trauma patients often uncovers findings that are unrelated to the trauma. These are termed as incidental findings and identifying them in radiology examination reports is necessary for appropriate follow-up. We developed and evaluated an automated pipeline to identify incidental findings at sentence and section levels in radiology reports of trauma patients. Methods We created an annotated dataset of 4,181 reports and investigated automated feature representations including traditional word and clinical concept (such as SNOMED CT) representations, as well as word and concept embeddings. We evaluated these representations by using them with traditional classifiers such as logistic regression and with deep learning methods such as convolutional neural networks (CNNs). Results The best performance was observed using word embeddings with CNNs with F 1 scores of 0.66 and 0.52 at section and sentence levels respectively. The F 1 score was statistically significantly higher for sections compared to sentences (Wilcoxon; Z p F 1 score improved significantly from 0.46 to 0.52 when using pre-trained embeddings (Wilcoxon; Z p Conclusion The results show that the best performance was achieved by using embeddings with CNNs at both sentence and section levels. This provides evidence that such a pipeline is capable of accurately identifying incidental findings in radiology reports in an automated manner.