2024 World Conference on Lung Cancer (WCLC) - Posters-P2.05B.02 Automated Extraction of Key Entities from Thorax CT Reports Using NER with Prompt Engineering

Pdf Summary

The study titled "Automated Extraction of Key Entities from Thorax CT Reports Using NER with Prompt Engineering" explores the use of Named Entity Recognition (NER) for extracting critical clinical information from unstructured Turkish Thorax CT reports. Despite the advances in Electronic Medical Records (EMRs), extracting standardized data remains challenging due to terminology inconsistencies, context-dependent meanings, and privacy regulations.<br /><br />The researchers employed a many-shot learning approach utilizing OpenAI's Large Language Models (LLMs), specifically the Google Gemini 1.5 Pro, to automatically extract entities such as anatomy, observation presence, absence, uncertainty, and impressions. The study demonstrated promising results with an overall accuracy of 0.98 and a macro-averaged F1 score of 0.96, indicating proficient performance even for complex entities.<br /><br />The research method involved using 518 example sentences from 100 reports, leveraging a large context prompt that maximizes the model's context window. A focus on many-shot prompting, extensive context, and detailed instructions helped the researchers achieve superior performance over previous models like GPT-4 and BioClinicalBERT.<br /><br />This study highlights the effectiveness of LLMs in non-English NER tasks in medical settings and suggests that prompt engineering can replace model fine-tuning, enhancing clinical workflows, research, and patient care across languages.<br /><br />Additionally, specialized scripts developed in a GitHub repository were used for manipulating NER data, converting annotation files, and calculating performance metrics. The study, approved by Baskent University, indicates that large context and detailed guidelines significantly aid in the accurate classification of clinical entities, thus advancing clinical data extraction methodologies in radiology.

Asset Subtitle

Ozden Altundag

Meta Tag

Speaker Ozden Altundag

Topic Pulmonology & Staging

Keywords

Named Entity Recognition

Thorax CT Reports

Prompt Engineering

Large Language Models

Clinical Information Extraction

Many-shot Learning

Google Gemini 1.5 Pro

Non-English NER

Clinical Data Extraction

Radiology