2023 World Conference on Lung Cancer (Posters)-EP06.01. Concordance Between Generative AI GPT-4 and NCCN Guidelines: Biomarker-Driven Treatment Strategies in NSCLC

EP06.01. Concordance Between Generative AI GPT-4 and NCCN Guidelines: Biomarker-Driven Treatment Strategies in NSCLC - PDF(Slides)

Back to course

Pdf Summary

Researchers compared the biomarker-based treatment recommendations for non-small cell lung cancer (NSCLC) between the Generative Pretrained Transformer 4 (GPT-4), an AI system developed by OpenAI, and the 2023 National Comprehensive Cancer Network (NCCN) guidelines. Despite GPT-4's training data predating September 2021, the researchers decided to assess the 2023 guidelines to provide more up-to-date information to patients. They used 12 biomarker descriptors and three prompt templates to query the AI system and determine the concordance with the NCCN guidelines. GPT-4's outputs were scored based on the inclusion of NCCN-concordant treatments.<br /><br />The study found that GPT-4 offered moderately accurate biomarker-driven treatment guidance for NSCLC based on pre-September 2021 data. However, patients were advised to consult healthcare professionals for personalized and accurate medical recommendations. All GPT-4 outputs included at least one NCCN-concordant recommendation, resulting in a 73% concordance rate for first- and second-line biomarker-based treatment options and 100% for the preferred option. None of the outputs were factually incorrect or unrelated to the input.<br /><br />The overall concordance between GPT-4 and NCCN guidelines was 77%, with non-concordance mainly due to outdated information, omission, or alternative therapy options. The study emphasizes the importance of evaluating the concordance between AI-generated advice and clinical guidelines, especially considering the increasing popularity of generative AI and the online search for health-related information.<br /><br />In conclusion, GPT-4 showed moderate accuracy in providing biomarker-driven treatment guidance for NSCLC. While it can be a useful tool, patients should still consult healthcare professionals for personalized and accurate medical recommendations.

Asset Subtitle

Hye Sung Kim

Meta Tag

Speaker Hye Sung Kim

Topic Pathology & Biomarkers: Artificial Intelligence in Pathology

Keywords

biomarker-based treatment recommendations

non-small cell lung cancer

GPT-4

NCCN guidelines

up-to-date information

concordance

personalized medical recommendations

treatment options

outdated information

generative AI