2023 World Conference on Lung Cancer (Posters)-P1.24. The Paradoxical Behavior of Kappa When Quantifying Agreement in Histological Subtype Classification among Thoracic Pathologists

P1.24. The Paradoxical Behavior of Kappa When Quantifying Agreement in Histological Subtype Classification among Thoracic Pathologists - PDF(Abstract)

Back to course

Pdf Summary

This research presentation at the WCLC 2023 conference discusses the limitations of Cohen's Kappa statistic when quantifying agreement between pathologists in histological subtype classification. The study focuses on lung cancer pathology and examines the strengths and limitations of frozen sections in predicting final histology. Two pathologists independently reviewed frozen sections from stage I lung adenocarcinomas and determined the presence or absence of five histological patterns.<br /><br />The results showed that while there was substantial agreement in four of the five histological patterns based on Cohen's Kappa, the lowest Kappa value was found in the acinar pattern, despite having the highest observed agreement. This indicates a paradoxical result where the Kappa value does not accurately reflect the level of agreement.<br /><br />To address this paradox, the study introduces Gwet's AC1 as a "paradox-resistant" alternative to Cohen's Kappa. Gwet's AC1 values were close to or higher than Cohen's Kappa across all five patterns, providing a more accurate quantification of agreement. These results highlight the importance of using alternative measures in interrater agreement studies, especially in settings with high feature prevalence.<br /><br />The study concludes that future interrater agreement studies should report Gwet's AC1, as well as observed proportions of positive and negative agreement, in addition to or in place of Cohen's Kappa. This approach ensures a more comprehensive and clinically relevant interpretation of interrater reliability studies in thoracic pathology.<br /><br />Overall, this research highlights the limitations of Cohen's Kappa and introduces an alternative measure that can provide more accurate results in histological subtype classification agreement among pathologists.

Asset Subtitle

Kay See Tan

Meta Tag

Speaker Kay See Tan

Topic Pathology & Biomarkers: Prognostic Biomarkers

Keywords

Cohen's Kappa statistic

agreement

pathologists

histological subtype classification

lung cancer pathology

frozen sections

Gwet's AC1

alternative measure

inter-rater agreement studies

interrater reliability studies