Use of Machine Learning to Develop a Clinical Prediction Model for Survival in Oral Cancer

Presentation: AHNS-086
Topic: Mucosal - HPV Negative
Type: Oral
Date: Thursday, May 2, 2019
Session: 4:05 PM - 5:00 PM Scientific Session 11 - Biomarkers
Authors: Omar Karadaghy, MD, Matt Shew, Jacob New, Andres Bur
Institution(s): University of Kansas Medical Center

Importance: Oral cancer (OC) is a significant public health concern in the United States and globally. Predicting OC survival through the use of prediction modeling has been underutilized, and the development of a well-constructed model would augment our ability to provide absolute risk estimates for individual patients. Artificial intelligence (AI) has been increasingly used to develop prediction models in health care with promising results, and may serve to improve our current methodology for model development.

Objective: To develop a prediction model for 5 year overall survival in patients OC using a machine learning (ML) approach and compare this prediction model to the current TNM classification system.

Design, Setting, and Participants: Retrospective cohort study of patients diagnosed or treated for OC from the National Cancer Database (NCDB) between 2004 and 2013. Patients were excluded if the treatment was considered palliative or if staging demonstrated T0 or Tis.

Main Outcomes: Primary endpoint was development and comparison of predictive algorithms for 5 year overall survival using ML and TNM classification.

Methods: Patient social, demographic, comorbidity, tumor, treatment facility, treatment type, and outcome information were obtained from the NCDB. ML software, Microsoft Azure Machine Learning Studio, was used to analyze the data. The data was split into an 80/20 distribution for training and testing, respectively. The data was explored using two-class decision forest and model hyperparameters were tuned to optimize performance. The data was scored using the test data set and permutation feature importance scores were used to determine the factors used in the model’s prediction. The performance of the ML model was compared to that of the pathological TNM staging through measures of discrimination using the c-statistic, accuracy, and precision.

Results:  From the cohort of 37,353 eligible OC patients, 5,471 patients were excluded due to missing clinical staging or survival information. Mean patient age was 62 years (SD= 13), 20,609 (65%) were male, and 28,263 (89%) were white. There were 17,547 reported deaths and a 5 year overall survival of 55%. The median time of follow-up was 46 months (range 0-156 months). The ML model identified insurance status, pathological and clinical T classification, age, regional positivity of lymph nodes, and education level as the most significant variables. When applied to the testing data (n=7470), the calculated c-statistic was .79 (95% CI .78 to .80), the accuracy was 72%, and the precision was 70%. In comparison, the calculated c-statistic to assess discrimination of TNM staging system was .61 (95% CI .61 to .62), the accuracy was 60%, and the precision was 59%.

Conclusions: Using ML, a survival prediction model for oral cancer was created using socioeconomic, demographic, clinical, and pathological data for 31,882 patients from the NCDB. The developed prediction model correctly predicted 5-year overall survival better than TNM pathological stage alone in all metrics. The results of this study highlight the potential impact of ML algorithms, which when applied to large patient datasets, can accurately predict important clinical outcomes and may ultimately contribute to improved patient care.