Application of Machine Learning Models to Predict Recurrence After Surgical Resection of Nonmetastatic Renal Cell Carcinoma

BACKGROUND: Predictive tools can be useful for adapting surveillance or including patients in adjuvant trials after surgical resection of nonmetastatic renal cell carcinoma (RCC). Current models have been built using traditional statistical modelling and prespecified variables, which limits their performance. OBJECTIVE: To investigate the performance of machine learning (ML) framework to predict recurrence after RCC surgery and compare them with current validated models. DESIGN, SETTING, AND PARTICIPANTS: In this observational study, we derived and tested several ML-based models (Random Survival Forests [RSF], Survival Support Vector Machines [S-SVM], and Extreme Gradient Boosting [XG boost]) to predict recurrence of patients who underwent radical or partial nephrectomy for a nonmetastatic RCC, between 2013 and 2020, at 21 French medical centres. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: The primary end point was disease-free survival. Model discrimination was assessed using the concordance index (c-index), and calibration was assessed using the Brier score. ML models were compared with four conventional prognostic models, using decision curve analysis (DCA). RESULTS AND LIMITATIONS: A total of 4067 patients were included in this study (3253 in the development cohort and 814 in the validation cohort). Most tumours (69%) were clear cell RCC, 40% were of high grade (nuclear International Society of Urological Pathology grade 3 or 4), and 24% had necrosis. Of the patients, 4% had nodal involvement. After a median follow-up of 57 mo (interquartile range 29-76), 523 (13%) patients recurred. ML models obtained higher c-index values than conventional models. The RSF yielded the highest c-index values (0.794), followed by S-SVM (c-index 0.784) and XG boost (c-index 0.782). In addition, all models showed good calibration with low integrated Brier scores (all integrated brier scores <0.1). However, we found calibration drift over time for all models, albeit with a smaller magnitude for ML models. Finally, DCA showed an incremental net benefit from all ML models compared with conventional models currently used in practice. CONCLUSIONS: Applying ML approaches to predict recurrence following surgical resection of RCC resulted in better prediction than that of current validated models available in clinical practice. However, there is still room for improvement, which may come from the integration of novel biological and/or imaging biomarkers. PATIENT SUMMARY: We found that artificial intelligence algorithms could better predict the risk of recurrence after surgery for a localised kidney cancer. These algorithms may help better select patients who will benefit from medical treatment after surgery.