External validation of a prognostic model for mortality among patients with non-small-cell lung cancer using the Veterans Precision Oncology Data Commons.

Summary

A prognostic model developed to predict survival in patients with non-small-cell lung cancer maintained strong predictive accuracy when applied to a separate cohort of Veterans. These findings suggest that while the model effectively distinguishes patient risk, local calibration is necessary to ensure precise survival estimates for the Veteran population.

Key result

The Military Health System prognostic model for non-small-cell lung cancer mortality demonstrated strong discrimination in Veterans Health Administration patients, with time-dependent AUCs of 0.788, 0.806, 0.780, and 0.779 for predicting survival at 1, 2, 3, and 5 years.

Abstract

BACKGROUND: There is wide interest in developing prognostic models in non-small-cell lung cancer (NSCLC) due to the heterogeneity of the disease. Models developed at other healthcare institutions may not be directly applicable for patients treated at the Department of Veterans Affairs (VA). External validation of a candidate prognostic model among VA patients would be crucial before it can be implemented to aid clinical decision-making. METHODS: A prognostic model for mortality developed in the Military Health System (MHS) was applied to data from the VA Precision Oncology Data Repository (VA-PODR), which is available to researchers inside and outside the VA at the Veterans Precision Oncology Data Commons (VPODC). Measures of discrimination and calibration were calculated for the MHS model. The MHS model was also refitted in VA-PODR data using the same risk factors to compare the effect of specific factors and predictive performance when the model is developed using VA data. RESULTS: Time-dependent AUC of the MHS prognostic model was 0.788, 0.806, 0.780, and 0.779 for predicting survival at 1, 2, 3, and 5 years following diagnosis, respectively. Significant discrepancies were found between predicted and observed rates of survival, particularly for later years. When the model is refit in VA-PODR data, it achieved cross-validated AUCs of 0.739, 0.773, 0.769, and 0.807 at the same time points, and discrepancies between predicted and observed survival were reduced. CONCLUSIONS: Validation of the MHS prognostic model in VA-PODR demonstrates that its discrimination remains strong when applied to VA patients. Nevertheless, further calibration to VA data may be needed to improve its risk estimation performance. This study highlights the utility of VA-PODR and the VPODC as a national resource for developing analytic tools that are well adapted to the Veteran population.

Key figure