Determining a Clinically Applicable Cutoff in AI Algorithms for Predicting Clinical Deterioration: A Workload-Constrained, Alarm-Based Approach
Determining a Clinically Applicable Cutoff in AI Algorithms for Predicting Clinical Deterioration: A Workload-Constrained, Alarm-Based Approach
YangJaewon Jang, Yong Jun Choi, Taeyong Sim, Ki-Byung Lee, Ji-Hyun Kim, Eun Young Cho, Yuhyun Choi, Sungsoo Hong, Bo Mi Jung, Soo-Jeong Kim, Won Gi Hong, and Jae Hwa Cho
Background: This retrospective study introduces an AI-driven VitalCare-Major Adverse Event Score (VC-MAES) developed to predict major in-hospital adverse events and determine optimal cutoff thresholds. VC-MAES was originally developed to predict a composite outcome including unplanned intensive care unit (ICU) transfer, in-hospital cardiac arrest, and mortality; the present study evaluates its performance for the composite of unplanned ICU transfer and in-hospital cardiac arrest. Methods: Patients aged ≥19 years in Yongin Severance Hospital between 1 March 2020, and 31 December 2022, were included. The primary outcome was clinica l deterioration, defined as unplanned ICU transfer or in-hospital cardiac arrest. Secondary outcomes included model performance metrics (area under the receiver operating characteristic (ROC) curve, sensitivity, specificity, F1 score) and optimal alarm frequency (number of alarms per 100 patient-days) across cutoff determination methods (Youden’s index, F1 score, Euclidean distance, and alarm-based approach); they were used to determine optimal cutoff values. 20 July 2026. Results: VC-MAES achieved an area under the ROC curve of 0.895(full evaluable sample) at 6-h intervals, outperforming traditional early warning systems.
