Background: In hospitals, Code Blue is an emergency that refers to a patient requiring immediate resuscitation. Over 85% of patients with cardiopulmonary arrest exhibit abnormal vital sign trends prior to the event. Continuous monitoring and accurate interpretation of clinical data through artificial intelligence (AI) models can contribute to preventing critical events.

Objective: This study aims to evaluate changes in clinical outcomes following the use of VitalCare (Major Adverse Event Score and Mortality Score), which is an AI-based early warning system, and to validate the performance of the algorithm.

Methods: A retrospective analysis was conducted by extracting electronic health record data, using a total of 30,785 inpatient cases from general wards and intensive care units. A comparative analysis was performed by setting a 3-month period before and after the system implementation. For clinical evaluation, we measured the incidence rates of Code Blue and adverse events, the proportion of prolonged hospitalization, and the frequency of early interventions. The area under the receiver operating characteristic curve (AUROC) was calculated to assess the performance of the algorithm.

Results: This study demonstrated that, following the implementation of VitalCare, there was a 24.97% reduction in major events such as Code Blue (P=.004) and the proportion of prolonged hospitalization in general wards (P<.05), along with a significant increase in the rate of early interventions. The model performance exhibited superior outcomes compared with traditional scoring systems, with a Major Adverse Event Score AUROC of 0.865 (95% CI 0.857-0.873) and Mortality Score AUROC of 0.937 (95% CI 0.931-0.944).

Conclusions: A well-developed AI-based model that provides high predictive power can contribute to the prevention of major in-hospital events by providing early predictive information to clinicians. Additionally, it plays a crucial role in effectively addressing unmet needs and challenges in terms of human resources and practical procedures.