BMJ Health & Care Informatics 2025

Validation of an Artificial Intelligence-based Algorithm for Predictive Performance and Risk Stratification of Sepsis using Real-world Data from Hospitalized Patients: A Prospective Observational Study

Ji-Hyun Kim, KyungHyun Lee, Kwang Joon Kim, Eun Yeong Ha, In-Cheol Kim, Sun Hyo Park, Chi-Heum Cho, Gyeong Im Yu, Byung Eun Ahn, Yeeun Jeong, Joo-Yun Won, Taeyong Sim, Hochan Cho, Ki-Byung Lee

Objective The heterogeneous nature of sepsis renders determining its underlying causes difficult, which may delay diagnosis and intervention. VitalCare-SEPsis Score (VC-SEPS) is a deep learning-based algorithm that predicts sepsis and monitors patient conditions based on electronic medical record data. However, few studies have prospectively compared medical artificial intelligence software algorithms and traditional scoring systems to predict sepsis. This prospective observational study attempted to validate the predictive performance and risk stratification of VC-SEPS for early prediction of sepsis. Methods In this prospective observational study, we collected electronic medical record data from 6,797 patients hospitalised at Keimyung University Dongsan Hospital, Daegu, South Korea. The final version of the analysed set included 6,455 patients, 325 of whom were diagnosed with sepsis. Results The area under the receiver operating characteristic curve of VC-SEPS was 0.880, indicating its superiority over traditional scoring systems. The algorithm performance showed a consistent trend within 24 hours. On patients’ initial admission, the VC-SEPS was associated with the risk of developing sepsis, and the score accurately predicted sepsis by an average of 68.05 min compared with diagnosis time by an operational definition of sepsis. Discussion VC-SEPS could assist medical staff with early diagnosis and intervention in clinical practice by providing a sepsis risk score. Prompt recognition assisting recognition can significantly help shorten the time between recognition and intervention in clinical decision-making processes. Conclusion This study suggests that using a clinical decision support system can help improve hospital workflows as well as the quality of medical care.