JMIR Medical Informatics 2021. Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: A Retrospective Study

2021.10.29

With the recent attention in healthcare AI, there have been many attempts to increase the quality of patient treatment using AI models. To further push the horizon of healthcare AI, AITRICS developed a deep learning model that predicts death, sepsis, and acute kidney injury using data from intensive care units (ICU). The model is designed to be clinically applicable and outperforms conventional scoring systems, as well as other machine learning methods. The manuscript regarding this is accepted in JMIR2021 with the title “Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: A Retrospective Study” Given the nature of ICU patients, predicting adverse outcomes are critical for clinicians to rapidly intervene with appropriate treatments. Conventional prediction scores – APACHE [1], SAPS, and MPM - calculate the scores based only on the patients’ status at admission to ICU. However, considering that most ICU patients are in a fragile state, predicting outcomes using time-series data in real time is necessary to providing patients with appropriate treatments in a timely manner. Despite the progress in AI technology, many existing prediction models suffer from two problems. Their performances tend to decrease as the prediction time gets further away from the actual event time. Also, EMR data inevitably contain missing, incorrect, and delayed data. To solve such problems, our model is developed to make predictions at three different times and is tested on data with errors. As a result, our mortality prediction model recorded AUROCs of 0.990, 0.984, and 0.982 for models predicting 12, 6, 3 hours in advance of the actual event time, respectively. AUROCs of sepsis prediction model 2, 4, and 6 hours in advance were 0.766, 0.751, and 0.738, respectively, outperforming the golden standard medical score SOFA [2] and LR, XGB machine learning models. Lastly, the model predicting acute kidney injury showed AUROCs of 0.804, 0.766, and 0.738 in advance of 3, 6, and 12 hours. This outperformed LR and XGB models. We also evaluated model robustness against erroneous data by adding noise and deleting parts of them. The AUROCs of all models except ours decreased when fed with error data - in mortality prediction, AUROCs of LR and XGB decreased by more than a factor of 600 and 200 compared to ours, respectively. The following image shows our model deployed on application. For each model, it can alarm the clinicians about patients who are likely to experience death, sepsis, and acute kidney injury within the selected time. With this model, we look forward to providing the frontier clinicians with trustworthy assistance and patients with more focused treatment. Reference: [1] Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med United States; 1985 Oct;13(10):818–829. PMID:3928249 [2] Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med. 2009;37(5):1649-1654. doi:10.1097/CCM.0b013e31819def97