EMPLOYING AUTOMATED MACHINE LEARNING TO PREDICT BOWEL SURGERY RISK IN IBD PATIENTS POST-CDI: A COMPREHENSIVE ANALYSIS

Background: Machine learning holds great potential in enhancing patient care by early prediction of clinical outcomes, yet the challenge lies in determining the most suitable model from a pool of constantly increasing options. Automated Machine Learning (AutoML) frameworks assist in accelerating model selection by running trials on data using cutting-edge ML models. In the context of our study on C. Difficile Infection (CDI) in Inflammatory Bowel Disease (IBD) patients, we used an AutoML framework on Google and Mayo Clinic’s AI Factory to predict the surgical risk of bowel surgery within a year from their first CDI episode.
Methods: This study included adult IBD patients with a primary CDI diagnosis from 2012 to 2021. The dataset was derived from electronic health records and included features based on patient demographics, comorbidities, care setting, treatment plans, procedures, medications, and laboratory tests. The time frame was 3 months before initial CDI until 1-year after or 1 month prior to bowel surgery, whichever occurred earlier. The outcome was any bowel surgery within 1-year post-initial CDI. We split the data into an 80-20 training-test division for model training and testing.
AutoML was used to run 300 trials across various models including configurations of boosted trees (XGBoost) and neural networks which are fundamental to deep learning and consist of interconnected nodes (neurons) organized in hidden layers for identifying patterns in data. Model selection focused on hyperparameters such as dropout rate, hidden layer count, and size. Performance was assessed using the metrics AuROC, F1-score, and Recall. Sampled Shapley was used for feature importance analysis.
Results: The cohort included 2495 patients with diagnosis of IBD (72.4% UC and 27.1% Crohn’s disease) and CDI out of which 500 underwent bowel surgery within 1 year after initial CDI (median time for 92 days). AutoML identified a neural network model featuring an architecture with 4 hidden layers, a dropout rate of 0.75 to prevent overfitting and utilized dense skip connections for complex pattern recognition as having the best prediction metrics. The model achieved a robust AuROC of 0.946, F1 score of 0.88, with a high precision and recall of 87.9%, with a true-positive rate of 87.88%, indicating its effectiveness in identifying patients at increased risk (Figure 1). Feature importance analysis using Sampled Shapley (Figure 2), identified treatment for first CDI, values of creatinine, bilirubin and oral vancomycin use as significant predictors.
Conclusion: The model identified through AutoML was able to predict bowel surgery risk in IBD patients in within 1 year after initial CDI with high accuracy. These results highlight the potential utility of AutoML in streamlining the identification of suitable machine learning model and parameters for a given dataset.