Tim SchubertCardiovascular diseases (CVDs), e.g. heart failure or stroke, are the leading cause of global mortality and majorly contribute to reduced quality of life. Cardiology is a prime example for the hard challenges awaiting those who develop machine learning (ML) for real-world settings. AI tools can empower clinicians in areas such as prevention, prediction, detection, diagnosis, treatment, and care.
Given its high relevance, the van der Schaar lab has contributed to the field with a variety of research projects, inventing new cutting-edge ML along the way.
To learn more about the potential of machine learning in combating cardiovascular disease, we invite you to watch the recording of our 26 June Revolutionizing Healthcare session where we shine a spotlight on AI and machine learning for cardiology.
Risk predictions, e.g. of mortality risk, allow for the optimisation of treatment plans for CVD patients. Typical risk predictions in cardiology often do not perform well across the whole patient population. Machine learning methods can learn risk predictors agnostically and substantially improve predictions. One of the most exciting advancements in the field of risk prediction is the advent of automated machine learning (AutoML). CVD has been one of the first fields in which we applied AutoPrognosis, a cutting-edge AutoML framework developed by the lab, which optimises the modelling process and makes cutting-edge, interpretable ML accessible for non-domain experts. Employing AutoPrognosis on UK Biobank data, we were able to discover novel CVD risk predictors and improve predictions for subpopulations, e.g. individuals with diabetes.
Cardiovascular disease risk prediction using automated machine learning: A prospective study of 423,604 UK Biobank participants
Ahmed M. Alaa, Thomas Bolton, Emanuele Di Angelantonio, James H. F. Rudd, Mihaela van der Schaar
PLOS One 2019
▶️ Abstract
Identifying people at risk of cardiovascular diseases (CVD) is a cornerstone of preventative cardiology. Risk prediction models currently recommended by clinical guidelines are typically based on a limited number of predictors with sub-optimal performance across all patient groups. Data-driven techniques based on machine learning (ML) might improve the performance of risk predictions by agnostically discovering novel risk predictors and learning the complex interactions between them.
We tested (1) whether ML techniques based on a state-of-the-art automated ML framework (AutoPrognosis) could improve CVD risk prediction compared to traditional approaches, and (2) whether considering non-traditional variables could increase the accuracy of CVD risk predictions.
Making machine learning interpretable is paramount in medicine. Predictions must be clinically sound and meaningful and explainable to clinicians and, most importantly, their patients.
We pioneered several interpretable ML methods. For example, our lab established “Stratified Linear Models” (SLIM) to identify different risk predictors for different risk strata in a population of patients with heart failure. These models are based on Trees of Predictors (ToP), an interpretable type of model envisioned by the lab, which outperforms traditional regression methods. With ToP, we were able to identify new significant predictors, such as rales and shortness of breath at rest, particularly in high-risk patients. SLIM provided more accurate mortality predictions and better identification of risk predictors across strata.
Interpretable Machine Learning Identifies Risk Predictors in Patients With Heart Failure
William Zame, Jinsung Yoon, Folkert Assenbergs, Mihaela van der Schaar
Circulation 2018
▶️ Abstract
Introduction: Accurate identification of mortality risk and risk predictors is vital to optimize the management and treatment of patients with heart failure. Our goals were to predict risk of 1-year mortality, to stratify patients according to risk, and to identify risk predictors for patients within each risk stratum.
Hypothesis: That a novel machine learning method can be used to identify risk strata and identify risk predictors within each risk stratum.
Methods: “Stratified Linear Models” (SLIM) is a machine learning method built on the principle that different risk predictors are most relevant for different risk strata within a population. We apply this method to the Meta-analysis Global Group in Chronic Heart Failure (MAGGIC) dataset of 30,389 patients who have had heart failure, of whom 5,723 died within one year.
Results: SLIM achieves better predictive performance than the MAGGIC risk score for mortality among patients with heart failure. Among the risk predictors used by the MAGGIC score, SLIM discovers that some are of very different importance within different risk strata. For example, the New York Heart Association (NYHA) class has high predictive importance within the 20% of the population at lowest risk but low predictive importance within the 20% of the population at highest risk. SLIM also discovers that Rales and Shortness of Breath at Rest, neither of which is used in the MAGGIC score, have higher predictive importance within the 20% of the population at highest risk than any of the risk predictors used by the MAGGIC score. The relative importance of risk predictors within risk groups is shown in Figure 1; lighter colors indicate higher positive predictive value within the indicated risk quintiles.
Survival prediction, e.g. before and after heart transplantation, can inform transplantation and treatment decisions based on individual predictions for patients on a transplant waitlist. Better predictions prior to heart transplantation may also increase the number of successful transplantations. A Newsweek article strikingly highlights how important it is to be able to precisely determine which patients need heart transplants most urgently, and how “scarily” accurate our methodology is.
We were able to improve individualised pre- and post-transplant survival predictions using interpretable ToPs (Trees of Predictors). Crucially, this has been one of the first interpretable models ever built, which highlights our role as pioneers in the field. ToPs discover specific clusters within patient populations and the optimal predictive model for these clusters. Factoring in the differences between clusters within the same patient population, offers a more personalised, accurate approach that can enhance decision-making for patients, clinicians, and policymakers, not only in cardiology but across medical specialties.
Personalized survival predictions via Trees of Predictors: An application to cardiac transplantation
Jinsung Yoon, William Zame, Amitava Banerjee, Martin Cadeiras, Mihaela van der Schaar
PLOS One 2018
▶️ Abstract
Background
Risk prediction is crucial in many areas of medical practice, such as cardiac transplantation, but existing clinical risk-scoring methods have suboptimal performance. We develop a novel risk prediction algorithm and test its performance on the database of all patients who were registered for cardiac transplantation in the United States during 1985-2015.
Methods and findings
We develop a new, interpretable, methodology (ToPs: Trees of Predictors) built on the principle that specific predictive (survival) models should be used for specific clusters within the patient population. ToPs discovers these specific clusters and the specific predictive model that performs best for each cluster. In comparison with existing clinical risk scoring methods and state-of-the-art machine learning methods, our method provides significant improvements in survival predictions, both post- and pre-cardiac transplantation. For instance: in terms of 3-month survival post-transplantation, our method achieves AUC of 0.660; the best clinical risk scoring method (RSS) achieves 0.587. In terms of 3-year survival/mortality predictions post-transplantation (in comparison to RSS), holding specificity at 80.0%, our algorithm correctly predicts survival for 2,442 (14.0%) more patients (of 17,441 who actually survived); holding sensitivity at 80.0%, our algorithm correctly predicts mortality for 694 (13.0%) more patients (of 5,339 who did not survive). ToPs achieves similar improvements for other time horizons and for predictions pre-transplantation. ToPs discovers the most relevant features (covariates), uses available features to best advantage, and can adapt to changes in clinical practice.
Conclusions
We show that, in comparison with existing clinical risk-scoring methods and other machine learning methods, ToPs significantly improves survival predictions both post- and pre-cardiac transplantation. ToPs provides a more accurate, personalized approach to survival prediction that can benefit patients, clinicians, and policymakers in making clinical decisions and setting clinical policy. Because survival prediction is widely used in clinical decision-making across diseases and clinical specialties, the implications of our methods are far-reaching.
Competing risks in patients with (potential) co-morbidities complicate the assignment of optimal treatments. For example, the risk of a cardiac disease can determine the mortality risk of a cancer patient undergoing surgery. Adjusting for competing risks is crucial to accurately estimate the probability of the event of interest. Deep multi-task Gaussian process (DMGP) models, a technique developed by the lab, can jointly assess a patient’s risk for multiple (competing) adverse outcomes. We demonstrate the approach for CVD and cancer. Taking competing risks into account makes this strategy superior to state-of-the-art survival models and has higher utility in real-world settings.
Deep Multi-task Gaussian Processes for Survival Analysis with Competing Risks
Ahmed M. Alaa, Mihaela van der Schaar
NeurIPS 2017
▶️ Abstract
Designing optimal treatment plans for patients with comorbidities requires accurate cause-specific mortality prognosis. Motivated by the recent availability of linked electronic health records, we develop a nonparametric Bayesian model for survival analysis with competing risks, which can be used for jointly assessing a patient’s risk of multiple (competing) adverse outcomes.
The model views a patient’s survival times with respect to the competing risks as the outputs of a deep multi-task Gaussian process (DMGP), the inputs to which are the patients’ covariates. Unlike parametric survival analysis methods based on Cox and Weibull models, our model uses DMGPs to capture complex non-linear interactions between the patients’ covariates and cause-specific survival times, thereby learning flexible patient-specific and cause-specific survival curves, all in a data-driven fashion without explicit parametric assumptions on the hazard rates.
We propose a variational inference algorithm that is capable of learning the model parameters from time-to-event data while handling right censoring. Experiments on synthetic and real data show that our model outperforms the state-of-the-art survival models.
For many medical disciplines, several related but different databases exist. ML models typically require large amounts of data during training. Translating from one dataset to the other to jointly use them for modelling can improve the predictive performance of ML models.
Our approach, RadialGAN, allows for related datasets to be jointly used for modelling, a breakthrough, especially for settings in which high quality data is rare and fragmented. By solving feature and distribution mismatch, RadialGANs open the door to effective transfer learning. The practical utility of this approach was demonstrated using 14 different heart failure datasets for improved predictive modelling.
RadialGAN: Leveraging multiple datasets to improve target-specific predictive models using Generative Adversarial Networks
Jinsung Yoon, James Jordon, Mihaela van der Schaar
ICML 2018
▶️ Abstract
Training complex machine learning models for prediction often requires a large amount of data that is not always readily available. Leveraging these external datasets from related but different sources is therefore an important task if good predictive models are to be built for deployment in settings where data can be rare.
In this paper we propose a novel approach to the problem in which we use multiple GAN architectures to learn to translate from one dataset to another, thereby allowing us to effectively enlarge the target dataset, and therefore learn better predictive models than if we simply used the target dataset.
We show the utility of such an approach, demonstrating that our method improves the prediction performance on the target domain over using just the target dataset and also show that our framework outperforms several other benchmarks on a collection of real-world medical datasets.
Generating synthetic data to augment small datasets and make them accessible for ML methods, is another approach to the problem of low-data settings. We recently introduced Curated LLM (CLLM), which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. To balance the utility of LLMs for data generation against the potential downside of noisy, irrelevant data, CLLM includes a post-generation data curation mechanism, thus offering the best of both worlds. We demonstrate its potential using multiple real-world datasets, one of them the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) dataset.
Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in ultra low-data regimes
Nabeel Seedat, Nicolas Huynh, Boris van Breugel, Mihaela van der Schaar
ICML 2024
▶️ Abstract
Machine Learning (ML) in low-data settings remains an underappreciated yet crucial problem. Hence, data augmentation methods to increase the sample size of datasets needed for ML are key to unlocking the transformative potential of ML in data-deprived regions and domains. Unfortunately, the limited training set constrains traditional tabular synthetic data generators in their ability to generate a large and diverse augmented dataset needed for ML tasks. To address this challenge, we introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. However, not all the data generated by LLMs will improve downstream utility, as for any generative model.
Consequently, we introduce a principled curation mechanism, leveraging learning dynamics, coupled with confidence and uncertainty metrics, to obtain a high-quality dataset. Empirically, on multiple real-world datasets, we demonstrate the superior performance of CLLM in the low-data regime compared to conventional generators. Additionally, we provide insights into the LLM generation and curation mechanism, shedding light on the features that enable them to output high-quality augmented datasets.
Estimations of causal individual treatment effects empower clinicians to optimise treatment plans to the specific patient seeking their help. Average treatment effects in medicine are typically investigated in large-scale, costly randomised controlled trials (RCTs). The van der Schaar lab has developed a groundbreaking machine learning method, SyncTwin, that leverages time-series data in EHRs, constructs a synthetic twin of the patient through various steps, and can estimate individual treatment effects for that patient. Two facts about this breakthrough are specifically noteworthy: 1) an error bound for the estimated effect is calculated, allowing the clinician to accept or reject the recommended treatment with consideration of an interpretable measure of trustworthiness, and 2) SyncTwin is able to successfully reproduce findings of a real-world RCT, the Heart Protection Study (HPS). This demonstrates the power of SyncTwin to provide actionable intelligence for cardiovascular medicine by leveraging EHR data.
SyncTwin: Treatment Effect Estimation with Longitudinal Outcomes
Zhaozhi Qian, Yao Zhang, Ioana Bica, Angela M. Wood, Mihaela van der Schaar
ICML 2024
▶️ Abstract
Most of the medical observational studies estimate the causal treatment effects
using electronic health records (EHR), where a patient’s covariates and outcomes
are both observed longitudinally. However, previous methods focus only on ad-
justing for the covariates while neglecting the temporal structure in the outcomes.
To bridge the gap, this paper develops a new method, SyncTwin, that learns a
patient-specific time-constant representation from the pre-treatment observations.
SyncTwin issues counterfactual prediction of a target patient by constructing a
synthetic twin that closely matches the target in representation. The reliability
of the estimated treatment effect can be assessed by comparing the observed and
synthetic pre-treatment outcomes.
The medical experts can interpret the estimate
by examining the most important contributing individuals to the synthetic twin.
In the real-data experiment, SyncTwin successfully reproduced the findings of a
randomized controlled clinical trial using observational data, which demonstrates
its usability in the complex real-world EHR.
Mihaela van der Schaar
Marika Niihori
