We discuss the future of clinical trials with clinicians

Randomised controlled trials (RCTs) are the gold standard for evaluating new treatments. Phase I trials are used to evaluate safety and dosage, Phase II trials are used to provide some evidence of efficacy, and Phase III trials are used to confirm the efficacy of the new treatment in comparison with the reference treatment. Traditional free-standing parallel-group RCTs may, however, not always be the most practical option for evaluating certain treatments, since they are costly and time-consuming to implement, and often exclude the majority of patients with a condition from investigation.

RCTs are complex projects involving multiple data sources, design choices and analytical methods along their multiple phases. Improving clinical trials with machine learning is, therefore, one of the most significant and challenging topics in the field.

The above graphic gives an overview about the key stages of a clinical trial and the challenges playing into each of them. For a deeper dive into some of these aspects, have a look at our Research pillar on clinical trials, as well as our recent think piece on the topic.

Given the increasing attention to next-generation clinical trials, powered by AI and machine learning, the van der Schaar lab dedicated two full Revolutionizing Healthcare engagement sessions on 25 May and 7 July 2022 to this topic.

In these sessions, we invited

to first establish the state-of-the-art and challenges in regard to clinical trials, and then discuss possible solutions to these problems.

The first question we asked was:

What is wrong with ‘traditional’ randomised clinical trials?

Clinical trials – difficult, expensive, and ripe for disruption

Our experts, especially Prof Eoin McKinney and Prof Mihaela van der Schaar, established the current ‘gold standard’ of randomised clinical trials as expensive, time-consuming, restricted to the population included in the study, and to be simple grouped mean effect comparisons. A very limited observation of pre-specified subgroups with only small steps to individualised treatments.

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

We conduct clinical trials based on an included population and our inclusion and exclusion criteria are typically almost comically restrictive, so we always get the RCT result that then forms the basis of our clinical practice. Yet, the population that we do perform the trials on is demonstrably not representative of the people we give the medicine to afterwards. Around half of trials exclude over 3/4 of patients with that disease. Yet we still extrapolate that evidence to very different populations: where it’s been systematically evaluated, we see that almost two-thirds of treated patients would have been excluded from the original study showing that treatment’s efficacy.

So whilst we like to think that randomised controlled trials are a gold standard; whilst we like to think they are a firm and robust evidence based – clearly that’s not the case. We perform trials in a small minority of people and then extrapolate it well beyond those confines. That’s arguably not appropriate, and isn’t really the evidence-based medicine we like to think we practice.

Fundamentally, the way clinical trials are run has not changed in decades despite major technological advances. Prof Eoin McKinney establishes the base challenges we are facing: The trials produce significant amounts of data that increase throughout a trial which is hard to handle. Also, while clinical trials are tremendously expensive, the success rate can be rather low, ranging from 20-30% for phase 2 trials, and even phase 3 trials are in the range of 50-70% success rate.

These issues come alongside multiple biases in effect size assessment (particularly using real-world data), no clear way of combining quantitative benefits and quantitative risks, rare hard outcomes, troubles recruiting the right sort and number of patients for trials, and the afromentioned over-extrapolation.

The panel established the following goals any next-generation improvement should include:

• It must improve patient benefit
• It must reduce the required patient numbers by preventing drop-outs
• It must reduce cost-per-patient through improved recruitment, monitoring, and operations
• It must improve the probability of success through better recruitment, prediction, trial design, and analytics

Overall: Clinical trials have to become cheaper, shorter, and more successful.

So far, machine learning has not been systematically used for the design and conduct of clinical trials. However, there have been ML and AI algorithms trialled through software as a medical devise. It has been used to identify biomarkers and for meta-analysis. There is scope for methods to have a much broader impact on the conduct and design of clinical studies, as well as drug development.

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

ML can improve and give us more effective trials that should snowball. Take a simple example and look at the success rate of clinical trials from phase one to phase four that include a biomarker of some sort – there’s a consistent difference in the efficacy rate of those including markers compared to those that do not. It shows that we can do it, we just have to do it more.

How can we conduct more effective clinical trials?

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

I think being able to stratify and make our treatment more individual – and by that I mean whether we’re selecting which people should be treated at which time point they should be treated or with which dose they should be treated – any of those factors could produce individualised therapy. The big challenge for machine learning is whether we can learn what we should before we’ve started the clinical trial. We have prior data on the disease where other interventions have been used and we may have prior information on similar interventions or analogous interventions whether they’re in that disease or in another context.

Important here is the understanding that, even if there is not much data for a specific intervention or disease, that there are similar experiences to draw and learn from – both for the clinician and the ML algorithms.

Prof Carsten Utoft Niemann, MD

Associate Professor and Head of the Chronic Lymphocytic Leukemia (CLL) Laboratory and Clinical Research Program for CLL at Rigshospitalet’s Department of Hematology, Copenhagen; Founder of the PreVent-ACaLL phase 2-3 trial, the first Machine Learning based clinical trial in CLL

I strongly believe that building on prior clinical trial data and on prior real-world data, and laboratory knowledge, we can integrate that into platform trials – essentially turning the full population into clinical trial data having the option to go for what we would consider standard of care at a given time point or to go for the experimental treatment (similar to randomisation) based on their subgrouping.

Using historic and surrogate data is only the first step. Going further, ML generated data can fill in further gaps in the setup of a clinical trial, leading to a smarter design, improved recruitment, and higher success rates.

The above graphic shows where, in the run of a clinical trial, machine learning can help with the creation of a more successful and efficient study. For a more detailed break down please have a look at this article.

This can help us deal with disease specific challenges more effectively.

Prof Pierre Marquet, MD

Head of the ‘Pharmacology, Toxicology and Pharmacovigilance’ Department at the University Hospital of Limoges; Director of the U1248 research unit ‘Pharmacology & Transplantation’ at the French Institute of Medical and Health Research; President of the European Association of Clinical Pharmacology and Therapeutics

Sometimes you do not have biomarkers but you may have statistical or AI based scores for patients that may play the role of biomarkers.

Prof Carsten Utoft Niemann, MD

Associate Professor and Head of the Chronic Lymphocytic Leukemia (CLL) Laboratory and Clinical Research Program for CLL at Rigshospitalet’s Department of Hematology, Copenhagen; Founder of the PreVent-ACaLL phase 2-3 trial, the first Machine Learning based clinical trial in CLL

The aim is to combine data markers and biomarkers – that is an area where I want to put your interest. I think if we can combine these data, both the clinical databases and the biomarker and translational studies, and use them for modelling across different trials, we’ll be actually able to combine data and biomarkers into subgroups with specific risks.

We do not have to stop at biomarkers. They are a great example for where ML can surrogate data and offer a broader picture. Real-World data implementation is then increasing the basis for a smart design even further.

Prof Carsten Utoft Niemann, MD

Associate Professor and Head of the Chronic Lymphocytic Leukemia (CLL) Laboratory and Clinical Research Program for CLL at Rigshospitalet’s Department of Hematology, Copenhagen; Founder of the PreVent-ACaLL phase 2-3 trial, the first Machine Learning based clinical trial in CLL

ML gives us the chance to use real world data as more or less randomised clinical trial by identifying the outcome for a patient similar to the one I have in front of me in the clinic. This could help us using real world data to extrapolate, in a smarter way, the findings from clinical trials and help us to provide a better prediction of what would be the right treatment for a specific patient.

Prof Richard Peck, FRCP

Honorary Professor in Pharmacology and Therapeutics at the University of Liverpool; Former Global Head of Clinical Pharmacology at Roche

We have to talk about disease heterogeneity and multi-morbidity. Most diseases are diagnosed today using 19th and early 20th century technology. We could, instead, be diagnosing diseases using more recent technologies as well, including genetics, omics and molecular biology, and we’d come up with much more specific diagnoses even within the current diagnostic labels. I think that’s one of the key areas of opportunity for me. This is a pattern recognition and classification problem which computers are very good at to allow us to go into much more granularity about the patterns than we were capable of doing 100 years ago.

However, machine learning alone does need these large amounts of data only to then often provide a black box output that is difficult to interpret for clinicians. What we need to do is to marry pharmacological and machine learning models creating a more efficient and more interpretable hybrid model.

Are adaptive trials (next-generation trials) useful?

Our experts talked not only a lot about how to more efficiently use data and improve clinical trials but also how to make them more adaptive – to learn while doing and improving the design along the way.

Prof Richard Peck, FRCP

Honorary Professor in Pharmacology and Therapeutics at the University of Liverpool; Former Global Head of Clinical Pharmacology at Roche

We can start tackling the problem of which patient should we be treating. I hope that this can, to some extent, be informed by what we learn within a trial, adapting it in response to emerging data to learn more about which patients respond best and which biomarkers might be predictive of response.

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

Adaptive trials just make a lot more sense than a conventional trial. You don’t need to wait to the end to get a binary result whether a single intervention has made your pre-specified endpoint. That just doesn’t feel smart enough. Being able to understand what you’re doing as you’re doing it has to be a conceptually better approach.

One problem our experts identified with adaptive trials lies on a higher level – with the regulators. Getting regulatory acceptance for a methodology that is different to what is conventionally done might be problematic, hence why pharma companies just do what regulators demand and adaptive trials might not fit the bill.

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

Beginning a trial without being completely confident that you know the result; if the result has not been pre-specified and you’re going to learn it as you go along – knowing that that’s going to be acceptable for regular approval downstream has to be demonstrated before people will buy into doing it in the first place.

To tackle this problem, it is necessary to structure the trial design comprehensively and to make the adaptive process understandable for regulators – but also to manage risks that come with ‘learning by doing’.

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

For me, perhaps, their success hinges most on an ability to identify, in an a priori way, the right surrogate markers and the right surrogate end points to base the adaptation on. I would be nervous about learning those as you go because, for me, the risk of overfitting to a comparatively small population is quite strong. However, if you can have those defined up front – and ML approaches are ideally placed to define those adaptation circuits using much larger datasets – then an adaptive design could prove very powerful.

How can we improve outcomes for patients?

Classical clinical trials, as mentioned before, usually focus on one intervention at a time. Next-generation adaptive clinical trials could deal with multiple treatments and make the assessment more granular, answering questions like how to treat, when to give the treatment, and when to stop the treatment, as well as reducing risks.

Prof Pierre Marquet, MD

Head of the ‘Pharmacology, Toxicology and Pharmacovigilance’ Department at the University Hospital of Limoges; Director of the U1248 research unit ‘Pharmacology & Transplantation’ at the French Institute of Medical and Health Research; President of the European Association of Clinical Pharmacology and Therapeutics

This is something that can probably be prepared for the clinical trials. How am I going to measure the risk and how am I going to integrate different types of risks, and how do I come up with a definitive score of risk – this could solve many problems and make a difference for some drugs.

Now, we have historical, synthetical, observational data, and electronic health records. We want to design smart clinical trials that are adaptive along the way. How can we approach this without producing black box data?

At the planning stage, SyncTwin can incorporate observational data and results from prior trials, estimate individual effects for a previous intervention or one  that’s similar to the one you’re considering making in your clinical trial, and allow you to identify which individuals might be most appropriate and consequently recruit better. You could estimate individual effects for different doses of an intervention and, consequently, get an idea of what dosage should be used in a subsequent study with regards to conduct. Identifying good subpopulations in the right patient to recruit, and clusters who are responding better. You can understand what are the traits of the individuals that are responding better.

Prof Pierre Marquet, MD

Head of the ‘Pharmacology, Toxicology and Pharmacovigilance’ Department at the University Hospital of Limoges; Director of the U1248 research unit ‘Pharmacology & Transplantation’ at the French Institute of Medical and Health Research; President of the European Association of Clinical Pharmacology and Therapeutics

The SyncTwin algorithm might be very useful to try and run personalised or precision medicine clinical trials. In this way it would be possible to tailor the drug dose or the nature of the drug used based on the predicted outcome or the counterfactual outcome for the patient. That would be a great way to help selecting patients.

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

I find it absolutely fascinating that from a health record data set, the individual effects can be estimated sufficiently accurately to reproduce the same effect seen  in a comparable clinical study. Although, of course, estimating this population level effect is not really what ITE methods are built to do: they can go much further to identify the individual effect of an intervention for each patient, rather than be limited to an average effect in a group of patients.

Further still, they may allow us to estimate not only the effect of a treatment but also the optimal timing of that intervention (by combining prediction with treatment effect estimation). This extends well beyond what clinical trials are designed to do. I can’t think of a clinical trial that has asked the question of when a treatment should stop. But asking whether and when a treatment would lose its effect is something that this can tackle.

How can we transform post-marketing re-evaluation of benefit-risk balance and assess efficacy?

Our experts discussed what is coming after the clinical trial stage, establishing how we focus on safety monitoring while having severe problems at keeping track of changes in efficacy:

• How does efficacy change in the real-world population?
• How is efficacy affected by changing practice? 
• How is efficacy affected by changing comorbidities?
• How is efficacy affected by changing demographics?

Prof Eoin McKinney, MD

University lecturer in renal medicine, University of Cambridge; Honorary consultant in nephrology and transplantation, Cambridge University Hospitals NHS Foundation Trust

The way we treat patients is dynamic. The clinical trial result you get in 2022 may not be the result that you expect to get in 2032, because other treatments for other coexisting conditions will have changed and the context of medicine will have changed. There is scope, using estimation of individual treatment effects in real-world data, to overhaul the ad-hoc and clumsy way we conduct post-market surveillance at the moment. We could serially estimate treatment efficacy and how it changes over time (and in different subpopulations) by using real-world data.

That feeds forward in letting us know whether our treatments have become more or less efficacious with time and who we should use them on. It can also feed backwards into the next round of trial design, telling us who which patients to focus on and potentially what to test in clinical studies. Systematically collection of clinical data that is usable with federated data access approaches, swarm learning methods or Trusted Research Environments – those are ways in which health data can become more accessible and useful in the future.

Prof Richard Peck, FRCP

Honorary Professor in Pharmacology and Therapeutics at the University of Liverpool; Former Global Head of Clinical Pharmacology at Roche

There is still a problem in understanding what’s the best dose of a drug even at the population level and even for average patients. There is a big opportunity to understand dose post-approval and how that might vary between populations. That’s the point at which a drug actually gets expanded out into a much broader patient population and into phenotypes that would not be eligible for the clinical trial in the first place.

Full recording of our 25 May and 7 July Revolutionizing Healthcare sessions

If you want to learn more about next-generation clinical trials, please have a look at our Research pillar and our summary on how the van der Schaar Lab is trying to revolutionise clinical trials.

Join our community!

If you’re a practicing clinician would like to be part of our ongoing discussions exploring the future of machine learning for healthcare, please sign up for our Revolutionizing Healthcare roundtable series.