Andreas BedorfRecent progress in machine learning for healthcare has resulted in major advances in prediction and generative modelling. However, this progress means little if humans and machines do not partner effectively. Other than enigmatic foreign entities, machines have to become fully trusted co-workers of modern clinicians.
What is needed? The key to a productive partnership between AI and the end-user (e.g., the clinician) is an engaging reciprocal relationship. Let us deconstruct what this means:
A machine learning model that appears as a “black box”, i.e., is sufficiently complex that it is not straightforwardly interpretable to humans, can easily be a deterrent to clinicians, as they find it difficult to follow its reasoning and trust its suggestions. After all, clinicians are increasingly looking to base treatment decisions for their patients on machine learning supported data. Interpretable outputs can be more readily understood and trusted, making them more actionable. Furthermore, patients themselves need to be presented with a comprehensible basis for a decision in order to decide whether to accept a proposed treatment course.
Machines are encroaching upon the well-established clinician-patient relationship. It is, therefore, important to ensure that machines are accepted as trustworthy co-workers by the former and as a reliable decision-base for the latter.
What can we do? While there has been some significant progress in the field of explainable AI (XAI), it has largely focused on understanding models, debugging them, and (hopefully) building trust. However, there is a lot more to offer. Here we present a roadmap we are working on for forging human-machine partnerships in the healthcare domain.
Our approach is to develop a unifying framework demonstrating how XAI, and its accomplishments in interpretability and trust-building, can be combined with ways of allowing models to learn from and about humans and their preferences, such as inverse decision modelling.
Earlier, we defined the need for interpretability in healthcare – for the machine to become a trusted co-worker and a solid basis for patient decisions. There are four current state-of-the-art approaches to supervised machine learning interpretability, as well as unsupervised XAI (the user does not know the meaning of the black-box output they are trying to interpret).
Feature-based interpretability
These methods provide explanations in terms of the importance or relevance of the input features either globally or on an instance-wise basis. Popular approaches, such as gradient-based and perturbation-based methods have received plenty of attention, it is insufficient and we need to think beyond feature-based interpretability.
Example-based interpretability
Case-based reasoning has long been used by humans to “explain by example” but only recently have machine learning interpretability methods been proposed that adopt this approach.
Causal interpretability
In medicine in particular, statistical associations are often insufficient – doctors wish to know whether a certain patient characteristic or variable caused the outcome or was the result of the disease. In addition, recommendations of treatments are interventional in nature and require counterfactual explanations.
Equations and meta-models
Sometimes machine learning models are not acceptable to the end-user. For example, The American Joint Committee on Cancer guidelines require prognostic models to be formulated as transparent equations. Despite this, machine learning models can still be useful: meta-modelling can convert black-box models into transparent risk equations.
Unsupervised XAI
Supervised prediction is just one aspect of machine learning and while it has been the focus of XAI, many machine learning methods are unsupervised. Indeed, unsupervised learning is of significant importance in medicine (e.g., phenotyping). Understanding these models is important but currently very challenging.
Typically interpretability methods, including many of those discussed thus far, rely on the supervised target – the user knows the meaning of the black-box output they are trying to interpret. As a result, such methods are not applicable in the unsupervised case.
Recently, machine learning models have been proposed that seek to capture many facets of human decision-making and learning. These include: how humans acquire new information, how they pay attention to such information, how their beliefs may be represented, how their internal models may be structured, how these different levels of knowledge are leveraged in the form of actions, and how such knowledge is learned and updated over time.
What is next? Ultimately, for machine learning to be truly impactful, especially in high-stakes scenarios such as medicine, we will need to form that aforementioned human-machine partnership. Using XAI to explain machines to their human partners and, in return, establishing methods to allow models to learn from and about humans, will “close the loop,” with interpretability as the key interface between human and machine.
This will lead to machines being less of a daunting force encroaching upon the clinician-patient relationship and more of a transformative asset, closing the even bigger loop of medical decision making.
What have we been doing so far? Over the past years, the van der Schaar Lab has put significant effort into paving the road to a more effective human-machine partnership and closing the loop. The ideas and methods developed here will help to produce tools that identify drivers of decision making, establish feedback loops between AI and clinicians for future iterations of decision support tools, and produce increasingly transparent yet sophisticated solutions.
You can find some major milestones in our research here:
Nabeel Seedat
Mihaela van der Schaar
Jeroen Berrevoets
