Trusting Black-Box Algorithms? Ethical Challenges for Biomedical Machine Learning

Based at the intersection of AI ethics and bioethics, this cumulative doctoral thesis investigates the question if and under which conditions we can and should trust black box algorithms used for medical purposes, with a specific focus on psychiatry. To do so, it sheds light on epistemic and ethical questions arising from opaque machine learning techniques in eight independent, peer-reviewed papers that form the core of this thesis and reflect its two-pronged approach: the first four chapters investigate general ethical questions of trust and trustworthiness of medical machine learning, driven by considerations from philosophy and science and technology studies (chapters 3-6), while the remaining four chapters relate abstract theoretical considerations to particular applications of machine learning in psychiatry, drawing also on empirical methods (chapters 7-10). The stage is set in chapter 1, providing a brief introduction to the topic, and chapter 2, which introduces the theoretical and methodological framework of the thesis, embracing an integrated approach to empirically informed bioethical deliberation.
The general part of the thesis starts with chapter 3, which defends the notion of trust in medical machine learning against recent criticism and suggests a novel, dimensional model of trust in the spirit of Daniel Dennett. The following three chapters (4-6) investigate properties of machine learning-based appliances that make them trustworthy. Chapter 4 scrutinizes algorithmic fairness through the lens of William James’ pragmatist theory of truth. Chapter 5 investigates the possibility of explaining and understanding medical machine learning, drawing on Karl Jaspers’ Psychopathology. Chapter 6 argues with Onora O’Neill why transparency as mere disclosure it too little to generate trust in medical machine learning and suggests embracing an approach of intelligent openness instead.
The following four chapters aim to root these conceptual considerations in practice, by looking at specific applications of ML in psychiatry and neuroscience, and by engaging with relevant stakeholders through semi-structured interviews. To provide an insight into the challenges posed by machine learning in mental health, chapter 7 systematizes ethical questions that arise from computational methods employed to diagnose, treat, and predict schizophrenia, following the principlist framework of Tom Beauchamp and James Childress. Checking these considerations against the attitudes of researchers in the field, chapter 8 provides a unique contribution to the existing literature insofar as it is the first article that examines attitudes and expectations of experts on psychiatric machine learning towards ethical questions, drawing on a sample from Germany and Switzerland. Chapter 9 examines these empirical findings further, exploring the impact of machine learning on psychiatric nosology. Finally, chapter 10 gives an outlook to the future by addressing necessary changes in the training of junior doctors, arguing for the ongoing importance of an education informed by historical reflection. Chapter 11 completes the dissertation, summarizing and discussing the different findings in light of each other. It also acknowledges its limitations and provides suggestions for further research.