Trustworthy AI in the healthcare sector: Challenges and solutions illustrated by three real-world examples.
Bernd Beckert bernd.beckert@isi.fraunhofer.de
AI in the health sector promises innovations not only in medical care, but also in on-site care and in nursing care. In the future, AI-based patient touchpoints could replace human General Practitioners, for example. In the so-called One Minute Clinic, diagnostics can be performed by an AI that has been trained on the basis of a data set of several hundred thousand cases. In the future, there will also be AI applications that provide primary care physicians in remote or underserved areas with the latest scientific results and provide them with relevant data in an understandable form. And in nursing care, AI can be used to create individual data-based healing plans.
Software companies, medical technology manufacturers and start-ups are constantly bringing new AI applications to market that promise greater efficiency, cost reductions and more time for what is important to practitioners.
But what about the reliability of such systems? What data is being used for training? Do the data reinforce established patterns of discrimination? What does the use of such systems mean for the doctor-patient relationship? What about transparency and explainability? What happens when the system « learns » new patterns based on new data and recommendations change?
Questions like these are being discussed under the heading of « Trustworthy AI. » This is a concept developed in 2019 by a group of high-level experts in the EU and has since been the requirement for AI « made in Europe. » « Trustworthy AI, » as it will be developed in Europe, offers the opportunity to move beyond purely commercial purposes and to develop sustainable and equitable AI. Trust in AI is the key element for the future acceptance of AI, especially in the health sector, be it implemented in Europe or in developing countries.
According to the EU concept, trustworthy AI comprises seven dimensions: Human agency and oversight (1), robustness and safety (2), privacy and data governance (3), transparency (4), diversity, non-discrimination and fairness (5), societal and environmental wellbeing (6), and accountability (7).
Since the concept was formulated, there have been many attempts to translate these dimensions into practical guidelines and checklists to make them applicable in the development and implementation of AI. However, practical examples and best practices are still rare.
What is missing is a systematic analysis of implementation strategies from practical examples which can provide insights into which approaches are the most suitable in which application field. This contribution will address this deficit by analysing three exemplary implementations of AI in the health sector. The exemplary implementations will be a personalized health app (1), a diagnostic image recognition system (2), and a treatment recommendation system based on extensive medical data (3).
Because there is no standardized approach yet, how trustworthy AI can be put into practice in the health sector, this contribution will help to build up expertise in this field and collect the information available. The empirical data will come from projects of cooperation partners and from companies that have reported on their respective strategies in an expert symposium organized by the author and his team in October 2022. From this team, one person is expected to co-author this contribution.
The contribution will provide recommendations on how to develop and implement trustworthy AI in the health sector and it will give recommendations to companies, start-ups, manufacturers and healthcare institutions on what they can do to develop and implement trustworthy AI systems for the benefit of all.