As AI becomes increasingly embedded in healthcare, questions about privacy, data protection and patients’ rights are becoming more urgent. Through its AI in the Healthcare Project, Liberties and its partners, together with independent expert Júlia Keserű, are investigating how AI systems are being used in healthcare across Europe, who controls the sensitive data they rely on, and whether patients’ rights are adequately protected. In this interview, Júlia discusses the project's findings, the power structures shaping health AI, and what it will take to make these technologies more transparent and accountable.

What is your background and your current role in this project?

My career has been rooted in civil society from the start. I've spent most of my career at the intersection of data-driven technologies, social justice and human rights activism. I’ve tried to understand how emerging technologies shape activism and society, and how activists can respond, and even lead the way, in showing how technology can be a force for good.

I began my career in Hungary, as part of what was then a nascent open government movement. From there I spent a few years in Washington DC, at the Sunlight Foundation, which combined data-driven advocacy and journalism to push the US government towards stronger disclosure standards. Later I joined the Engine Room, a global organisation that helps civil society use technology and data responsibly, through both hands-on support and knowledge creation. While at the Engine Room, I was involved in various practitioner research commissioned by INGOs, intergovernmental institutions and foundations, who wanted to understand a very specific slice of the emerging technology ecosystem: what their moral responsibility was in the space, what ethical implications they had to consider, and what that meant in practical and technical terms.

Register to the research launch webinar (24th November) - Auditing Health AI: Money, Rules, and Patient Data in the EU

We looked at a wide range of emerging technologies: blockchain, biometrics, digital IDs, predictive analytics, and, increasingly, what is being called AI, but is really a set of very diverse technologies under one label. In each case we tried to understand how these technological solutions affect certain communities or play into thorny social issues.

More recently, I spent two years as a Mozilla Foundation Senior Tech Policy Fellow. There I began focusing on what I call the excess collection of bodily data. That work drew me toward projects that uncover how the tech industry is collecting more and more information about our bodies, and how research and better policy can protect the digital integrity of the body. Above all, I’m interested in how we can get people to genuinely care about the developments happening inside the tech sector and its outsized grip on our lives.

Becoming a parent in the age of mass digital surveillance, AI-generated content, deepfakes and sensitive data leaks made me first think hard about the boundaries of privacy in the digital age, and how I can prepare my daughter for what’s coming, without really knowing what the world will look like by the time she grows up, or what role technology will play in her life. A huge one, that’s for sure.

Do you see AI as being like just another tool, or do you see this as being some kind of drastic shift, a new epoch if you could call it that?

I definitely think AI is a drastic shift, though maybe not for the reasons people usually expect.

First, it’s worth being precise about definitions since ‘AI’ has become an umbrella term for so many different things, including uses of machine learning that were around long before people started throwing the word around. When people talk about AI today, especially its harms, they often mean only generative AI tools; however, AI-adjacent systems, such as predictive algorithms, facial recognition and other machine learning models, have been with us for quite some time.

But what makes this a new epoch isn’t the technology itself. Yes, the science behind it is remarkable, built on decades of research, but that’s not what sets it apart – rather, it’s the extreme concentration of power that has come with it. Very few companies now hold enormous financial, political and technological power, decide where innovation goes, and accumulate resources on a scale we’ve never seen before.

I started my career believing that if people had access to data and digital tools, for example, to see how public officials spend money, our societies would become more democratic and accountable. Looking at where we are now, I don’t think that theory has proven completely wrong: people do have more access to crucial information about how the world works, but it’s increasingly difficult to decide whether data is credible or not. At the same time, the same technologies that gave us better insight into government spending also unleashed enormous harm: in the job market, through mass surveillance and spreading disinformation, by making online abuse possible on a grand scale, and through their burden on the environment. I still believe the same technologies could be used in very different, much better ways (for instance, to keep those in power to account), but the real question right now is whether our societies can bear the burden and the costs of the way they’re being built and deployed right now. I am not so sure they can.

That’s also why I think the public anger and frustration we’re seeing is more than justified. People are reacting to a handful of companies reshaping their jobs, their information environment and their communities without asking them, and to the costs being pushed onto everyone else.

What is your theory of change, including the methodology, on this project?

I became very interested in how the mass collection of bodily data affects our individual and societal wellbeing, partly because of my own experience as a patient. I had cancer not so long ago, and went through treatment at a time when healthcare was digitising fast. Hospitals started using AI tools to predict how a particular tumour is going to respond to a particular treatment, to diagnose sickness earlier, to optimise internal workflows, even to assist with surgeries and plan treatments. We know from anecdotal evidence that some of these tools are clearly being pushed faster than they can be properly evaluated. And yet, we know very little about how hospitals think about the risks of AI beyond meeting EU regulatory requirements. How do they decide which tools to adopt and how to integrate them? How do they weigh costs against benefits? How do they think about harms? When and how do they consult patients on how AI is becoming part of their medical journey? How do they turn research into innovation while staying compliant?

The theory of change for this project was this: when people genuinely understand how AI systems are affecting their medical journeys, they are more likely to trust the process, and the healthcare sector needs that trust if it wants to roll out these new tools and digitise our data. We don’t need to reinvent consent entirely, but we do need much better ways of making that information accessible to patients.

To understand what’s really happening in healthcare, in this research we followed the chain from Brussels down to the bedside. We started with the money: which companies are building and selling health AI in Europe, who funds them, and therefore who shapes how our health data gets used. Then we looked at the law on paper versus in practice: how well the regulations actually protect patients’ rights and data, and what burdens they place on hospitals. With that in mind, we turned to the hospitals themselves: how they approach AI, and how much control they really have over the systems they procure. Finally, we asked how all of this lines up with what patients expect from health AI, and with how their data is shared along the way.

The research we conducted to get our answers draws on a mix of methods. We analysed investment, public procurement and EU grant data, and reviewed the relevant EU legislation alongside existing research. We interviewed more than 50 people, from hospital AI leads and compliance officers to developers and patient advocates. We also used tools more familiar from investigative journalism: freedom of information requests to hospitals in nine EU countries, data access requests to AI companies under the GDPR, and direct inquiries to hundreds of health AI developers.

Who could be interested in following and learning from this project?

Generally, our findings should interest anyone who wants to understand the financial reality of health AI in Europe, including investors, policy analysts and those working on EU tech sovereignty. Our analysis of investment and procurement data shows who actually funds the sector, which companies win hospital contracts, and how far Europe depends on US vendors. That picture often looks quite different from how EU funding strategies present it.

Beyond that, our research speaks to several distinct audiences. One is healthcare professionals in EU hospitals and healthcare institutions, those already experimenting with AI, those who don’t use AI tools yet, and those who don't realise they already do, which happens more often than you think. For them, the research might offer a practical map of a confusing landscape and some highlights of what’s working well in other hospitals and countries, and suggest practical ways to communicate about AI systems to both staff and patients.

Another audience is health AI developers. This research might give them a clearer picture of the European market, who actually wins hospital contracts, how fragmented the single market still is, and how public and private funding shape which tools get built. It also maps where EU rules overlap and contradict each other.

Lawmakers, at both the EU and national levels, are also a key audience, especially for the second phase of the project, when we move from research to advocacy. Regulators matter here too: data protection authorities and the bodies overseeing the AI Act and medical devices are the ones who will turn these rules into practice.

Lastly, we also want to reach civil society organisations, patient advocacy groups and journalists with this project, part of which is about testing accountability tools, like freedom of information (FOI) and data access (DSAR) requests, in a field where they’ve rarely been used before, to audit public interest AI systems.

What could be the short- and mid-term impact of this project?

In the short term, the biggest impact is filling an evidence gap. There’s a lot of talk about AI in European healthcare, but very little hard data on who is actually supplying these tools, how hospitals use them, and what happens to patient data along the way. This is one of the first attempts to map that across several EU countries, and the timing matters as several key decisions are being made right now: the Digital Omnibus changes to the GDPR, the revision of the medical device rules, and the national choices on how the European Health Data Space will work, including exceptions to the patient opt-out. We want our findings in front of the people making those decisions while there’s still room to shape them.

We also want others to pick up the methods. Freedom of information and data access requests are rarely used to scrutinise hospital AI, and we’ve learned a lot about what works, including which kinds of requests are more likely to get a meaningful response from hospitals. I’d love to see organisations across the EU, from journalists to patient groups and digital rights organisations, become confident in auditing healthcare AI tools that rely on sensitive data, and in using the legal avenues that already exist.

In the mid-term, the goal is to move from evidence to change. I think we’ll be able to surface some genuinely creative solutions from hospitals, and we’ll also have a more solid understanding of what responsible AI in healthcare actually means, both under the current legal framework and in an imagined, better one. Honestly, I don’t think future-proof legislation is realistic. But given how hard it is to change laws, especially at the EU level, aiming for something that is at least present-proof feels both achievable and important. In practice, that means advocating for consent mechanisms that give patients real information, and for more transparency about which AI systems are used in clinical care.

Finally, I’d like this work to help shift the public conversation a bit. If more patients start asking their doctors how their data is used and who is involved, hospitals and companies will have to get better at answering.

How can this project help shape the future of AI?

Healthcare is one of the most sensitive places AI is being deployed, so it’s a good test of whether our rules and institutions can actually keep up with its pace. If we can’t protect patients’ data and bodily autonomy in hospitals, where consent and safety are supposed to be sacred, it’s hard to imagine doing better anywhere else.

By showing where regulation works on paper but not in practice, who really holds power in the market, and which accountability tools still work, the project will offer a practical model for making AI systems more transparent and accountable in other high-stakes sectors too. Ultimately, all the projects that I’m interested in and invested in try to help shift the question from “how fast can we adopt AI?” to “who does it serve, and on whose terms?”

Related articles

Register to the research launch webinar (24 / 11) - Auditing Health AI: Money, Rules, and Patient Data in the EU

Safe AI For Patients: Building Accountability in Digital Healthcare