Héloïse Eloi-Hammer, exploring the intersection between the design and use of legal artificial intelligence

I started by doing a preparatory course in arts and social sciences. Right from the first year, I fell in love with sociology at first sight, and I knew I wanted to do research in sociology. So I did three years of preparatory studies, then I enrolled at the ENS Paris-Saclay, where I completed a dual bachelor’s degree in economics and sociology. After that, I did a quantitative Master’s 1 programme run in partnership with the ENSAE and the University of Versailles Saint-Quentin. I then went on to do the qualitative Master’s 2 programme in ‘Interdisciplinary Practice in the Social Sciences’ at ENS-Ulm. I wanted to master both quantitative and qualitative methods, whilst retaining the interdisciplinary dimension that I’d enjoyed during my preparatory classes.

I met Dominique Cardon during my Master’s studies; he supervised my first-year Master’s dissertation and then my second-year Master’s dissertation. That’s how I came to know the Médialab. He became my co-supervisor for my PhD, alongside Laurence Dumoulin.

The aim of my thesis was to present a comparative study of the design and use of legal artificial intelligence. There was quite a lot of research on the design of these tools and their use, notably experiments carried out with judges to see how the use of artificial intelligence might influence decision-making. My idea was to study what was happening at the intersection of these two areas. This seemed important to me, as it is a field where professionals can be heavily involved in the development of these tools.

I decided to adopt a particular approach, inspired by Science and Technology Studies, which are highly valued at the Medialab. My idea was to track court rulings, as these are handed down in the courts and then, through open data on court rulings, are transformed into data. It is this data that feeds into the artificial intelligence models, which are then used by legal professionals in the cases they are handling. The aim was therefore to identify this cycle of court rulings and to focus on each stage so as not to overlook tasks that might be considered less important.

During my Master’s degree, I also worked on technical systems: in my first year of the Master’s (M1) on online activism at Greenpeace, and in my second year (M2) on the local algorithms used by Parcoursup. Originally, I wanted to continue my M2 research, but it was a subject that had already been extensively covered. So I turned my attention to the issue of justice, which had received far less attention, particularly in sociology, and which also interested me. I had an interest in the legal field, as several people I knew had studied law and I myself had done a work placement with solicitors whilst at secondary school.

I therefore decided to work on the subject of justice. I soon got in touch with Laurence Dumoulin, who had just launched the Just-IA project, which focused precisely on these themes, and she helped me draft my research proposal. She is a sociologist of law and was also interested in technical systems within the legal field, which complemented Dominique’s work well, as he was more focused on the sociology of science and technology.

I used truly traditional, highly qualitative sociological methods. I conducted a survey based on interviews with the developers of legal artificial intelligence and with the users – in other words, lawyers. I also carried out observational research, both within the companies that produce the tools, within law firms, and in relation to clients’ training policies. I also observed a trial of a tool that was carried out at a law school. And, of course, I carried out analyses and did a bit of archival research; I collected case files produced by lawyers to see how the use of these tools fitted in with the other methods they employed. Otherwise, I relied heavily on reading investigations, case files and reports to supplement my analysis.

The difficulty was that my subject is very hard to access, as the companies that design these tools are private, so there are issues of trade secrets, and there is certain information they are absolutely unwilling to disclose. It was tricky to gain access to the field and find companies willing to let me carry out my investigation. Even once I’d obtained that authorisation, I couldn’t necessarily access very specific information. I think that to overcome this difficulty, you need to know the right person and ask them at the right time.
Sometimes there are also internal issues within companies that further complicate matters. Sometimes investigations are commissioned, and there may be rival factions within organisations – some in favour of, others against, the promotion of new technologies, particularly when it comes to artificial intelligence. Investigations are sometimes used as weapons in these power struggles. And that’s why I think it’s so difficult to gain access.

I would say that the first contribution is that it proposed a cross-examination of the design and use of a device. This is something we do fairly regularly in Science and Technology Studies (STS), but which, in sociology, is not done systematically. I think this is very important, because it helps to deconstruct the idea that, on the one hand, there is technology, and on the other, users who are powerless in the face of what the tools do. On the contrary, my thesis demonstrates that users are very active: they are involved in the design process through testing, and are by no means passive, as they are perfectly capable of repurposing the tools they use.
The second contribution is that my survey of developers showed that referring to ‘legal artificial intelligence’ in the singular did not really make sense, given the diversity of the services produced. For example, in the case of a legal search engine, depending on how it is programmed, it can produce very different results and have very different impacts on legal practice. There is also diversity in terms of usage, meaning that, depending on how professionals utilise the tools, this has a completely different impact on the legal concepts they employ.

My experience at the médialab was a positive one. I was welcomed into the small group of PhD researchers that was already there as soon as I arrived. I was also involved in a research project, which I eventually left quite quickly, but in any case I was made to feel very welcome there.

I think it’s important not to put too much pressure on yourself. When you start a PhD, there are three main requirements: you need to be part of a research project and play an active role in it, produce academic work and get it published, and teach. When you start your first year of your PhD and are given all this information, it’s a bit overwhelming, daunting and makes it feel like an insurmountable task. I think it’s important to focus on your research work. Of course, it’s very useful to teach, publish and be involved in research projects, but I think you have to do each thing in its own time, focus on the research work and let the rest develop gradually as you go along.

Yes, I’ll carry on with my research; I’m actually looking for a contract at the moment. I’ll continue to work on the topics that interest me and apply for positions at the CNRS and as a senior lecturer. I intend to stay on this research path.