Meryem Bezzaz: I am interested in the framing of European energy policies by what is known as the energy trilemma: reconciling security of supply, affordability and environmental objectives. I use R to download large quantities of public documents available online: reports produced by various actors, as well as legislative texts (EU directives, regulations, etc.) on European energy policies. Next, I clean them up (convert PDFs to text files, remove images, etc.) and transform them into data (depending on the case, the unit of analysis can be words, sentences or paragraphs): I create a text database so that I can then analyse it. The next step will be to look for certain keywords, links between the three concepts of the trilemma, but also changes in framing over time and trends according to the types of authors and documents: can we deduce differences in terms of political priorities? This aspect is only part of my thesis: I have also conducted interviews with the people who produced these documents. 

Malo Jan: I am also using CSS techniques to collect and analyse data for my thesis, whose focus is on the evolution of partisan competition on climate issues. In other words, how political parties position themselves on climate issues today compared to a few years ago, in different European countries. To do this, Luis and I collected party communications in a dozen European countries: social media posts, parliamentary speeches, press releases, etc. We're talking about millions of texts in different languages. The next step is to identify, among all these texts, those that deal with environmental and climate issues. For this, I use machine learning: in a small sample of texts, I manually define (or annotate, or code) which texts deal with climate-related issues and which do not. Next, I train a natural language processing (NLP) model that will be able to replicate the decisions I have made. Once the model is working well, it allows me to see to what extent the parties address the issue of climate change in their publications. And then, within this subset, I can more accurately measure party positions: I try to identify different types of discourse on climate issues, for example opposition to or support for climate policies. These new methods give us the means to measure things that we couldn't measure before. 

Luis Sattelmayer: For my part, I do similar things, but on immigration. I am interested in how mainstream parties in Europe engage with this flagship topic of the far right. To do this, I work on the corpus built jointly with Malo (PartySoMe, Party Social Media). I then seek to determine whether the positions taken by mainstream parties on immigration can explain their electoral decline. To do this, I incorporate the measurement obtained (i.e. a position that is more or less favourable to immigration) into an inferential statistical model. 

Selma Sarenkapa: For me, the first step was to collect newspaper archives from two countries, Germany and France (approximately three million articles published in 12 newspapers since 1995). After cleaning up the corpus, I trained a model to automatically detect and classify certain social groups such as women, immigrants and investors. In a second step, I seek to find out how these groups are portrayed in the media: are they mentioned in a positive or negative light? And in what context (what topics do the articles cover)? For this step, it may be necessary to switch back and forth between unsupervised and supervised approaches [see box]: first see what themes the model proposes unsupervised, and whether they make sense, and if not, return to a supervised approach.

Selma: We realised that several of us were using fairly similar methods. So, the idea was to meet other doctoral students from different universities who also employ these approaches to study topics related to political competition and polarisation in political communication, on social media, in traditional media, and in public policy. The aim was to facilitate a productive exchange of ideas on each other's doctoral work.

Luis: We invited doctoral students we already knew and others we didn't know yet: this is a way to create a network. It also allowed us to spread the word that there is a whole group of people at Sciences Po who are interested in Computational Social Sciences, including of course the medialab [represented at these days], but not only. 

Meryem: Beyond the thesis presentations, which showed different methods at different stages of progress and raised different questions, I also enjoyed the discussions with senior researchers such as Alex Kindel (medialab), who reminded us of the importance of constructive validity (the ability of a model to reflect what it is supposed to measure) and predictive validity (the ability to predict future results). Jan Rovny emphasised the importance of a solid theoretical foundation, and Ronja Sczepanski spoke about publishing in computational social sciences. 

Malo: The aim of this course was to train students with basic knowledge of programming to go a little further than what is covered in the master’s programme. Luis and I trained ourselves, but we think this is something that should be part of research training at Sciences Po these days. Over the course of a week, we provided an overview of CSS techniques. We demonstrated how to start from a research question, what can be measured in texts, how to build a corpus and how to collect data online. We also covered the latest advances in analysis, including what artificial intelligence and large language models (BERT, ChatGPT, etc.) can offer. Each day, we covered a different type of method: supervised methods, unsupervised methods [see box], how to convert textual data into digital representation so that it can then be processed using statistical methods, etc. 

Luis: We structured the course into three sessions each day: a first session on theory, a second session demonstrating examples of computer code, and a third session for students to practise using practical cases that we had prepared. For example, we asked them to annotate texts collaboratively, with 20-30 texts to code each, which were then used to train a model to show them how it works. 

The advantage of this one-week format is that students left with a comprehensive understanding of the different concepts and methods, their advantages and disadvantages, and scripts that they can reuse. But such an intense week does not necessarily leave participants enough time to develop their projects. A number of them either come with projects in mind or realise during the week what they could do with these methods in relation to their research interests. Offering a semester-long course would allow for better support for these projects. 

In any case, it was a pleasure to see such enthusiasm (we even had to create a waiting list at one point). This shows that there is a real demand, in addition to the two courses on R that we each taught in the first and second semesters of the master’s programme respectively. 

Luis: There methods are less effective for certain tasks: if we are trying to measure a concept that is difficult to explain (and difficult to detect systematically, even by humans), no matter how advanced and sophisticated the method may be, it will not work. If we choose a supervised method, the concept we are interested in must be well defined.

Malo: Supervised methods are useful to measure social science concepts in large data sets. The advantage really comes with the volume of data, because the goal is to automate something that would take much longer if done manually from start to finish. So, we also need to ask ourselves whether our data volume is sufficient to warrant this type of method. 

In fact, the question is one of available time and the level of commitment we are willing to put into such a task, and the answer varies from person to person. A senior researcher with a substantial budget could pay 30 research assistants to analyse 50,000 texts manually. But as I am a PhD student with limited time to write my thesis and no one to work for me, the trade-off is different. 

Luis: There are ethical issues, but these are the same ones that other colleagues handling other types of data also face: where is the data stored? Depending on the nature of the data collected, can we entrust it to OpenAI or Google servers, or should we do it locally, on our own machines? 

Malo: On the other hand, a distinctive feature of most of the data we process using these types of methods is that it was not originally produced for research purposes. Of course, we collect and format it so that we can analyse it, but we are not the source. Unlike qualitative interviews or surveys, where respondents know that it is for research purposes and give their consent, we collect texts and data that are produced for a completely different purpose than the one for which we use them. When it comes to data from applications, we did not ask the people using these applications if they agreed to their data being made available in a particular format. This raises questions in terms of making the data available [to the rest of the academic community] and the replicability of the research. 

Luis: Finally, the use of large language models has significant energy costs and environmental impacts.

Meryem: The question makes me think of a recent study that Étienne Ollion (sociologist, CNRS research professor) presented during our workshop. With colleagues, he answered the question: "Can generative AI imitate humans in responding to opinion polls?" What they show is that the answers provided by ChatGPT-type bots are biased, and that this bias varies depending on the topics covered. 

Malo: The most recent advances concern the processing of audio or images, or even what is known as multimodal data (simultaneous analysis of images and audio or text). For example, if we want to analyse parliamentary debates, it may be useful to collect not only the content of the speeches, but also the gestures, tone of voice, etc. 

Meryem: Beyond audio and images, will the use of generative AI influence texts, sources and corpora, with perhaps more standardised language? Now that analysis tools are becoming more widely available, will documents become easier to extract and analyse, or will certain players make data more difficult to obtain or monetise access to it? 

Luis: If models are trained on texts created by ChatGPT, which are not always of good quality, these models may lose performance. We could see a vicious circle emerge: AI-generated content, on which AI models are trained to create other content, and so on. 

Malo: All the tasks we've mentioned (scraping, data cleaning, analysis) must be coded. So, at some point, you have to get your hands dirty, so to speak. But with a basic level of statistics and some programming skills, you can quickly understand the main methods and how to use them. And now, tools like ChatGPT can also help generate code. Of course, you still have to know how to use it, understand what it's telling you and to what extent it's what you want. But compared to 2-3 years ago, the cost of entry has fallen significantly. 

Selma: These methods are also becoming more and more accessible because more and more people are sharing what they do [for example on GitHub and HuggingFace – see end of article], offering training, publishing scientific articles on their methods, like Ronja Sczepanski here: so it's becoming easier to understand how it works. 

Luis: Even if you want to analyse manually and not do quantitative analysis, it is possible to automate everything upstream of the analysis, particularly large-scale data collection: instead of manually downloading a thousand PDF documents from a website, a script written in two minutes can do it in a single operation. Similarly, tedious copy-and-paste tasks can be automated. For people who conduct interviews and have to transcribe them, there are also AI-based automatic transcription tools, such as Whisper or Notta. There are more and more softwares, making these methods much more accessible.