Conférence ESRA ‘Non Probability Samples’ : 16 & 17 mars 2017 à Sciences Po

Sciences Po accueille la conférence internationale ‘Inference from Non Probability Samples’ de l’association européenne European Survey Research Association (ESRA), co-organisée par l’équipe ELIPSS du CDSP et la revue Survey Research Methods. Une quarantaine de personnes participent à l’évènement.

Keynotes
Jelke Bethlehem (Leiden University, The Netherlands)
Andrew Gelman (Columbia University, New York, USA)

Scientific Organizers
Nick Allum (University of Essex, Colchester, UK)
Ulrich Kohler (University of Potsdam, Germany)
Laurent Lesnard (Sciences Po, CDSP, Paris, France)

 #NonProbParis

Objet de la conférence

In survey research, the data collected with a survey are the facts we know, from which we want to infer certain characteristics of a defined population (descriptive inference) or about a parameter of a more general data generating process (causal inference). In probabilty samples, the process that creates the observed data from the population is well known, making descriptive inferential statements from the observed data to the population relatively unproblematic.

While causal statements require further assumptions, probabilty samples ease the inference from respondents to (classes of) human beings in general subject to assumptions being met. Probability samples have thus always been an important criterion for the quality of empirical research with surveys.

However, the role of probabilty samples for survey research have come under pressure from various directions. Unit nonresponse places a question mark on the assumption that a probabilty sample by design remains to be a probability sample in practice. In recent years, economics witnessed an experimental revolution that brought an increased use of experimental designs with highly selective research units away from the field and into computer labs, and this experimental revolution has also reached other disciplines of the social sciences.

The technical innovation of web surveys has made the design of survey experiments very easy in practice although the classic distinction between internal and external validity is still a factor in judging the worth of such experiments for robust causal inference. While the response rates in classic survey modes diminish, self-selected samples of respondents are increasingly available via social networking and specialized internet platforms.

The purpose of the conference is to discuss the possibilities of drawing inferences from social science data without the use of probability samples, or from probability samples with low rates of response. What kind of research questions can be answered using self-selected samples, and what measures can be taken to combat selection bias?

Programme de la conférence

#DAY 1 - 16 MARS 2017

Statistical Issues

Nonprobability sampling as model construction: Expanding beyond the ideal of randomization : Andrew Mercer (Pew Research Center, University of Maryland)

A Partially Successful Attempt to Integrate a Web-Recruited Cohort into an Address-Based Sample : Phillip S. Kott (presenter), Matthew Farrelly, Kian Kamyab (RTI International)

A test of sample matching using a pseudo-web sample : Jack Gambino, Golshid Chatrchi (Household Survey Method Division, Statistics Canada)

Expanding the toolbox: inference from non-probability samples using machine learning : Joep Burger, Bart Buelens, Jan van den Brakel (Department of Methodology, Statistics Netherlands, Heerlen, the Netherlands; Department of Quantitative Economics, Maastricht University, Maastricht, the Netherlands)

Investigation into the use of weighting adjustments for non-probability online panel samples : Dina Neiger, Darren W. Pennay, Andrew C. Ward, Paul J. Lavrakas (ANU Centre for Social Research and Methods, Australian National University; Institute for Social Science Research, University of Queensland; NORC at the University of Chicago; Office of Survey Research at Michigan State University)

A bootstrap method for estimating the sampling variation in point estimates from quota samples : Jouni Kuha, Patrick Sturgis (London School of Economics and Political Science; ESRC National Centre for Research Methods, School of Social Sciences, University of Southampton)

Keynote

Looking for rigor in all the wrong places : Andrew Gelman (Columbia University, New York, USA)

---

#DAY 2 - 17 MARS 2017

Comparison of Probability and Non-Probability Samples

The Accuracy of Online Surveys: Evidence from Germany : Annelies G. Blom, Daniela Ackermann-Piek, Susanne Helmschrott, Carina Cornesse, Christian Bruch, Joseph W. Sakshaug (Department of Political Science, School of Social Sciences, University of Mannheim; Collaborative Research Center 884 ‘Political Economy of Reforms’, University of Mannheim; GESIS – Leibniz Institute for the Social Sciences; University of Manchester, Manchester, UK)

Assessing the accuracy of 51 non-probability online panels and river samples: A re-analysis of the Advertising Research Foundation (ARF) online panel comparison experiment. : Mario Callegaro, Yongwei Yang, Katherine Chin, Ana Villar, Jon A. Krosnick (Brand Studio, Research at Google UK)

Keynote

The perils of non-probability sampling : Jelke Bethlehem (Leiden University, Institute of Political Science)

How to collect non-probability samples

An Empirical Process for Using Non-probability Survey for Inference : Robert Tortora, Ronaldo Iachan (ICF)

Inbound Call Survey (ICS) – A New Methodology : Karol Krotki, Burton Levine, Georgiy Bobashev, Scott Richards (RTI; Reconnect Research)

In search of best practices : Sander Steijn, Joost Kappelhof (The Netherlands Institute for Social Research/SCP)