Numbers for policy: Practical problems in quantification
The course introduces concepts of practice and ethics of quantification, seen as an antidote to inconsiderate uses of numbers both in academia and in society. It shows the pitfalls to be avoided and offers - with examples, tools and recipes for reasonable uses of quantitative methods. The course aims at practitioners, post-docs and PhD students with an interest in the use of evidence for policy.

The course has ended. Slides for all presentations can can be found here.
While this is not a course on impact assessment, it gives impact assessors an extra gear, plus the set of skill needed to tell apart defensible versus indefensible assessments and risk analyses, enabling participants to spot bogus, implausible or irrelevant quantifications.
The course also invites reflection on the emergence of problems in the system of quality control of science which has in statistical and mathematical modelling a point of intense vulnerability and friction.
Examples from epidemiology to criminology, from pharmacology to psychology, from big data to unethical use of algorithms – will be discussed. Elements of sociology of quantification and ethics of algorithms will also be part of the teaching.
The course includes an analysis of the genesis of the Cartesian Dream of prediction and control of nature and society thanks to the power of a mathematized science. We discuss the incredible success of the dream, as well as his historical, philosophical and ecological critique.
Technical material will be presented on statistical procedures and malpractices (p-hacking, p-HARKing) and how to address them.
The course takes inspiration from the works of Silvio Funtowicz and Jerome R. Ravetz, the fathers of post-normal science (PNS). Reference books of these authors for the course are about science and society, science and power, and uncertainty and quality.
PNS-inspired practices advocated and described in the course are:
- Pedigrees for quantification such as NUSAP,
- Sensitivity auditing, quantitative story-telling and ethics of quantification
The course will also include elements of technical sensitivity analysis.
Program
Day one Monday, 27. August
Morning 9.00-13.00
- An introduction (1): The social organization of science, Matthias Kaiser
- An introduction (2): The case of the p-test and science’s reproducibility problems, Andrea Saltelli
- Prodromes of post normal science (1): Silvio Funtowicz
- NUSAP’s history and practice: Jeroen van der Sluijs
Afternoon 14.00-16.00
- Practicum with Jeroen van der Sluijs
Day two, Tuesday, 28. August
Morning 9.00-13.00
- On Reductionism: Ragnar Fjelland
- Prodromes of post normal science (2): Silvio Funtowicz
- Sensitivity analysis: Andrea Saltelli
- The licence not to quantify. Qualitative versus quantitative analyses: Bruna De Marchi
Afternoon 14.00-16.00
- Practicum with Bruna De Marchi
Day three, Wednesday, 29. August
Morning 9.00-13.00
- When laypeople are right and experts are wrong. Lessons from Love Canal: Ragnar Fjelland
- Ethics of quantification: Andrea Saltelli
- When numbers are controversial: Jeroen van der Sluijs
- Gentlemen don’t publish: by Silvio Funtowicz
Afternoon 14.00-16.00
- Practicum with Andrea Saltelli
Day four, Thursday 30. August
Morning 9.00-13.00
- Sociology of quantification: Alexandra Theben
- Sensitivity auditing and quantitative storytelling. The seven rules with illustrations: Andrea Saltelli
- Climate controversies: Jeroen van der Sluijs
- Issues with ethics. Hippocratic Oath and policy failures: Matthias Kaiser
Afternoon 14.00-16.00
- Practicum with Jeroen van der Sluijs
Day five, Friday 31. August
Morning 9.00-13.00
- The now of science: Silvio Funtowicz
- Ethics of algorithms: Alexandra Theben
- Bees and pollinators: Jeroen van der Sluijs
- Scenario design: conflicted facts and values, Mimi Lam
Afternoon 14.00-16.00
- Practicum with Andrea Saltelli
Day six, Saturday 1. September
Morning 10.00-12.30
- General discussion and feed-back from the participants opened by two short presentation by Hans-Martin Füssel and Elodie Thirion: Was the course useful / actionable?
Duration of lessons: Morning: from 0900 to 1300, 4 times 45 minutes (total lecture time180 m per day) with three 15 m breaks. Lunch 13.00 to 14.00. Afternoon group work 14.00-16.00
Supporting material
Excerpts from Ravetz, J., 1971, Scientific Knowledge and its Social Problems, Oxford University Press.
Chapter 8 from Winner, L., 1986. The Whale and the Reactor: a Search for Limits in an Age of High Technology. The University of Chicago Press.
Excerpts from E. F. Schumacher, 1973, Small Is Beautiful. Economics as if People Mattered, Harper Perennial 2010.
Accreditation
Requirements for 3 ECTS version:
a) preparation: reading of course material
b) preparation: submission of 1-2 page text where they explain their "problem" or "concern" or other reflection that fit the course topic
c) attendance
d) short "reflection memo" (1-3 page text) after the course which demonstrates having reflected on its content and assimilated some of its key messages.
Requirements for 5 ECTS version:
a) preparation: reading of course material
b) preparation: submission of 1-2 page text where they explain their "problem" or "concern" or other reflection that fit the course topic
c) attendance
d) submission of outline/topic (10-15 page text) for final essay which demonstrates having reflected on its content, assimilated some of its key messages, and put it to critical use in the context of the student's problem or concern
e) final essay (10-15 page text)
Essays will be marked pass or fail, the course participants themselves are each responsible for course approval at home institution / department.