Numbers for policy: Practical problems in quantification
The course introduces concepts of practice and ethics of quantification, seen as an antidote to inconsiderate uses of numbers both in academia and in society. It shows the pitfalls to be avoided and offers - with examples - tools and recipes for reasonable uses of quantitative methods. The course aims at practitioners, post-doc and PhD students with an interest in the use of evidence for policy.
While this is not a course on impact assessment, it gives impact assessors an extra gear, plus the set of skill needed to tell apart defensible versus indefensible assessments and risk analyses, enabling participants to spot bogus, implausible or irrelevant quantifications.
The course also invites reflection on the emergence of problems in the system of quality control of science which has in statistical and mathematical modelling a point of intense vulnerability and friction.
Examples from epidemiology to criminology, from pharmacology to psychology, from big data to unethical use of algorithms – will be discussed. Elements of sociology of quantification and ethics of algorithms will also be part of the teaching.
The course includes an analysis of the genesis of the Cartesian Dream of prediction and control of nature and society thanks to the power of a mathematized science. We discuss the incredible success of the dream, as well as his historical, philosophical and ecological critique.
Technical material will be presented on statistical procedures and malpractices (p-hacking, p-HARKing) and how to address them.
The course takes inspiration from the works of Silvio Funtowicz and Jerome R. Ravetz, the fathers of post-normal science (PNS). Reference books of these authors for the course are about science and society, science and power, and uncertainty and quality.
PNS-inspired practices advocated and described in the course are:
- Pedigrees for quantification such as NUSAP,
The course will also include elements of technical sensitivity analysis.
- An introduction (1): The social organization of science by Matthias Kaiser
- An introduction (2): The case of the p-test by Andrea Saltelli
- On Reductionism by Ragnar Fjelland
- Prodromes of post normal science by Silvio Funtowicz
- NUSAP’s history and practice by Jeroen van der Sluijs
- When laypeople are right and experts are wrong. Lessons from Love Canal by Ragnar Fjelland
- Gentlemen don’t publish, by Silvio Funtowicz
- The licence not to quantify. Qualitative versus quantitative analyses by Bruna De Marchi,
- Ethics of quantification by Andrea Saltelli
- Sensitivity analysis by Andrea Saltelli
- When numbers are controversial by Jeroen van der Sluijs
- Issues with ethics. Hippocratic Oath and policy failures by Matthias Kaiser
- Sociology of quantification- trainer to be decided
- Ethics of algorithms - trainerto be decided
- Climate controversies by Jeroen van der Sluijs
- Bees and pollinators by Jeroen van der Sluijs
- Sensitivity auditing. The seven rules with illustrations by Andrea Saltelli
- Quantitative story telling by Andrea Saltelli
- The now of science by Silvio Funtowicz
- Richard Feynman between little science and big science (with documentary) by Silvio Funtowicz
Duration of lessons: Morning: from 0900 to 1300, 4 times 45 minutes (total lecture time180 m per day) with three 15 m breaks. Lunch 13.00 to 14.00. Afternoon group work 14.00-16.00
Saltelli, A., Stark, P.B., Becker, W., and Stano, P. , 2015, Climate Models as Economic Guides. Scientific Challenge or Quixotic Quest? Issues in Science and Technology (IST), Volume XXXI Issue 3, Spring 2015.
Jeroen P. van der Sluijs, 2012, Uncertainty and Dissent in Climate Risk Assessment: A Post-Normal Perspective, Nature and Culture 7(2), Summer 2012: 174–195 © Berghahn Journals doi:10.3167/nc.2012.070204.
Jeroen P. van der Sluijs, James S. Risbey and Jerry Ravetz, 2005, Uncertainty Assessment of VOC Emissions from Paint in the Netherlands Using the NUSAP System, Environmental Monitoring and Assessment (2005) 105: 229–259, DOI: 10.1007/s10661-005-3697-7.
Saltelli, A., Funtowicz, S., 2014, When all models are wrong: More stringent quality criteria are needed for models used at the science-policy interface, Issues in Science and Technology, Winter 2014, 79-85.
Funtowicz, S.O. and Ravetz, J.R. (1985) Three Types of Risk Assessment: A Methodological Analysis, in: V.T. Covello, J.L. Mumpower, P.J.M. Stallen and V.R.R. Uppuluri (eds) Environmental Impact Assessment, Technology Assessment, and Risk Analysis, pp. 831–48, New York: Springer.
De Marchi, B. (2015) “Risk Governance and the integration of scientific and local knowledge” Ch. 9, pp. 149-165 in Fra.Paleo, U. (Ed.) Risk Governance. The Articulation of Hazard, Politics and Ecology. Berlin: Springer. ISBN 978-94-017-9328-5.