Senter for vitenskapsteori


Numbers for policy: Practical problems in quantification

The course introduces concepts of practice and ethics of quantification, seen as an antidote to inconsiderate uses of numbers both in academia and in society. It shows the pitfalls to be avoided and offers - with examples - tools and recipes for reasonable uses of quantitative methods. The course aims at practitioners, post-doc and PhD students with an interest in the use of evidence for policy.

While this is not a course on impact assessment, it gives impact assessors an extra gear, plus the set of skill needed to tell apart defensible versus indefensible assessments and risk analyses, enabling participants to spot bogus, implausible or irrelevant quantifications. 
The course also invites reflection on the emergence of problems in the system of quality control of science which has in statistical and mathematical modelling a point of intense vulnerability and friction.
Examples from epidemiology to criminology, from pharmacology to psychology, from big data to unethical use of algorithms – will be discussed. Elements of sociology of quantification and ethics of algorithms will also be part of the teaching. 

The course includes an analysis of the genesis of the Cartesian Dream of prediction and control of nature and society thanks to the power of a mathematized science. We discuss the incredible success of the dream, as well as his historical, philosophical and ecological critique.

Technical material will be presented on statistical procedures and malpractices (p-hacking, p-HARKing) and how to address them.

The course takes inspiration from the works of Silvio Funtowicz and Jerome R. Ravetz, the fathers of post-normal science (PNS). Reference books of these authors for the course are about science and society, science and power, and uncertainty and quality.  
PNS-inspired practices advocated and described in the course are:

- Pedigrees for quantification such as NUSAP,

- Sensitivity auditing, quantitative story-telling and ethics of quantification

The course will also include elements of  technical sensitivity analysis.


  1. An introduction (1): The social organization of science by Matthias Kaiser
  2. An introduction (2): The case of the p-test by Andrea Saltelli
  3. On Reductionism by Ragnar Fjelland
  4. Prodromes of post normal science by Silvio Funtowicz
  5. NUSAP’s history and practice by  Jeroen van der Sluijs
  6. When laypeople are right and experts are wrong. Lessons from Love Canal by  Ragnar Fjelland
  7. Gentlemen don’t publish, by Silvio Funtowicz
  8. The licence not to quantify. Qualitative versus quantitative analyses by  Bruna De Marchi,
  9. Ethics of quantification by Andrea Saltelli
  10. Sensitivity analysis by Andrea Saltelli
  11. When numbers are controversial by Jeroen van der Sluijs
  12. Issues with ethics. Hippocratic Oath and policy failures by Matthias Kaiser
  13. Sociology of quantification- trainer to be decided
  14. Ethics of algorithms - trainerto be decided 
  15. Climate controversies by Jeroen van der Sluijs
  16. Bees and pollinators by Jeroen van der Sluijs
  17. Sensitivity auditing. The seven rules with illustrations by Andrea Saltelli
  18. Quantitative story telling by Andrea Saltelli
  19. The now of science by Silvio Funtowicz
  20. Richard Feynman between little science and big science (with documentary) by Silvio Funtowicz


Duration of lessons: Morning: from 0900 to 1300, 4 times 45 minutes (total lecture time180 m per day) with three 15 m breaks. Lunch 13.00 to 14.00. Afternoon group work 14.00-16.00

Supporting material

Excerpts from Ravetz, J., 1971, Scientific Knowledge and its Social Problems, Oxford University Press.

Andrea Saltelli, Silvio Funtowicz, 2017, What is science’s crisis really about? FUTURES, Volume 91, Pages 5-11, https://doi.org/10.1016/j.futures.2017.05.010

Ragnar Fjelland, 2016, When Laypeople are Right and Experts are Wrong: Lessons from Love Canal, HYLE – International Journal for Philosophy of Chemistry, Vol. 22 (2016), 105-125.

Chapter 8 from Winner, L., 1986. The Whale and the Reactor: a Search for Limits in an Age of High Technology. The University of Chicago Press.

Excerpts from E. F. Schumacher, 1973, Small Is Beautiful. Economics as if People Mattered, Harper Perennial 2010.

Saltelli, A., Stark, P.B., Becker, W., and Stano, P. , 2015, Climate Models as Economic Guides. Scientific Challenge or Quixotic Quest? Issues in Science and Technology (IST), Volume XXXI Issue 3, Spring 2015.

Jeroen P. van der Sluijs, 2012, Uncertainty and Dissent in Climate Risk Assessment: A Post-Normal Perspective, Nature and Culture 7(2), Summer 2012: 174–195 © Berghahn Journals doi:10.3167/nc.2012.070204.

Funtowicz, S & Ravetz J, undated, NUSAP - The Management of Uncertainty and Quality in Quantitative Information.

Jeroen P. van der Sluijs, James S. Risbey and Jerry Ravetz, 2005, Uncertainty Assessment of VOC Emissions from Paint in the Netherlands Using the NUSAP System, Environmental Monitoring and Assessment (2005) 105: 229–259, DOI: 10.1007/s10661-005-3697-7.

Saltelli, A., Annoni, P., 2010, How to avoid a perfunctory sensitivity analysis, Environmental Modeling and Software, 25, 1508-1517.  

Saltelli, A., Funtowicz, S., 2014, When all models are wrong: More stringent quality criteria are needed for models used at the science-policy interface, Issues in Science and Technology, Winter 2014, 79-85.

Funtowicz, S.O. and Ravetz, J.R. (1985) Three Types of Risk Assessment: A Methodological Analysis, in: V.T. Covello, J.L. Mumpower, P.J.M. Stallen and V.R.R. Uppuluri (eds) Environmental Impact Assessment, Technology Assessment, and Risk Analysis, pp. 831–48, New York: Springer.

De Marchi, B. (2015) “Risk Governance and the integration of scientific and local knowledge”  Ch. 9, pp. 149-165 in Fra.Paleo, U. (Ed.) Risk Governance. The Articulation of Hazard, Politics and Ecology. Berlin: Springer. ISBN 978-94-017-9328-5.