Balancing Evidentiary Value and Sample Size of Adaptive Designs with Application to Animal Experiments

Speaker: Leonhard Held, University of Zürich

Abstract: Reducing the number of experimental units is one of the three pillars of the 3R principles (Replace, Reduce, Refine) in animal research. At the same time, statistical error rates need to be controlled to enable reliable inferences and decisions. This paper proposes to adopt diagnostic likelihood ratios and the diagnostic odds ratio to statistical hypothesis tests and to adjust it for sample size to obtain a novel measure for the evidentiary value of one experimental unit. The experimental unit information index (EUII) is based on power, Type-I error and sample size, and has attractive interpretations both in terms of frequentist error rates and Bayesian posterior odds. We introduce the EUII in simple statistical test settings and show that its asymptotic value depends only on the assumed relative effect size under the alternative. We then extend the definition to adaptive designs where early stopping for efficacy or futility may cause reductions in sample size. Applications to group-sequential designs and a recently proposed adaptive statistical test procedure show the usefulness of the approach when the goal is to maximize the evidentiary value of one experimental unit. A reanalysis of 2738 animal experiments with simulated results from (post-hoc) interim analyses illustrates the possible savings in sample size.

This is joint work with Fadoua Balabdaoui, Saverio Fontana and Samuel Pawel

Biography: Leonhard Held earned his Ph.D. in Statistics at LMU Munich in 1997 under the supervision of Ludwig Fahrmeir. In 2000- 2006 he was Lecturer, Senior Lecturer and Associate Professor for Biostatistics at Imperial College London, Lancaster University and LMU Munich, respectively. In 2006 he joined the University of Zurich (UZH) as Full Professor. He delivered the Armitage Lecture at the University of Cambridge in 2015 on probabilistic forecasting of infectious disease spread. His current research interests are in reproducibility and replicability, clinical trials methodology, evidence synthesis and meta-science. He is founding Director of the UZH Center for Reproducible Science and Research Synthesis and Open Science Delegate of the University Board. 

Hosts: Prof. E. Wit and prof. D. Sulem