Reproducibility in Computational Science

This talk considers the implications from viewing computation as an instrument in scientific discovery. How do we bring greater transparency and verifiability to data-driven research? How can we reasonably compare computational methods that purport to answer the same research question, but give different conclusions? How might variability in software or hardware impact the finding? More fundamentally, what makes computational results trustable? And what do the answers to these questions imply for the dissemination of computational research? How best should data and code be shared, licensed, and made legally reusable? I will present frameworks and examples for addressing these questions, using concepts from statistical inference, cyberinfrastructure, and software engineering.

Readings for this lecture
  1. Reproducible Research in Computational Harmonic Analysis
  2. The Legal Framework for Reproducible Scienti c Research
  3. Reproducibility
  4. Enhancing Reproducibility for Computational Methods

Victoria Stodden

Victoria Stodden is an associate professor in School of Information Sciences at University of Illinois Urbana-Champaign. She is a leading figure in the area of reproducibility in computational science, exploring how can we better ensure the reliability and usefulness of scientific results in the face of increasingly sophisticated computational approaches to research. Her work addresses a wide range of topics, including standards of openness for data and code sharing, legal and policy barriers to disseminating reproducible research, robustness in replicated findings, cyber infrastructure to enable reproducibility, and scientific publishing practices. Stodden co-chairs the NSF Advisory Committee for CyberInfrastructure and is a member of the NSF Directorate for Computer and Information Science and Engineering (CISE) Advisory Committee. She also serves on the National Academies Committee on Responsible Science: Ensuring the Integrity of the Research Process.

Previously an assistant professor of statistics at Columbia University, Stodden taught courses in data science, reproducible research, and statistical theory and was affiliated with the Institute for Data Sciences and Engineering. She co-edited two books released in 2014—Privacy, Big Data, and the Public Good: Frameworks for Engagement published by Cambridge University Press and Implementing Reproducible Research published by Taylor & Francis. Stodden earned both her PhD in statistics and her law degree from Stanford University. She also holds a master’s degree in economics from the University of British Columbia and a bachelor’s degree in economics from the University of Ottawa.