.md .pdf

Contents

Statistics 159/259, Spring 2022 Course Summary

Overview

This course teaches “the why and how” of reproducible and collaborative research by combining questions of good computational practice in science, open science and statistical data analysis, in the context of today’s research environment. We will interleave practical topics in software engineering and statistical computing with broader discussions on elements of the philosophy of science and the foundations of statistics.

More details can be found in the syllabus.

Key Resources

• Communication: class Piazza.

• Lectures will be recorded and posted in the Kaltura system (visible via bCourses), but attendance is mandatory. (Much of the pedagogical value of the class is in participating in discussions and code reviews).

• Course readings that are not easy to find free on the web or through the UC Berkeley Library will be posted to bCourses.

• Computing resources

  • We will use Jupyter notebooks. We will start with hosted notebooks on our Stat 159 JupyterHub. Later in the term, we will discuss installing Jupyter on your own device. The JupyterHub server will have all the packages you need pre-installed.

  • The sources for class notes and most other materials are available on github, with a rendered version here.

  • Assignments should be submitted by pull request to your private repositories using the GitHub Clasroom.

  • Whenever you need to work with GitHub, remember to activate GitHub authentication from the JupyterHub, by running the command github-app-user-auth at a terminal and following the instructions. If once authenticated you can’t push to a given repo, it may be that you forgot to add that repo/org to your setup of the authentication app, go here to configure the app’s permissions.

• A note on the Berkeley Library EZProxy: Some of the resources listed here are scientific articles available only behind journal paywalls. If you haven’t already, you should configure your web browser to use the campus library EZProxy so you can access them even if you are working from an off-campus network.

Textbook and supporting materials

While not strictly a textbook for this course, we will rely heavily on the excellent, openly licensed: Research software engineering in Python. We will complement it with these other scientific python resources:

• Katy Huff’s - Effective Computation in Physics.

• Jake van der Plas’ A Whirlwind Tour of Python.

• Stefan van der Walt’s Python Survival Pack and Elegant SciPy Book. The full book and all the notebooks are available.

• Josh Bloom’s Python for Data Science Berkeley Course.

• Lecture notes on scientific python

• Getting started with Python for research, a gentle introduction to Python in data-intensive research.

• Python for Data Analysis, 2nd Edition, by Wes McKinney, creator of Pandas. Companion Notebooks

• Effective Pandas, a book by Tom Augspurger, core Pandas developer.

• And we’ll use these Earth Science resources for our domain focus:

  • Ryan Abernathey’s research computing for Earth Sciences.

  • Brain Rose’s Climate Laboratory.

  • Lisa Tauxe’s Python for Earth Science Students

• Git and git workflows

  • Introduction to Git.

  • Immersion course

  • git-scm guide

  • Statlab development git workflow

• Continuous integration

  • GitHub actions

• Miscellaneous computing tutorials

  • Berkeley Statistical Computing Facility tutorials

Other bibliography

Above are a list of books and websites mostly focusing on computational skills, and this is a list of all the bibliography we’ll refer to in the course. Some of these will become assigned readings, while others are available for your reference.

PLOS Ten Simple Rules

The PLOS Ten Simple Rules collection has many short, valuable papers full of relevant, practical advice in this space. A few that stand out, though many (if not most) are worth your time, are “Ten simple rules for …”:

Computational research

• making research software more robust.

• writing and sharing computational analyses in Jupyter Notebooks.

• Effective Computational Research.

• Reproducible Computational Research.

Open Source Software and Open Science

• Taking Advantage of Git and GitHub.

• the Open Development of Scientific Software.

• documenting scientific software.

• helping newcomers become contributors to open projects.

• Cultivating Open Science and Collaborative R&D.

Data Management

• the Care and Feeding of Scientific Data.

• Digital Data Storage.

• responsible big data research.

The art of research

• Effective Statistical Practice.

National Academies Reports

These are key reports produced by the National Academies of Science, Engineering and Medicine. They were created by teams of world experts in the field, and inform policy in multiple areas:

• Reproducibility and Replicability in Science, 2018. The previous link contains multiple resources on this topic, including overview videos, from a large effort comissioned by the National Academies of Science, Engineering and Medicine. For reading, this NCBI link has both HTML and PDF download options.

• Statistical Challenges in Assessing and Fostering the Reproducibility of Scientific Results, 2016.

• Open Source Software Policy Options for NASA Earth and Space Sciences, 2018.

• Open Science by Design, Realizing a Vision for 21st Century Research (2018).

• Developing a Toolkit for Fostering Open Science Practices, 2021

Other general references on reproduciblity and open science

• Millman and Pérez 2014, Developing open source scientific practice.

• Keith Baggerly and the Potti & Nevins Cancer Scandal:

  • 2012, 60 minutes segment about Duke Clinical Trials (also available on vimeo).

  • 2018, The Importance of Reproducible Research in High-Throughput Biology, talk at UW.

• Barba 2016, Top-10 Readings in Reproducibility, a syllabus on reproducible research by Prof. Lorena Barba. Of particular interest from Barba is the Barba 2012, Reproducibility PI Manifesto, with slides available here, as well as

  • Barba 2018, Terminologies for Reproducible Research

  • Mesnard and Barba 2017, Reproducible and Replicable Computational Fluid Dynamics: It’s Harder Than You Think.

  • Mesnard and Barba 2020, Reproducible Workflow on a Public Cloud for Computational Fluid Dynamics.

• Wilson et al, 2012 - Best Practices for Scientific Computing

• Turk 2013, How to Scale a Code in the Human Dimension

• Granger and Pérez 2021, Jupyter: Thinking and Storytelling With Code and Data.

• Vicente-Saenz et al, 2018, Open Science now: A systematic literature review for an integrated definition

• The Limits of Reproducibility in Numerical Simulation.

• The Practice of Reproducible Research, Case Studies and Lessons from the Data-Intensive Sciences. An online (and printed) book produced by Berkeley researchers. It includes the excellent Achieving Full Replication of our Own Published CFD Results, with Four Different Codes.

• Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico. A special issue of Philosophical Transactions of the Royal Society A dedicated to this topic, with multiple valuable articles, of which the following are just a few:

  • Clementi and Barba 2021, Reproducible validation and replication studies in nanoscale physics.

  • Gundersen 2021, The fundamental principles of reproducibility.

  • Krafczyk et al. 2021, Learning from reproducing computational results: introducing three principles and the Reproduction Package.

Reproducibility and earth/climate science

• One of the National Academies reports above commissioned a paper by Bush et al. (2020) titled Perspectives on Data Reproducibility and Replicability in Paleoclimate and Climate Science.

• Liu et al. 2019, improving reproducibility in Earth science research.

• Geyer et al. 2021, Limits of reproducibility and hydrodynamic noise in atmospheric regional modelling.

• Feulner 2016, Science under Societal Scrutiny: Reproducibility in Climate Science.

• Hoffimann et al. 2021, Geostatistical Learning: Challenges and Opportunities

• Gentemann et al. 2021, Science Storms the Cloud.

• Abernathey et al. 2021, Cloud-Native Repositories for Big Scientific Data.

next

Statistics 159/259: Weekly Plan