Short Courses on the Conduct of Reproducible Aging Research with Big Data

ABSTRACT
A sea-change in attitudes towards reproducible research in social science and epidemiology has occurred over
the past 30 years. Reproducibility has moved from a footnote to center-stage and is now recognized as an
essential component of scientific rigor. The concepts of reproducible science relate not only to the capacity to
reproduce the work of a specific study when using the same data, but to the larger ecosystem in which
research is planned, fielded, critiqued, and interpreted. Systemic biases and error-prone research pipelines
both compromise reproducibility and are now recognized to hinder scientific progress. The nature of research
has also evolved, with increasing emphasis on data analysis, growing access to extensive computational
power, and large, complex data sets. Behavioral and social science research on healthy aging faces special
concerns for reproducibility and these concepts should be integral to training on aging research.
The University of California, San Francisco Training in Reproducible Research on Aging for Social
Science and Epidemiology (UCSF-TRASE) program will develop (AIM 1) intensive short courses on
reproducible research perspectives and skills. We propose four brief (3-day), intensive training modules. Each
module combines didactics and experiential learning, with a substantive focus on health disparities and aging,
and methodologic focus on causal inference. Module 1 introduces concepts of reproducibility for research on
population health and aging. This module is appropriate for researchers and consumers of scientific research
and will provide critical evaluation skills relevant for reviewing journal articles and grant applications,
interpreting published findings, and leading research Module 1 assumes a basic scientific research background
but will be accessible to, for example, practicing physicians, science journalists, administrators, as well as
graduate and post-doctoral trainees. Module 2 provides skills for implementing reproducible analyses, such as
pre-registration; statistical coding hygiene and evaluation; transparent reporting; and documentation, including
for collaborative projects. Module 3 addresses fielding primary data collection to foster reproducibility,
considering study design, statistical power, protocol documentation, data quality control. Module 4 provides
training on integrating evidence to enhance reproducibility of scientific advances, e.g., meta-research,
evidence triangulation approaches. Each training module can stand alone or be combined for a more
comprehensive skill set. We emphasize hands-on skills building to learn best practices in the context of
contemporary problems. The modules build on the outstanding foundation in the existing UCSF training
programs, using many activities already demonstrated to succeed in our other training programs, and curating
for the intensive short-course format to provide participants, across career stages, with both conceptual and
technical skills to enhance reproducibility. We propose (AIM 2) to roll out the modules with ongoing evaluation
and refinement and (AIM 3) use multiple dissemination strategies to maximize the impact of the curriculum.

Grant Number: 5R25AG078149-04
NIH Institute/Center: NIH
Principal Investigator: JUNE CHAN