Center for Critical Assessment of Genome Interpretation

Genomic data hold the promise of revolutionizing our understanding and treatment of human disease. Multiple
barriers stand between the acquisition of the data and realizing these and other benefits. Rapid accumulation
of genomic data far exceeds our capacity to reliably interpret genomic variation. New developments in artificial
intelligence and machine learning, combined with increased computing power and domain knowledge, provide
hope for the deployment of enhanced computational tools in both basic research and clinical practice. Use of
these methods critically depends upon reliable characterization of their performance.
The Center for Critical Assessment of Genome Interpretation (C-CAGI) will address these needs, through
objective evaluation of the state of the art in relating human genetic variation and health. CAGI has had five
editions since 2010 with 50 challenges posed to the community taken on by hundreds of predictors, leading to
scores of publications about prediction methods and their assessment. We propose for C-CAGI to continue to
advance the field of variant interpretation through the following Specific Aims:
1. Develop community experiments to evaluate the quality of computational methods for interpreting
genomic variation data. C-CAGI will conduct community experiments in which participants make bona fide
blinded predictions of disease related phenotypes on the basis of genomic data. We will engage a diverse
predictor community to spur innovation. The CAGI Ethics Forum will vet studies to ensure that privacy and
sharing maintain the highest standards and will educate the community.
2. Assess the quality of current computational methods for interpreting genomic variation data;
highlight innovations and progress at interactive conferences. Predictions will be evaluated by
independent assessors, who will be supported by new assessment approaches from C-CAGI. Results will be
presented at CAGI experiment conferences with deep technical engagement, which will be interleaved with
reflective CAGIâ meetings that create an environment for a comprehensive evaluation of the field, facilitating
identification of major bottlenecks and problems faced by the current genome interpretation approaches.
3. Broadly disseminate the results and conclusions from the CAGI experiments and analysis. C-CAGI
will outreach to the broader scientific and clinical community through its publications, and the creation of a
calibrated reference integrated into the most common workflows for ready adoption. CAGI will also be
represented at international meetings with presentations and workshops.
4. Operate effectively and responsively. C-CAGI will operate efficiently as it closely interacts with hundreds
of participants. CAGI will build upon a robust information infrastructure that securely facilitates data
dissemination, prediction submission, and assessment.

Grant Number: 5U24HG007346-08
NIH Institute/Center: NIH
Principal Investigator: Steven Brenner