Deep learning technologies for estimating the optimal task performance of medical imaging systems

ABSTRACT
Modern medical imaging systems comprise complicated hardware and sophisticated computational methods.
Given the sheer number of system parameters that impact image quality, the large variety in objects to be
imaged, and ethical concerns, the assessment and refinement of emerging imaging technologies via clinical
trials often is impossible. For these reasons, there is great interest in virtual imaging trials (VITs) that permit the
automated simulation and analysis of clinically relevant imaging experiments. During the development and
refinement of new imaging technologies via VITs, there is an important need for assessing objective image
quality measures (OIQMs) that quantify the best possible utility of the resulting images for different diagnostic
tasks—independent of the ability of the observer (human or algorithm) who interprets the images. In effect, such
OIQMs can reveal the extent to which task-related information is present in imaging data and thus can be
potentially extracted by a human observer or other numerical algorithm that is sub-optimal; this can permit the
identification of opportunities for improved image processing or other technology changes that lead to improved
performance on diagnostic tasks.
The broad objective of the proposed research is to address this challenge by developing the next generation
of open source and modality-agnostic computational methods for computing OIQMs that quantify the best
possible performance of an imaging system—the so-called ideal observer performance—for clinically relevant
tasks. Estimation of the best achievable performance of medical imaging technologies using realistic stochastic
digital object phantoms and clinically relevant diagnostic tasks has been a holy grail for the medical image-quality
assessment field, and the lack of success to date has limited the field to unrealistic object models and tasks for
decades. When employed in VITs, our new methods will permit assessment of the amount of task-relevant
information in image data and will accelerate the refinement and translation of promising new imaging
technologies to the clinic. The Specific Aims of the project are: Aim 1: To develop and validate ambient
generative adversarial networks (AmGANs) for creating ensembles of clinically relevant digital phantoms; Aim
2: To develop methods for estimating the optimal task performance of an imaging technology; Aim 3: To use the
developed tools for assessing deep learning-based image restoration.
The developed computational tools for computing OIQMs will be made open source. This will open entirely
new avenues for assessing and refining emerging medical imaging technologies with a level of rigor and clinical
relevance previously not possible.

Grant Number: 5R01EB034249-03
NIH Institute/Center: NIH
Principal Investigator: Mark Anastasio