PathCAM: connecting the digital data pipeline in diagnostic pathology with onboard-camera variable resolution slide imaging

Project Summary: This project aims, for the first time, to bring digital pathology image generation and assis-
tance seamlessly into the standard clinical workflow by developing PathCAM, a Computer Assisted Microscope
system for pathologists. The field of digital pathology (DP) at first glance seems poised to make significant im-
pacts in precision medicine. However, the generation of the digital images at the heart of DP is currently outside
of the analog clinical workflow used for the overwhelming majority of clinical slide diagnoses. This is because,
until now, digital image generation in pathology has been relegated to acquisition in expensive and slow whole
slide imaging (WSI) systems, that are outside of the standard clinical workflow. Thus, the current analog micro-
scope workflow will be the prevalent standard for now and the foreseeable future, meaning that any form of
digital assistance for pathologists, via artificial intelligence (AI) or external second opinions, is currently signifi-
cantly disconnected from practice. Moreover, hospitals and patients that do not have access to WSI systems on
a routine basis cannot benefit from these assists. Our low-cost proposed solution is based on a simple insight:
using video feeds from “onboard cameras” commonly installed on clinical microscopes, the camera and associ-
ated software can build an exact digital record of the clinical review. Through our technical innovations, these
images will be created passively, requiring no change in pathologist behavior. Our approach will provide a
pathologist with a real-time preview of what they explored on a slide, giving them feedback on the “whole picture”
of their review and context of their evaluation. Moreover, it allows the direct integration of AI assistance technol-
ogies during clinical review. A transformative aspect to our approach is that images collected during review that
are captured regioselectively from the pathology slide across resolution scales, will be combined into a single
variable resolution image (VRI), which contains an optimal sampling of the tissue section, containing the exact
balance of magnification and resolution needed to render the clinical decision. This optimal image is ideal for
immediate data transfer for a remote pathology consultation – with no waiting for a separate digitization system
to scan the slide. PathCAM’s applicability to both real-time digital assistance and remote second opinions will be
studied in this work. Beyond the direct benefits to clinicians, the data produced by the PathCAM approach can
be transformative for DP technologies going forward. PathCAM can cheaply image data from millions of slide
reviews that would be otherwise be lost, and would enable more equitable data collection via wide and immediate
open-source deployment through our collaboration with Kitware Inc. PathCAM also captures rich information
about how pathologists reach clinical decisions (i.e., diagnostic provenance). These records of how pathologists
clinically search and view slides across resolution scales and relationships between spatiotemporal attention
and diagnosis can support the development of new self-supervised AI algorithms that learn from, but are not
limited by, human-machine interaction data, as well as new human-in-the-loop decision support systems.

Grant Number: 5R01GM143789-04
NIH Institute/Center: NIH
Principal Investigator: JONATHAN BROWN