ERI: A New Uncertainty Modeling Framework for Snapshot Compressive Imaging

Computational imaging technologies are increasingly required to operate in real-world applications, offering high-fidelity visual output under complex lighting environments. Among these, snapshot compressive imaging (SCI) is a promising technique that retrieves high-dimensional signals from 2D optically compressed measurements. Incorporating modern AI techniques, SCI has significantly advanced the capabilities of traditional optical sensing in various fields, including hyperspectral imaging, video compression, microscopy, and security. However, despite its promise, SCI remains sensitive and under-explored to uncertainties rooted in its hybrid structure – optical encoders and algorithmic decoders – stemming from imperfect optical hardware, algorithmic overfitting, and unpredictable environmental noise, which limits its deployment across diverse platforms and safety-critical systems. This research project aims to enhance the reliability and robustness of SCI in practical use by developing novel methods to study uncertainties at various system levels. The success of this project benefits a broad range of areas, including computational imaging, signal processing, remote sensing, AI photonics, and machine learning. Along with the proposed research, the project supports a comprehensive education and outreach agenda, encompassing cutting-edge undergraduate and graduate research activities, as well as engaging K-12 students through hands-on experiences with AI and imaging technologies.

The goal of this project is to develop a versatile bilevel optimization framework to study uncertainties in hybrid models that interweave physical optics and deep learning algorithms throughout SCI systems. In collaboration with domain experts, the investigator investigates new scientific knowledge to establish computational and optical underpinnings of SCI by modeling mask, weight, and data uncertainties. The research is unfolded into three thrusts: 1) reasoning mask uncertainties with scalable hyperparameter optimization techniques through a Bayes lens to automatically calibrate models across different hardware, 2) developing new post-training quantization algorithms combined with weight uncertainty and diffusion probabilistic models to combat large model sizes and high-dimensional reconstruction, and 3) building a dual-camera spectral SCI system to capture data noise in the real imaging process. The project bridges the gap between laboratory simulation and real hardware by developing an integrated uncertainty-aware SCI toolbox and collecting a dataset that co-registers compressed measurements with reference images.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Award Number: 2502050
Principal Investigator: Zhiqiang Tao
Funds Obligated: $200,000
State: NY