CAREER: Enabling Continual Multi-view Representation Learning: An Adversarial Perspective

Representation learning techniques attempt to extract and abstract key information (i.e., the features) from raw data to be used in analyses in a wide range of applications, such as cybersecurity, industry, finance, economics, and scientific discovery. As a critical step in machine learning systems, representation learning is meant to be robust in its capacity, regardless of the mutation of raw data due to noises or the variations of raw data caused by capture devices. In the era of big data, representation learning techniques are confronted with new challenges. Massive data collected from different sensors (e.g., the multi-view camera system) or presented in different modalities (e.g., audio-visual-text) have overloaded existing representation learning techniques. In addition, streaming data received from the Internet and sensitive data accumulated over time, such as personal albums and electronic health records, require the established representation learning model to adapt and account for incoming data. This project will develop a robust continual representation learning model to address these challenges. In real-world scenarios where data access is restricted (e.g., sensitive data) or the processing power of devices is limited (e.g., edge and mobile devices), stakeholders will benefit from the adaptive representation learning techniques to enable continual data analyses.

This project seeks to advance the fundamental understanding of continual multi-view robust representation learning by integrating machine intelligence and human knowledge in AI-enabled security contexts. There are three unique contributions. First, the project will investigate multi-view consistency pursuit to fuse knowledge and generate a view-invariant representation robust to domain shifts frequently encountered in real-world data. Second, this research will revisit and explore adversarial learning in multi-view contexts to enable new attack modes, including iterative, cross-view, and induced modes. Generated adversarial samples and training procedures will benefit and empower the acquired multi-view representation learning models to mitigate various forms of artificial noise. Third, new continual learning models will be created through a novel Memory Bounded Search Tree to enable the evolution of multi-view representation learning despite continual streams of data. Furthermore, to reduce the search space and uncertainty related to the data, this research will leverage human knowledge to acquire critical annotations and empirical strategies for the proposed continual learning models.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Award Number: 2528483
Principal Investigator: Ming Shao
Funds Obligated: $366,883
State: MA