SBIR Phase II: Real-Time Artificial Intelligence (AI) Bidirectional American Sign Language (ASL) Communication System

The broader impact of this Small Business Innovation Research (SBIR) Phase II project will result from an advanced communication system that uses cutting-edge computer vision and machine learning to enhance communications for the Deaf and Hard of Hearing (DHH) community. By offering expanded communication options beyond traditional methods, the proposed technology seeks to reduce social isolation, foster greater independence, and promote full integration of DHH individuals into the U.S. economy and stimulate economic activity. The technology is expected to unlock opportunities for DHH individuals to access a range of services and platforms, fostering economic growth and enabling businesses and governments to achieve greater productivity. Societally, it will improve educational and employment outcomes for DHH individuals, contributing to increased economic participation.

This Small Business Innovation Research (SBIR) Phase II project addresses the critical need for improved bi-directional communication between DHH and hearing people. The core intellectual challenge lies in developing robust artificial intelligence models given the inherent complexities of modeling a visual language and the scarcity of comprehensive data. Previous research has faced limitations, often resulting in models with inaccuracies, restricted domain applicability, or insufficient effectiveness. Building on successful Phase I efforts, which established technical viability of the proposed approach through novel dataset creation and data augmentation techniques, this project aims to overcome challenges associated with scaling the technology to deal with more complex real-world interactions. The proposed research will focus on refining the models developed in Phase I to create a comprehensive automatic two-way communication system.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Award Number: 2524698
Principal Investigator: Nicholas Wilkins
Funds Obligated: $1,250,000
State: NY