AI-Based Speech Enhancement for Hearing Aids

The greatest challenge to auditory communication is background noise, especially in the complex acoustic
environments that are experienced in daily life. This project proposes to develop a deep learning speech
enhancement (SE) algorithm that is robust to everyday environmental interference, improves speech intelligibility
for people with hearing loss and can is small enough to be embedded in chips used in hearing aids. Taking on
this challenge is inspired by the high-level performance of a small deep learning SE technology developed at
OmniSpeech that was embedded in headsets. This headset went to market on October 16, 2023 and has
received positive reviews. Unlike traditional signal processing algorithms that are only effective at improving
sound quality and comfort in noisy complex environments, large deep learning SE models can improve speech
intelligibility and preserve environmental sound recognition. This difference is due to the ability of deep learning
SE to (a) significantly reduce babble noise without leaving annoying artifacts (also called musical noise), (b)
remove unwanted transients (e.g., jackhammer) and (c) reduce environmental sounds overlapping with speech
even when the background sounds are dynamic and loud (e.g., babble, sirens, jackhammer). We aim to obtain
comparable results with a small deep learning SE model. Such a success will be a major advantage over other
SE models because of our model’s small size and thus, realistic potential to be incorporated in hearing aids and
other personal assistive listening devices for listeners with hearing impairment. We will use the Zilany auditory-
nerve (AN) models developed for normal-hearing and hearing-impaired ears, as well as correlation metrics, to
select a small deep neural network (DNN) model that results in noise suppression and best restores important
spectral and temporal characteristics of the signals to yield responses in impaired ears that are similar to those
of normal ears. The AN models will be used to inform the adaptation and refinement of our existing deep learning
SE to benefit listeners with hearing loss. Two versions of our algorithm will be developed targeting two levels of
aggressiveness in reducing unwanted noise. Low aggressiveness will maintain the quality and recognizability of
environmental sounds by limiting the amount of noise reduction. High aggressiveness will maximize speech
intelligibility by limiting the availability of some environmental sounds. We will evaluate the benefit of the two
versions of our SE technology in listeners with mild to moderate sensorineural hearing loss as indexed by
improvement in speech intelligibility and listening effort. The enhanced signal will be amplified using industry
standard hearing aid gain individualized for each listener. For this experiment, OmniSpeech will team up with
the Hearing Technology Lab directed by Eric Hoover at the University of Maryland. The overall goal of this Phase
I project is a proof-of-concept test of a viable deep learning SE technology. Phase II will build on the clinical trial
of Phase I and leverage the experience OmniSpeech is gaining with bringing speech technology to market in
embedded devices.

Grant Number: 1R41DC023169-01
NIH Institute/Center: NIH
Principal Investigator: Craig Birkhimer