Intelligent Text Input and Editing Methods on Smartphones for Blind Users

Inputting and editing text on smartphones play an important role in people’s daily lives. Studies have
shown that they constitute 40% of one’s smartphone usage. For blind smartphone users, text input and text
editing tasks remain a challenge with studies showing that their typing speed is only 4-5 Words Per Minute
(WPM) on soft keyboards compared to 36 WPM of sighted users. While the use of voicing text input is
becoming prevalent among blind users, studies have shown that they spend 80% of their time on the average
editing spoken text, which is a slow, laborious process. This lack of efficient and convenient text input and
editing methods can make it difficult for blind users to use their smartphones proficiently and productively
for their everyday activities such as staying informed, connecting with friends and colleagues, shopping,
entertainment, travel, and for just about everything.
This project will research and develop a new generation of intelligent text input and editing methods
that will substantially enhance the proficiency with which blind smartphone users can input and edit text.
To this end, the project has three overarching objectives: First, we will create a non-visual free-form gesture
input method that will enable blind users to enter a word or phrase with free-form touch gestures on the
default Qwerty soft keyboard on smartphones, rather than entering text one letter at a time, an arguably
slow process. It will bring the popular gesture typing method to blind users, and increase their text input
efficiency as it will eliminate the need to confirm each letter during text input. Second, we will create an
intelligent Braille soft keyboard that can automatically correct input errors and do auto-completion of
unseen letters, which until now has been unavailable for Braille. Auto-correction has been shown to reduce
the input errors by over 30% for typing on a Qwerty soft keyboard for sighted users, and introducing it to
Braille soft keyboards would result in an even greater error reduction as touch inputs of blind users are far
more error-prone. Third, we will leverage language models in combination with voicing and touch to
eliminate the tedious operations required by extant methods that rely exclusively on precise cursor
movements for editing erroneous text input, which can occur quite often with spoken text input. Based on
previous studies with sighted users that show a 30-40% gain in editing performance with this approach, we
expect even greater performance gains with blind users as cursor-based operations are far more challenging
for them. Successful accomplishments of these objectives will eliminate any barriers to productivity of
blind smartphone users and empower them to utilize the power and connectivity of these devices to fully
participate in this digitized economy as critical services are increasingly delivered via smartphone devices.

Grant Number: 5R01EY035688-02
NIH Institute/Center: NIH
Principal Investigator: Xiaojun Bi