NRI: Adaptive Teleoperation Interfaces for In-Home Assistive Robots

Mobile manipulators that can navigate and physically interact with their environment have the potential to
assist people with motor limitations in carrying out activities of daily living independently. Despite this
great potential, robots that can be safely deployed in the homes of these users do not yet exist. Until
recently one of the key reasons for this gap was the lack of available hardware platforms, but the release
of Stretch – a new low-cost, light-weight, inherently safe, and highly capable mobile manipulator – has
greatly reduced the barrier to in-home deployments. A second key reason is the difficulty of robust
autonomy given the vast variations across home environment. A practical, near-term solution is to have
robots teleoperated by the user, which is also preferred by users in the target group who want to keep
control over the robot. However, most existing teleoperations interfaces are not accessible to users with
motor limitations who might have restricted input. This project aims to tackle this challenge by developing
new systems and algorithms that enable adaptive accessible teleoperation interfaces for mobile
manipulators. The ability to adapt to the unique requirements and preferences of these users while
enabling the safest and most efficient operation of the robot is critical to our proposed solution.
The project involves the development of an integrated system called AccessTeleopKit implanted on the
Stretch robot platform, and shared as open-source software. New algorithms for automatically
customizing interfaces in AccessTeleopKit based on user input ability assessment, and automating
repetitive parts of common tasks from user demonstrations as they teleoperate the robot will be
developed. Contributions also include conceptual frameworks to represent teleoperation interfaces as
mappings between user inputs and robot control outputs with Finite State Machines; and the heuristically
assessing the accessibility of a teleoperation interface for users with different abilities. Evaluation will
involve user studies and multi-phase long-term deployments of the robots in five homes facilitated by
occupational therapists and will contribute new empirical findings about the usability, utility, and
acceptance of assistance robots in the home. New guidelines for occupational therapists to use
teleoperated robots in their practice will also be created.

Grant Number: 5R01EB034580-03
NIH Institute/Center: NIH
Principal Investigator: Maya Cakmak