ERI: RUI: Controlling and Designing Soft Material Robots for Mixed Reality Applications

This Engineering Research Initiation (ERI) Research in Undergraduate Institutions (RUI) grant will support research to contribute new knowledge related to a class of robots, promoting both the progress of science and advancing national prosperity. Soft robots are robots with bodies made from soft materials like rubbers and foams. Soft robots are a natural choice for applications requiring safe, close interactions between humans and robots. One particular class of applications that promises to benefit from soft robots are mixed reality applications, which combine physical and virtual elements to create immersive experiences that users can both touch and see. However, there currently exists no principled framework for controlling and designing soft robots capable of producing mixed reality experiences. This award supports fundamental research to endow soft robots with the capability to generate mixed reality experiences. These mixed reality experiences could include highly accurate surgical training simulators as well as educational demonstrations. This research has broad impacts in healthcare, manufacturing, education, and agriculture and will benefit the United States economy and society. This research will provide opportunities for undergraduates at a primarily undergraduate institution to engage in novel robotics research at the intersection of engineering and computer science.

Optimization-based techniques enable real-time solutions to the soft robot inverse kinematics problem. However, existing methods have limited application to mixed reality because they only consider physical robots in isolation and cannot coordinate movements with virtual content or maintain physical-virtual coherence during user interaction. This research addresses this gap by formalizing the simultaneous control of physical soft robots and corresponding virtual avatars and thereby endowing soft robots with the ability to generate mixed reality experiences. The research team will develop a unified model-based optimization framework for controlling mixed reality soft robots, including a novel user-aware immersion gap constraint, will develop a computational design algorithm for mixed reality soft robots, and will validate their methods using physical soft robots and VR headsets.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Award Number: 2451514
Principal Investigator: James Bern
Funds Obligated: $174,934
State: MA