Presentation
Potential for Robots to Assist People with Vision Impairments with Orientation and Mobility
SessionRobotics Summit - Session 4
DescriptionThe global population of people with vision impairments (PwVI) includes an estimated 596.2 million people with distance vision loss. Vision impairments often necessitate a combination of medical and navigational support, placing financial and logistical strain on healthcare systems, care partners, and family members. Limited access to quality assistive options further reduces the independence and quality of life for PwVI. Robotic systems have shown promise in improving orientation and mobility (O&M) for PwVI, but existing solutions have limitations in their adaptability and usability within unfamiliar spaces. This study explores the potential of assistive robotics to support PwVI in safe and efficient navigation through real-world environments, drawing insights from subject matter experts (SMEs, N = 9) in O&M training for PwVI and assistive technology.
To address these concerns and augment the quality of life of PwVI, we are developing an assistive guide robot. This WayBot disambiguates voice commands given by the PwVI user for tasks like leading the user to a specific object location; further, it can describe the surroundings to the user via speech. However, it is unable to manipulate it’s environment, such as for fetching objects requested by the user, or guiding their hand to pick up a small item. Nor does the robot explicitly plan or perform control based on a human model. Other robotics work provide can provide assistance to PwVI in different ways. Quadruped robots have seen a recent surge in application to guiding PwVIs as robotic seeing-eye dog configuration, using the harness as a physical mechanism for communication with the user. However, they may not always provide much assistance in unfamiliar environments where aid is needed most. Other robotic systems provide a wearable sensor(s), such as in the form of a necklace, identify and enumerate over speech the presence of objects to the user speech. While useful for giving high-level scene descriptions, these systems cannot physically navigate users. Further, there are several efforts in retrofitting white canes, the most common tool of PwVIs, with an array of sensors such as cameras and ultrasonics in order to convey a greater understanding of the environment around. Even other work seeks to help PwVIs guide their hand to an object of interest with a shoulder mounted camera, for everyday activities such as grocery shopping.
This aforementioned WayBot is based on the Stretch robot platform. Stretch is a lightweight and cost-effective mobile manipulator developed by Hello Robot. Stretch has a holonomic wheeled base, a telescoping arm with a vertical lift, and is equipped with a LiDAR sensor and an RGB depth cameras, allowing it to detect and respond to environmental obstacles. For interaction, it has a microphone and speaker array, facilitating voice communication. To enhance O&M support, we designed a haptic handle with dual Titan vibration motors, each independently actuated via Bluetooth to provide directional and proximity feedback. The system uses three types of haptic signals: indicating direction changes by activating the left or right motor, signaling proximity to obstacles through vibration intensity, and alerting users of narrow passages by engaging both motors simultaneously. The system was conceived as a building service robot in a hospital which would learn the space beforehand and be available for use by patients with visual impairments during their visit.
To investigate the feasibility and acceptance of Stretch as a robotic solution, we conducted semi-structured interviews with PwVI and SMEs. For this presentation, we are focusing on the study with the SMEs. The SMEs interviewed were O&M trainers, guide dog instructors, or assistive technology specialists for PwVI. The SMEs also had experience with various age groups, ranging from children to older adults. Interviews were conducted via Zoom to increase geographic reach and diversity in expertise.
The SMEs provided constructive feedback on the functional capabilities of Stretch, emphasizing key facilitators and barriers to its acceptance among PwVI. Customization emerged as a pivotal factor; SMEs suggested providing adjustable options for speed, voice tone, and communication levels to cater to individual user needs. These features, they argued, would facilitate broader adoption by allowing PwVI to tailor the device to their comfort levels and navigational preferences. Conversely, they advised against overly complex features that might overwhelm users in unfamiliar environments, advocating instead for simplified, intuitive feedback mechanisms. SMEs also advised that the robot’s navigation needs to consider the movements of the user especially in unfamiliar or unpredictable settings. The perception of safety and control was deemed essential for user trust, with SMEs suggesting that even with robotic support, PwVI should retain proficiency in traditional tools, such as white canes, for an additional layer of security.
Feedback from SMEs highlighted the value of interaction beyond simple audio cues, suggesting that PwVI often rely on non-verbal signals when guided by human aides. This informed the design recommendation for Stretch to detect and respond to user actions, such as pulling back or pushing forward on the handle, to adjust its movement speed accordingly. By replicating some of the nuanced feedback that a human guide might offer through physical adjustments and tone, the robot could deliver a more natural and comfortable guidance experience.
The study also focused on the importance of personalization and autonomy in fostering motivation for sustained robot use. Customizable features could help users adapt the device to meet specific goals, such as direct navigation to a target or leisurely exploration. These options could include varying the degree of feedback detail, speed adjustments, and vibration patterns. However, the wide array of choices poses the risk of overwhelming users. SMEs advised that personalization options should be simplified to maintain ease of use, especially for new users unfamiliar with robotic technology.
The study reveals potential for assistive robots to support PwVI, particularly through customizable, privacy-oriented, and simple-to-use feedback systems. The SME feedback provides essential design insights for optimizing acceptance and usability of the Stretch platform, emphasizing the need for adaptable and user-friendly features. This research contributes to the literature on robotic O&M aids for PwVI, furthering user-centered design and testing to maximize the impact and accessibility of assistive robots for vision-impaired individuals.
To address these concerns and augment the quality of life of PwVI, we are developing an assistive guide robot. This WayBot disambiguates voice commands given by the PwVI user for tasks like leading the user to a specific object location; further, it can describe the surroundings to the user via speech. However, it is unable to manipulate it’s environment, such as for fetching objects requested by the user, or guiding their hand to pick up a small item. Nor does the robot explicitly plan or perform control based on a human model. Other robotics work provide can provide assistance to PwVI in different ways. Quadruped robots have seen a recent surge in application to guiding PwVIs as robotic seeing-eye dog configuration, using the harness as a physical mechanism for communication with the user. However, they may not always provide much assistance in unfamiliar environments where aid is needed most. Other robotic systems provide a wearable sensor(s), such as in the form of a necklace, identify and enumerate over speech the presence of objects to the user speech. While useful for giving high-level scene descriptions, these systems cannot physically navigate users. Further, there are several efforts in retrofitting white canes, the most common tool of PwVIs, with an array of sensors such as cameras and ultrasonics in order to convey a greater understanding of the environment around. Even other work seeks to help PwVIs guide their hand to an object of interest with a shoulder mounted camera, for everyday activities such as grocery shopping.
This aforementioned WayBot is based on the Stretch robot platform. Stretch is a lightweight and cost-effective mobile manipulator developed by Hello Robot. Stretch has a holonomic wheeled base, a telescoping arm with a vertical lift, and is equipped with a LiDAR sensor and an RGB depth cameras, allowing it to detect and respond to environmental obstacles. For interaction, it has a microphone and speaker array, facilitating voice communication. To enhance O&M support, we designed a haptic handle with dual Titan vibration motors, each independently actuated via Bluetooth to provide directional and proximity feedback. The system uses three types of haptic signals: indicating direction changes by activating the left or right motor, signaling proximity to obstacles through vibration intensity, and alerting users of narrow passages by engaging both motors simultaneously. The system was conceived as a building service robot in a hospital which would learn the space beforehand and be available for use by patients with visual impairments during their visit.
To investigate the feasibility and acceptance of Stretch as a robotic solution, we conducted semi-structured interviews with PwVI and SMEs. For this presentation, we are focusing on the study with the SMEs. The SMEs interviewed were O&M trainers, guide dog instructors, or assistive technology specialists for PwVI. The SMEs also had experience with various age groups, ranging from children to older adults. Interviews were conducted via Zoom to increase geographic reach and diversity in expertise.
The SMEs provided constructive feedback on the functional capabilities of Stretch, emphasizing key facilitators and barriers to its acceptance among PwVI. Customization emerged as a pivotal factor; SMEs suggested providing adjustable options for speed, voice tone, and communication levels to cater to individual user needs. These features, they argued, would facilitate broader adoption by allowing PwVI to tailor the device to their comfort levels and navigational preferences. Conversely, they advised against overly complex features that might overwhelm users in unfamiliar environments, advocating instead for simplified, intuitive feedback mechanisms. SMEs also advised that the robot’s navigation needs to consider the movements of the user especially in unfamiliar or unpredictable settings. The perception of safety and control was deemed essential for user trust, with SMEs suggesting that even with robotic support, PwVI should retain proficiency in traditional tools, such as white canes, for an additional layer of security.
Feedback from SMEs highlighted the value of interaction beyond simple audio cues, suggesting that PwVI often rely on non-verbal signals when guided by human aides. This informed the design recommendation for Stretch to detect and respond to user actions, such as pulling back or pushing forward on the handle, to adjust its movement speed accordingly. By replicating some of the nuanced feedback that a human guide might offer through physical adjustments and tone, the robot could deliver a more natural and comfortable guidance experience.
The study also focused on the importance of personalization and autonomy in fostering motivation for sustained robot use. Customizable features could help users adapt the device to meet specific goals, such as direct navigation to a target or leisurely exploration. These options could include varying the degree of feedback detail, speed adjustments, and vibration patterns. However, the wide array of choices poses the risk of overwhelming users. SMEs advised that personalization options should be simplified to maintain ease of use, especially for new users unfamiliar with robotic technology.
The study reveals potential for assistive robots to support PwVI, particularly through customizable, privacy-oriented, and simple-to-use feedback systems. The SME feedback provides essential design insights for optimizing acceptance and usability of the Stretch platform, emphasizing the need for adaptable and user-friendly features. This research contributes to the literature on robotic O&M aids for PwVI, furthering user-centered design and testing to maximize the impact and accessibility of assistive robots for vision-impaired individuals.
Authors
Event Type
Robotics Workshop Submission
TimeSunday, March 303:20pm - 3:40pm EDT
LocationHarbour A/B