Smartphone app helps blind users navigate faster and avoid obstacles

For a blind or low-vision pedestrian, a few meters of GPS error can make the difference between staying on a sidewalk and missing a turn. A Harvard team has built a smartphone navigation system designed to close that gap.

The app, called Mobilio, combines the phone’s camera, GPS, motion sensors and, when available, LiDAR to guide users through streets and around obstacles. It also delivers personalized spatial audio cues that tell a person which way to turn.

Researchers at the Harvard John A. Paulson School of Engineering and Applied Sciences developed the system after surveying 112 people with blindness or visual impairment. Respondents identified three needs: turn-by-turn directions, continuous path guidance and obstacle avoidance.

Existing tools often cover only part of that list. White canes help detect nearby hazards but cannot provide directions to an unfamiliar destination. Guide dogs can help with obstacles and path guidance, but not full turn-by-turn routing. Electronic travel aids may offer more features, but many cost hundreds or thousands of dollars.

Mobilio is designed to run entirely on a smartphone. In the experiments, participants wore an iPhone 12 Pro on the chest, used open-ear headphones and carried a standard white cane.

The phone estimated movement through visual and inertial odometry, which combines camera data with motion-sensor readings. The system fused that information with GPS to improve positioning. A machine-learning model analyzed the camera feed and identified surfaces such as sidewalks, roads and crosswalks.

That model was trained on pedestrian-view images. Standard street-scene datasets often come from cameras mounted on cars, which can make them less useful for details that matter to someone walking.

“There’s a need for more data that comes from the perspective of humans walking along sidewalks, or walking outdoors,” said Raymond Liu, the study’s first author and a Harvard Ph.D. student.

Obstacle detection added another layer. Mobilio used LiDAR or camera-based depth estimation to map nearby objects in three dimensions. In technical tests, LiDAR estimated obstacle distance with an average error of 8.0%, while the camera-based method had a 12.2% error.

The visual and inertial odometry also performed well over a 300-meter outdoor path, with an average position error of 1.2 meters, or 0.4%.

Instead of relying only on spoken instructions, Mobilio uses directional beeps. A sound becomes more prominent in the left or right ear depending on where the user should turn, while pitch changes as the person lines up with the desired direction.

The team personalized those cues with a human-in-the-loop optimization method. During walking, the system adjusted audio rate, pitch range and directional scaling based on how accurately each participant responded.

Fourteen participants completed the personalization tests. Nine were blind and five were visually impaired. After optimization, their heading error fell by 19% compared with the generic starting settings.

The personalized settings varied widely from one participant to another, reinforcing the idea that a single feedback pattern may not work equally well for everyone.

“Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go,” said Patrick Slade, assistant professor of bioengineering.

The team tested Mobilio on a 150-meter outdoor route that included pedestrians, passing cars, nearby construction and maintenance noise. Participants navigated with a white cane under three conditions: guidance from a human, Mobilio or Google Maps.

Mobilio required 72% fewer verbal corrections and human guides required 82% fewer corrections than Google Maps.

Participants also completed the route 13% faster with Mobilio than with Google Maps. Human-guided walks were another 13% faster than Mobilio. In reliability, Mobilio performed similarly to a human guide in getting users to the destination without major errors.

A separate indoor obstacle course produced a different tradeoff. Participants using Mobilio and a cane made 41% fewer contacts with obstacles or the surrounding environment than when using a cane alone. However, they took 17% longer to complete the course.

The authors said the extra time may reflect the mental effort required to interpret the audio feedback while walking.

Participant surveys were favorable. Mobilio received an average System Usability Scale score of 79, placing it in the 84th percentile among aggregated usability studies cited by the researchers. All but one participant said they were interested in using Mobilio in the future.

“Having actual people test out your device is absolutely crucial because it’s impossible to predict how somebody is going to react to your device,” Liu said. “Especially within blindness and visual impairment, which is a huge, diverse set of impairments.”

The researchers caution that Mobilio still needs broader testing. Its computer-vision system can struggle when sidewalks resemble roads, crosswalk markings are badly faded or snow covers the ground.

The team also wants to test the app across more outdoor routes, geographic settings and weather conditions. Future versions would need a user interface that includes destination selection through a screen reader.

The smartphone placement may also change. Researchers are considering chest straps, waist harnesses, lanyards and possibly smart glasses paired with a phone.

Mobilio could combine several navigation functions now spread across different tools. Because it runs on a smartphone, it could avoid the cost and bulk associated with some specialized electronic aids.

The results also show why personalization may matter in assistive technology. Participants responded best to different audio settings, suggesting navigation cues may need to adapt to individual users rather than follow one fixed design.

The next step is wider community testing around Boston. The researchers plan to evaluate performance in more realistic daily settings and how well Mobilio fits into existing navigation habits. The Harvard Grid Accelerator is supporting efforts to translate the technology toward a broadly accessible product.

These resources examine how smartphones, computer vision, sensors and personalized guidance are being developed to improve independent travel for people who are blind or have low vision.

Wayfinding and navigation approaches supporting mobility for people who are blind or visually impaired: a scoping review
This 2026 review of 50 studies identifies persistent challenges in assistive navigation, including device design, technical performance, user interaction, personalization and integration into everyday travel. (Disability and Rehabilitation: Assistive Technology, 2026)

Technological Advancements in Human Navigation for the Visually Impaired: A Systematic Review
This systematic review examined 58 studies published from 2019 through 2024, covering smartphone systems, haptic feedback, LiDAR, GPS, artificial intelligence and obstacle-detection technologies. (Sensors, 2025)

Exploring the use of smartphone applications during navigation-based tasks for individuals who are blind or who have low vision: future directions and priorities
An international survey of 139 blind or low-vision participants explored which navigation apps people actually use and found that smartphone tools generally supplement, rather than replace, white canes and guide dogs. (Disability and Rehabilitation: Assistive Technology, 2025)

Evaluating the efficacy of UNav: A computer vision-based navigation aid for persons with blindness or low vision
Researchers tested a computer-vision navigation system with 20 blind or low-vision participants and found it performed at least as well as standard in-person travel directions, with better results on eight of nine measured outcomes. (Assistive Technology, 2025)

A comprehensive review of navigation systems for visually impaired individuals
This review traces assistive navigation from white canes to systems using smartphones, artificial intelligence, LiDAR, satellite navigation and 3D audio, highlighting the growing emphasis on portable and lower-cost technologies. (Heliyon, 2024)

Research findings are available online in the journal Nature Biomedical Engineering.

The original story "Smartphone app helps blind users navigate faster and avoid obstacles" is published in The Brighter Side of News.

Like these kind of feel good stories? Get The Brighter Side of News' newsletter.