Indoor Wayfinding:

Developing a Functional Interface for Individuals with Cognitive Impairments

Alan Liu, Harlan Hile,

Henry Kautz, Gaetano Borriello

Pat Brown, Mark Harniss,

Kurt Johnson

University of Washington

Computer Science & Engineering

University of Washington

Rehabilitation Medicine

---

Problem

• Difficulties in wayfinding diminish the quality of life of many individuals with cognitive disabilities

  • Unemployment

  • Less access to community services

  • Social isolation

---

Wayfinding systems

• Just-in-time transit directions on a PDA for bus users (Carmien, et al.)

  • Infrastructure

• Opportunity Knocks – infer when a user may need help (Patterson, et al.)

  • Learning user model

• Nursebot – physically guide person within assisted living home (Pollack, et al.)

  • “Follow robot” guidance strategy

---

Wayfinding system goals

---

Indoor environments

• We want to study outdoor, indoor, and mixed environments

  • Indoor is safer for initial studies

  • Indoor environments tend to be more uniform

---

User Interface questions

• What kinds of direction-giving strategies are effective inside a building?

• What situations can arise during use?

  • E.g., riding the elevator with others

• Which modalities do users prefer?

  • Is there a wide range of preference?

---

Prototype design

• Person carries mobile device that links with building infrastructure (WiFi)

• User receives messages consisting of images, audio, and text

• Directions and prompts

  • Directions guide along path

  • Prompts to help return to a path

• Design challenge: make messages “clear”

---

---

Photos of the environment

Icons for short prompts

Augmented/enhanced photos

---

Wizard of Oz infrastructure

• Experiment before building system

• Realistic experience for potential users

---

Location wizard interface

---

Navigation wizard interface

---

Study

• 7 participants with cognitive impairment

• Participants used interface to guide them through 3 routes

• 3 sets of modalities

  • Image and text

  • Audio and text

  • Image, audio, and text

• Done in our Computer Science & Engineering building

---

Participant demographics

---

Routes

Turns: intersections along the route where participants had to change their heading.

Intersections: all intersections along path.

---

Task completion

• 6 of 7 participants reached all their route destinations

  • Participant 4 unable to reliably distinguish left/right, struggled with text and audio modality

  • Participant 5 just as comfortable with a map

  • Participant 1 requires the use of powered wheelchair

---

Stated modality preferences

---

Findings

• Text

  • Read at own pace

  • Unfamiliar vocabulary, size (readability)

• Audio (less popular than text)

  • Liked in conjunction with other modalities

  • No easy playback, noisy surroundings

• Images

  • Disambiguation, helpful to distinguish left/right

  • Slow to process (suggests focus on landmarks)

• Arrows and highlighted regions

  • Simple helped show where to go / direct attention

  • Multi-turn, non-standard orientations confusing

---

Findings (cont’d)

• Message timing is critical

  • Exiting elevator, quick turns

• Device usage

  • Spectrum of usage (“cruise control”, stop and wait, refer occasionally)

• Device form factor

  • Conspicuousness, size, weight

  • Could be addressed by mobile phone

---

Conclusion

• User Customization

  • Using preferred modalities

  • Accommodating users with multiple impairments (e.g., mobility, hearing)

• System adaptation

  • Choosing appropriate message complexity

  • Determining the right timing for messages

• No “one size fits all” solution

---

Future work

• Expand system capabilities

  • Support outdoors, indoor/outdoor transition, linking to public transit aide

  • Non-assisted baseline comparison, more varied tasks (users’ own routes)

• Explore alternative user interfaces

  • Study efficacy of maps for more spatially aware

  • Mobile phone implementation

• Implementing wizarded functionality

  • Localization

  • Decision theoretic control prompting

  • Photo augmentation

---

Acknowledgments

Thanks to Brian Ferris, Susumu Harada, Anna Cavender, anonymous reviewers, DUB group.

This work is funded by the National Institute on Disability and Rehabilitation Research (NIDRR) Grant #H133A031739.

---

Arrows

• Most participants reported that overlaid arrows were useful

  • Atypical arrows caused some confusion

---

Timing of Directions

• Exiting the elevator on the correct floor

  • Passengers exiting on different floors

  • Additional sensors needed when no WiFi

• Close turns

  • Combine upcoming directions

  • Provide notice of how long current direction applies

• Lag between turning user and message

---

Device usage

• Always attentive

  • Only altering course when told

• Stop and wait

  • Waiting for a direction at every intersection

• Rarely attentive

  • Kept at side, occasionally checking screen

---

Device form factor

• Size, weight, conspicuousness

• Mobile phone

  • Vibration feedback

  • Wireless headset

  • Large enough for text?

---

Future Work (cont’d)

• Better correction strategies

  • Determining when user is actually confused

  • New directions rather than retracing steps

• Implementation on mobile phone

  • Camera for computer vision

  • Additional sensors over Bluetooth