As digital environments become increasingly immersive, multisensory design is transforming the way we interact with data, technology, and the world around us. However, ensuring these experiences are accessible to all remains a challenge. Traditional accessibility efforts have largely focused on visual-centric approaches, often excluding those who rely more on auditory, tactile, or cross-modal interactions.
A promising solution lies in sensory substitution techniques, which translate one sensory input into another. These techniques, often used in assistive technologies, have the potential to move beyond niche applications and become mainstream tools that enhance accessibility for everyone.
Beyond Visual-First Interfaces: Rethinking Multisensory Accessibility
Most digital interfaces prioritise visual information—charts, text, and images dominate how we consume data. However, not everyone experiences the world through sight. A more inclusive design approach considers:
- Sonification for Blind and Visually Impaired Users: Mapping data trends to sound (pitch rising for higher values) enables auditory pattern recognition.
- Haptic Feedback for Deaf and Hard-of-Hearing Users: Vibrations and force feedback provide real-time alerts and spatial awareness.
- Multisensory Adaptation for Neurodivergent Users: Some individuals process information better when it’s presented in multiple overlapping modalities, such as visual cues paired with subtle audio reinforcement.
Rather than designing separate assistive solutions, multisensory experiences should be natively inclusive, allowing users to select the sensory mode that best suits them.
Sensory Substitution: A Bridge to Universal Access
Sensory substitution devices (SSDs) replace information from one sensory modality with another, making data accessible in novel ways. For example:
- Visual-to-Auditory Substitution: Devices like The vOICe convert camera images into real-time soundscapes, allowing users to “hear” shapes and motion.
- Visual-to-Tactile Interfaces: Systems like BrainPort translate images into electrical pulses felt on the tongue, enabling spatial navigation for the visually impaired.
- Cross-Modal Mapping in Mainstream Design: Everyday interfaces can integrate these concepts—imagine a navigation app that offers both vibration-based and sound-based guidance, allowing all users to choose their preferred sensory format.
Despite their proven effectiveness, SSDs have not yet seen widespread adoption. A key challenge is that they are often designed only as assistive devices, rather than as features that could benefit all users in various contexts.
Real-World Applications of Inclusive Multisensory Design
By embedding sensory substitution and multisensory feedback into mainstream products, we unlock new ways of engaging with technology:
- Tactile Data Exploration: Raised surfaces, interactive touchpads, or vibration-based data encoding allow users to physically experience data trends.
- Multisensory VR & AR Experiences: Augmented and virtual reality environments can become more accessible by incorporating soundscapes, haptic responses, and cross-modal cues that extend beyond sight.
- Flexible Accessibility in Public Spaces: Interactive kiosks and wayfinding systems should support dynamic mode-switching, allowing users to receive information through visual, auditory, or tactile outputs based on their needs.
Designing for Multisensory Accessibility
To create truly inclusive multisensory experiences, designers must:
- Prioritize Sensory Adaptability – Allow users to customize how they receive information (toggling between visual, auditory, and tactile cues).
- Focus on Cross-Modal Integration – Ensure sensory inputs reinforce each other rather than competing (subtle haptic cues guiding users toward an audio source).
- Adopt a Universal Design Perspective – Move away from “assistive add-ons” and instead create mainstream products that naturally support diverse sensory abilities.
By making multisensory design accessible to all, we enhance usability for disabled users while also creating richer, more engaging experiences for everyone. Instead of viewing accessibility as an afterthought, it should be the foundation of future technology.
References
T. Lloyd-Esenkaya, V. Lloyd-Esenkaya, E. O’Neill, et al., “Multisensory inclusive design with sensory substitution,” Cognitive Research, vol. 5, no. 37, 2020, doi: 10.1186/s41235-020-00240-7.
M. Leung, “A look toward the future: The power of creating accessible multisensory experiences,” Accessibility.com, Feb. 19, 2024. [Online]. Available: https://www.accessibility.com/blog/a-look-toward-the-future-the-power-of-creating-accessible-multisensory-experiences. [Accessed: Jan. 31, 2025].
