Skip to main content

Multisensory experience

All experience is multisensory. Our senses are constantly combining and interacting to give us the best possible understanding of the world. This section could offer a range of creative provocations, both in finding novel techniques for simulating sensory experience and in combining the senses in new and interesting ways. What follows is not a comprehensive account of multisensory experience but an overview of some selected multisensory effects and interactions that might be particularly relevant for converged media production and experience.32

A word of warning. From an inclusion perspective, considering experiences in terms of multisensory works best when guided by openness rather than fixed assumptions about what people need. Haptic sensory substitution, for example, is often assumed to benefit all blind people; yet people's preferences, abilities and contexts differ greatly. Treating haptics as a tick-box inclusion measure risks overlooking this diversity and may even act to limit creative exploration for inclusion.

There are several ways in which multisensory experiences can be designed to promote inclusion:

  • Adaptation enhances accessibility by tailoring the sensory experience for diverse audiences. In Punchdrunk's Viola's Room, participants explore an environment while listening to an audio experience through headphones. Demonstrating adaptation, individuals with hearing loss in one ear can opt for mono audio to improve their experience.

  • Substitution allows for information that would be communicated in one sense to be communicated in another. If participants had a visual impairment, in addition to the audio narration, descriptions of the environment could be overlaid. In Punchdrunk's Viola's Room, multiple accessible listening options are available, including dialogue-enhanced and audio-described.

  • Integration, on the other hand, involves including additional sensory elements. Someone with audio processing needs might find subtitles helpful. Conversely, it is also important to consider allowing for some sensory elements to be switched off, such as flashing lights.