Feeling the Music: Haptics and the Future of Embodied Listening

What makes a cultural experience memorable? A study led by our Jun. Prof. Merle Fairhurst at Tate Britain showed that when interactive, multisensory stimuli are presented in a congruent way, memory is significantly enhanced (Fairhurst et al., 2017). This suggests that the more our senses are engaged together, the deeper the impact of the experience.

Music is a particularly powerful case in this sense. It naturally prompts us to tap our feet, sway, or even dance and sing along. Because music is already embodied in this way, it offers a unique opportunity to maximise multisensory engagement. And thanks to advances in immersive technologies, we can now explore how to extend the listening experience into other sensory domains.

For example, research has shown that musical features such as rhythm and dynamics can be translated into visual cues, helping newcomers to understand musical structures and improving accessibility for people with hearing impairments (Nanayakkara et al., 2013). Imagine experiencing a concert in virtual reality, with rhythm represented as pulsing shapes floating around you and the audience.

This is where haptics enters the stage. Haptic technologies, which communicate information through touch, can make music tangible. Imagine wearing a haptic vest during the above mentioned virtual concert. Imagine pulsing vibrations on your chest, in synchrony with the music rhythm; or music loudness conveyed through changes in the intensity of those vibrations. While still a developing field, researchers are already experimenting with vibrotactile actuators placed on the body to help participants with hearing impairments engage with music (Paisa et al., 2023). Others have explored multisensory augmentations of music for children with disabilities, opening new ways of learning and interaction (Frid et al., 2022).

The promise of multisensory music augmentation goes beyond accessibility. By making music a more embodied experience, haptics has the potential to deepen engagement, improve learning, and create new forms of artistic expression. In short, we may be moving towards a future where music is not only heard, but also seen, felt, and lived.

We'd like to leave you with two questions:

How can haptic technology enhance our emotional and physical connection to music beyond traditional listening experiences?

In what ways might multisensory approaches like touch and movement improve accessibility and inclusion in musical performances?