The long-term vision of my project is simple: – Record space. – Upload it. – Use it anywhere.
With enough participants, the Soundmap will become a searchable database of real-world acoustics – a kind of Google Street View for sound. Artists, researchers, and developers could then place sounds into acoustically true locations without ever going there.
Whether you want to preserve the sonic fingerprint of a historic building or make your VR app sound real, this approach makes spatial audio design more perceptual, democratic, and creative.
Just as photography changed how we see the world, acoustic photography could change how we hear it.
Most audio is designed in a vacuum – recorded dry and later drowned in artificial reverb. But what if we could design sound with real spaces in mind?
With my tool, I apply captured RIRs to original sounds, placing them into their authentic spatial context. A whisper in a cathedral, a guitar in a concrete stairwell, a voice in a silent forest – all can be recreated digitally and realistically.
This goes beyond fidelity – it’s about emotion, presence, and narrative. Sound design becomes site-specific. Just like film locations matter, so do acoustic locations.
Each space has a story to tell. I want to help sound designers listen to it.
In my first prototype, I’m developing a mobile tool that turns a smartphone into an acoustic camera. Just clap your hands or play a short sweep tone – the reflections tell us everything about how sound travels in the space.
My app records these Room Impulse Responses in 360° using either basic stereo mics or, in the future, Ambisonics capsules. It then uploads the result, together with GPS coordinates, to a crowdsourced global Soundmap.
We’re not just mapping the world visually. We’re learning to map it acoustically.
This could democratize spatial audio for music, game audio, VR, and sound preservation.
Have you ever walked into a space and instantly felt its unique sound atmosphere? Just like photographers capture the look of a space, we can now capture how a room sounds. My current research explores a concept I call acoustic photography – turning the complex echoes, reverberations and reflections of a room into a sonic “picture”.
Using a Impulse Response (IR) – an acoustic fingerprint of any environment – we can recreate its unique auditory qualities. With this, dry recordings can be placed into real acoustic environments using a method called convolution. The result? Audio that sounds like it was recorded in that space.
Music is often seen as a one-directional experience—composers create, performers interpret, and audiences listen. However, what if listeners could play a more active role in shaping the music they hear?
Movement as a Musical Element
In conventional concerts, the audience remains stationary while sound moves towards them. In contrast, this project leverages the open public space of the Joanneum Quarter to allow the audience to move through different acoustic environments, making movement an essential part of the musical experience.
This already begins with the Joanneum Quarters being an open public space, people cross during there everyday-life. There is no literal gate-keeping, which allows people to enter end leave the place and therefore the musical piece, whenever they like or stumble upon it.
Further the architectural design of the Joanneum Quarter introduces natural delays and phasing shifts as sound waves bounce off its curved glass structures as well as from the surrounding walls. This means that a e.g. melody played in one location may sound different depending on where a listener is standing. And because of time delays the layering of melodies result differently at different locations. As audience members walk through the space, their perception of the music changes, creating a dynamic and personal auditory experience.
Creating immersion via interfaces
Beyond the organic interaction caused by movement, the project considers additional ways to involve audiences directly in the performance. One may be the installation of interfaces:
By incorporating technical solutions such as speaker controls or digital interfaces, visitors can influence the composition itself. Simple adjustments—such as modifying the volume of different speakers—allow participants to shape their own experience. A more complex approach could involve digital interfaces, such as iPads placed around the space, where participants can select different musical elements for each acoustic funnel, effectively curating their own version of the performance.
Reimagining Concert Spaces: The Acoustic Landscape of Joanneum Quarter
Music and spaces have always shared a deep and inextricable relationship. The way sound interacts with space transforms the listening experience. By playing with it consciously we are creating an immersive experience that goes beyond traditional concert settings. This is the core idea behind the “Sounds of the Joanneum Quarters Graz” project—an innovative approach to ambient music and concert formats that redefines how audiences engage with sound.
The Vision: Transforming Public Spaces into Concert Venues
The project explores how the spatial dynamics of the Joanneum Quarter in Graz can shape a unique musical experience. Unlike conventional concert halls with fixed seating and predictable acoustics, this public space presents a lively environment where sound can evolve organically. The goal is to break down barriers associated with classical concert settings by offering an open, interactive listening experience that invites both intentional audiences and casual passersby.
Architectural Influence on Sound
A key element of this project is the relationship between sound and architecture. The Joanneum Quarter is defined by its distinct conical funnels, made of curved glass with a silk-screen print that filters light. These architectural features create natural acoustic properties that influence how sound behaves within this space. By treating these funnels as integral instruments, the composition can interact with the environment rather than simply existing within it.
There are two primary compositional approaches considered:
Treating each funnel as an individual instrument, crafting site-specific musical material that resonates with the unique properties of each space.
Creating soundscapes that work across all funnels, allowing listeners to move through the space and experience varied auditory perspectives.
Embodied Resonance is an experimental audio performance that investigates the interplay between trauma, physiological responses, and immersive sound. By integrating biofeedback sensors with spatial sound, this project translates the body’s real-time emotional states into an evolving sonic landscape. Through this process, Embodied Resonance aims to create an intimate and immersive experience that bridges personal narrative with universal themes of emotional resilience and healing.
Reference Works
Inspiration for this project draws heavily from groundbreaking works in biofeedback art. For instance, Tobias Grewenig’s Emotion’s Defibrillator (2005) inspired me to explore how visual imagery can serve as emotional triggers, sparking physiological responses that drive sound. Grewenig’s project combines sensory input with dynamic visual feedback, using breathing, pulse, and skin sensors to create a powerful interactive experience. His exploration of binaural beats and synchronized visuals provided a foundation for my use of AR imagery and biofeedback systems.
Another profound influence is the project BODY ECHOES, which integrates EMG sensors, breathing monitors, and sound design to capture inner bodily movements and translate them into a spatialized audio experience. This project highlights how subtle physiological states, such as changes in muscle tension or breathing rhythms, can form the basis of a compelling sonic narrative. It has inspired my approach to using EMG and respiratory sensors as key components for translating physical states into sound.
How Does It Work?
The performance involves the use of biofeedback sensors to capture physiological data such as:
Electromyography (EMG) to measure muscle tension
Electrodermal Activity (EDA/GSR) to track stress levels via skin conductivity
Heart Rate (ECG/PPG) to monitor pulse fluctuations and emotional arousal
Respiratory Sensors to analyze breath patterns
This real-time data is processed using software like Max/MSP and Ableton Live, which maps physiological changes to dynamic sound elements. Emotional triggers, such as augmented reality (AR) images chosen by the audience, influence the performer’s physiological responses, which in turn shape the sonic environment.
Core Components of the Project
Emotional Triggers and Biofeedback: The audience plays an active role by selecting AR-displayed imagery, which elicits emotional and physiological responses from the performer.
Sound Mapping and Generation: Physiological changes dynamically alter elements of the soundscape.
Spatial Audio and Immersion: An Ambisonic sound system enhances the experience, surrounding the audience in a three-dimensional sonic space.
Interactive Performance Structure: The performer’s emotional and physical state directly influences the performance, creating a unique, real-time interaction between artist and audience.
Why is This Project Important?
Embodied Resonance is an innovative approach to understanding how trauma manifests in the body and how it can be externalized through sound. This project:
Explores the intersection of biofeedback technology, music, and performance art
Provides a new medium for emotional processing and healing through immersive sound
Pushes the boundaries of interactive performance, inviting the audience into a participatory experience
Challenges conventional notions of musical composition by integrating the human body as an instrument
Why Do I Want to Work on It?
As a sound producer, performer, and music editor, I have always been fascinated by the connections between sound, emotion, and the body. My personal journey with trauma and healing has shaped my artistic explorations, driving me to create a performance that not only expresses these experiences but also fosters a shared space for reflection and empathy. By combining my technical skills with deep personal storytelling, I aim to push the boundaries of sonic expression.
How Will I Realize This Project?
Methods & Techniques
Research: Studying trauma, somatic therapy, and the physiological markers of emotional states.
Technology: Utilizing biofeedback sensors and signal processing tools to create real-time sound mapping.
Performance Development: Experimenting with gesture analysis and embodied interaction.
Audience Engagement: Exploring ways to integrate audience input via AR-triggered imagery.
Necessary Skills & Resources
Sound Design & Synthesis: Proficiency in Ableton Live, Max/MSP, and Envelop for Live.
Sensor Technology: Understanding EMG, ECG, and GSR sensor integration.
Spatial Audio Engineering: Knowledge of Ambisonic techniques for immersive soundscapes.
Programming: Implementing interactive elements using coding languages and software.
Theoretical Research: Studying literature on biofeedback art, music therapy, and embodied cognition.
Challenges and Anticipated Difficulties
Spatial Audio Optimization: Achieving an immersive sound experience that maintains clarity and emotional depth.
Technical Complexity: Ensuring seamless integration of biofeedback data into real-time sound processing requires rigorous calibration and testing.
Emotional Vulnerability: The deeply personal nature of the performance may present emotional challenges, requiring careful preparation.
Audience Interaction: Designing a system that effectively incorporates audience input without disrupting the emotional flow.
The project’s core goal is to create an embodied, immersive experience where the performer’s movements and physiological signals interact with dynamic soundscapes, reflecting states of stress, panic, and resolution. This endeavor seeks to explore the intersection of the body, trauma, and sound as a medium of expression and understanding.
Tasks Fulfilled by the Project:
Expressive Performance: Convey the visceral experience of stress and trauma through movement and sound.
Interactive Soundscapes: Use real-time biofeedback to dynamically alter sound parameters, enhancing the audience’s sensory engagement.
Therapeutic Exploration: Demonstrate the potential of somatic expression and sound for trauma exploration and healing.
Main Goals:
Develop a cohesive interaction between biofeedback, sound design, and movement.
Design an immersive auditory space using ambisonics.
Create an emotionally impactful narrative through choreography and sound dynamics.
Steps for Project Implementation
Identifying Subtasks:
Movement and Choreography Exploration:
Research and refine body movements that mirror states of stress and release.
Develop movement scores aligned with sound triggers.
Biofeedback and Technology Integration:
Select and test wearable sensors for movement and physiological signals (e.g., heart rate monitors, EMG sensors).
Map sensor data to sound parameters using tools like Max/MSP or Pure Data.
Sound Design and Ambisonics:
Create a palette of sound textures representing emotional states.
Test and refine 3D spatial audio setups.
Rehearsal and Iteration:
Practice interaction between movement and sound.
Adjust mappings and refine performance flow.
Determining the Sequence:
Begin with movement research and initial choreography.
Set up and test biofeedback systems.
Integrate sound design with real-time data mappings.
Conduct iterative rehearsals and refine dynamics.
Description of Subtasks
Required Information and Conditions:
Knowledge of movement techniques representing trauma.
Understanding biofeedback sensors and data processing.
Familiarity with ambisonic sound design principles.
Methods:
Employ somatic techniques and physical theater practices for movement.
Use biofeedback-driven sound generation software for real-time interaction.
Apply iterative testing and rehearsal methods for refinement.
Existing Knowledge and Skills:
Dance and performance experience.
Basic knowledge of sensor technologies and sound design tools.
Understanding of trauma’s physical manifestations through literature.
Additional Resources:
Sensors and biofeedback devices.
Ambisonic Toolkit and spatial audio software.
Research materials on trauma and biofeedback in art.
Timeline Overview
Current Semester – “Explore” Phase:
Research movement responses to stress and trauma.
Test sensors and sound mapping tools.
Document all findings to create the exposé and prepare for the oral presentation.
Second Semester – “Experiment” Phase:
Prototype interactions between movement, biofeedback, and sound.
Evaluate the feasibility and emotional resonance of the prototypes.
Incorporate feedback and iterate designs.
Third Semester – “Product” Phase:
Combine prototypes into a cohesive performance.
Optimize the interplay between sound and movement.
Conclude with final documentation and a presentation of the complete performance.
Questions for Exploration
What additional biofeedback sensors and sound techniques can enhance the performance?
How can movement scores effectively translate the emotional states into physical expressions?
What feedback mechanisms will refine the audience’s immersive experience?
A live performance where the body’s movement and physiological responses interact with real-time, 3D soundscapes, creating an auditory and sensory experience that embodies the physical and emotional states associated with trauma, stress, or panic.
Core Elements
Live Movement and Performance:
Physical Expression: Expressive body movements are used to convey states of stress, panic, and tension. Movements could be choreographed or improvised, incorporating controlled gestures, sudden shifts, and spasmodic motions that mirror the body’s natural reactions to trauma.
Sensor Integration: The performer will be equipped with wearable sensors (e.g., accelerometers, heart rate monitors, muscle tension sensors) to capture real-time data that triggers sound changes.
Sound Design and Biofeedback:
Real-time Data to Sound Mapping: The data from the sensors can be mapped to sound parameters such as volume, pitch, and spatial positioning.
Spatial Audio (Ambisonics): the 3D sound environment where the sound moves with the performer, simulating the feeling of being surrounded by or caught in an experience of panic.
Sound Layers and Textures: Layer sounds that range from chaotic, dissonant clusters to more open, calming tones, symbolizing shifts between heightened panic and brief moments of relief.
Interactive Performance Dynamics:
Feedback Loops: The performer’s movements could influence sound parameters, and changes in sound could, in turn, affect how the performer responds (e.g., sudden loud or abrupt sounds causing physical shifts).
Immersive Auditory Space: Spatial audio setup will immerse the audience, making them feel as though they are within the performance’s sonic realm or inside the performer’s body.
Choreography and Movement Techniques:
Imitating Panic and Stress:
Breath Control: Rapid, shallow breathing or uneven breathing patterns to simulate panic.
Body Tension and Release: Show how different areas of the body can tense up and release in response to imagined threats.
Sudden, Erratic Movements: Imitate fight-or-flight reactions through jerky, uncoordinated gestures.
Movement Scores: Create a set of movement phrases that can be triggered by specific sound cues, with each phase representing a different level of intensity or emotional state.
Implementation Steps:
Initial Research and Movement Exploration:
Spend time exploring how the body naturally responds to stress through dance or physical theatre techniques.
Record and analyze your body’s response to various stimuli to understand how to replicate these in a performance context.
Tech Setup and Testing:
Choose sensors capable of tracking movement and vital signs, such as wearable accelerometers and heart rate monitors.
Connect the sensors to real-time audio processing software (e.g., Max/MSP, Pure Data) to create dynamic sound generation based on data input.
Experiment with one biofeedback sensor (e.g., heartbeat or EMG) and connect it to sound manipulation software.
Test simple ambisonic setups to understand spatial audio placement.
Sound Design:
Use ambisonics to experiment with how sounds can be positioned and moved in 3D space.
Create a palette of sound elements that represent different stress levels, such as soft background noise, mechanical sounds, distorted human voices, and deep bass thuds.
Rehearsals and Iteration:
Conduct rehearsals where you practice the movement and sound interaction, making adjustments to the data-to-sound mappings to achieve the desired response.
Test with different inputs to refine the sonic representation of the body’s signals.
Refine the performance flow by timing the intensity of movements and sound shifts to ensure coherence and emotional impact.
Resources
Body and Trauma
The Body Keeps the Score by Bessel van der Kolk
Waking the Tiger: Healing Trauma by Peter Levine
Sound Design and Technology
Sound Design: The Expressive Power of Music, Voice and Sound Effects in Cinema by David Sonnenschein
Immersive Sound: The Art and Science of Binaural and Multi-Channel Audio edited by Agnieszka Roginska and Paul Geluso
Tools and Tutorials
Ambisonic Toolkit (ATK)
Cycling ’74 Max/MSP Tutorials
Artistic and Conceptual References
Janet Cardiff – Known for immersive sound installations, especially her 40-Part Motet.
Meredith Monk – Combines movement and sound to explore human experience.
Christine Sun Kim – Explores sound and silence through the lens of the body and perception.
Academic Research in Sound and Perception
Music, Cognition, and Computerized Sound: An Introduction to Psychoacoustics by Perry Cook
