The motivation of using different methodes to do the sound design for this surf movie comes from a lack of surf movies and documentaries that use sound design based on field recordings in this area. With this project I want to showcase how many layers of realness can be added to a surf documentary by using on set field recordings paired with sensor data to convey this experience of surfing on a much deeper level.
Therefore, the Sound design of this project is not seen as a post-processing effect but as an integral part of how motion and environmental interaction are perceived. The core idea and mission is to treat the surfer’s movement as a gestural performance and dance that modulates audio based on what is actually happening in the image. With the help of Pure Data, a modular synthesis environment, the motion data is mapped on audio processing parameters to underline this immersive and expressive sonic storyline.
Starting with the different sound design inputs that will be used in the Surf film, the primary audio material comes from a contact microphone imbedded in the surfboard. These are real, physical vibrations, bumps, hits, and subtle board resonances create the basic sonic texture of the piece. These raw recordings are used as:
- Input for audio modulation
- Triggers or modulating sources for effects like pitch shifting, filtering, and delay
Second core sound source is the Zoom H4n recorder mounted on the nose of the board. Here the focus lies strongly on field recordings in order to capture the raw sonic experience of the surf session.
Furthermore, the data of the sensor will be adjusting the soundscape, translating raw data into modulation for sound design.
Also, the internal audio of the GoPro Hero3 will be used to synchronize data in post processing and the recorded video will be a visual representation of the full experience.
Looking at the mapping part of the project, the x-IMU3 sensor provides multiple streams of data like acceleration, gyroscopic rotation, and orientation, that are mapped to sound parameters. Each data of movement is used differently:
Acceleration (X, Y, Z) modulates filter cutoff, grain density, or reverb size. Here the exact usage of modulation parameters will be discussed in the postproduction phase of the project.
Angular velocity controls pitch shift, stereo panning, or feedback loops.
Orientation (Euler angles or quaternion) is used to gate effects or trigger events based on the recorded movement thresholds.
The mappings will be adjusted in the following process and are designed to reflect the physical sensation of surfing in the most accurate way possible. Looking at the movement that is most important, the Cutback move, here a sharp move will translate in a spike in angular velocity. This spike can be translated in a big glitch sound effect. Here more research and test will be needed in order to find the best parameter settings for this modulation.
One possibility of audio modulation in Pure Data will be the granular synthesis. It allows to create evolving textures from short segments, like grain noise of the recorded contact mic sounds.
Further examples of possible modulations:
- Grain size – (short = more textural, long = more tonal)
- Playback speed and pitch
- Density and overlap of grains
SOUND NARRATIVE STRUCTURE
Looking at the storyline of the surf documentary one can pinpoint the following narrative structure of the sound design:
Before the surf / coastline of Morocco:
To catch the stressful coastal live of Morocco field recordings will be used to translate this feeling of stress, being overwhelmed (red mind). Here the recordings will be done by the Zoom H4n recorder.
Entering the water/ Paddling Phase:
As the surfer enters the water the stressful coastal sounds fade, and the listener will be surrounded by the sound of the ocean. Here it is important to translate the soundscape, which the surfer actually perceives. No further sound modulation is added here. The theory of the blue mind points out how much the noise of the ocean can regulate the nervous system. This will be translated to the sound design of this section of the movie, giving the listener the feeling of being in the present.
Catching the wave:
As soon as the surfer catches the wave and manages to stand up on the wave the dramaturgical main part of the composition begins. This will be initialized by a strong impact on the contact microphone, triggered by the jump of the person. This will also be measurable on the motion sensor with increase of speed. At this point of the composition the sound modulation starts.
Riding the wave / Cutbacks: At this stage of the movie the person feels a sensation of absolute presence and high focus. This natural high state gives a feeling that is hardly describable in words or images. Here the Sound Desing carries the listener through. Granular synthesis, stereo modulation and filtered resonance reflecting the physical and spiritual intensity in this moment. Here the tool of sound modulation is chosen intentionally to also create a contrast between the paddling stage of the movie.
End of the riding / Hit of the wave:
In the end of the movie the surfer will fall in the water creating a strong and impactful ending of the whole experience. This sudden cut will be auditory through a big amount of noise of the underwater recording. Nothing more than muffled wave sounds will be heard to empathize the feeling of being underwater. Sonic textures will decay leaving with a feeling of stillness after this intense movement.
With the help of this sonic structure both the physical and emotional journey of a surf session is captured and represented.
Considering the final sound piece a stereo format is the first output. Also including spatial depth will be achieved through modulation and stereo imaging based on the recorded motion data. Volume normalization and dynamic range control are applied in Davinci Resolve, however by respecting the intention of the sound piece to add less additional audio modulation by a software and only using techniques of audio manipulation using the sensory data.
The final audio and movie is intended for headphone or multichannel playback in an installations or possible surf exhibitions.
