The NIME 2024 paper Interactive Sonification of 3D Swarmalators by Pedro Lucas et al.—a project that merges swarm intelligence with sound and music systems in an unusual and intriguing way. Their work explores what happens when coupled oscillators (called “Swarmalators”) move in 3D space and interact through both spatial and phase dynamics, resulting in emergent sonic behavior.
What I Found Fascinating
First, the concept of “sound swarming” is compelling. Each swarmalator acts as a tiny sound generator (an oscillator), and together they form a swarm that evolves over time. As the swarm grows or changes state, the collective sonic output transforms, producing emergent, ambient textures. It’s like a synthetic ecology where sonic patterns ripple through space and time.
I really appreciated the balance between individual control (through the interactive swarmalator) and system-level complexity. The way one agent—controlled by a user—can gently nudge the entire swarm toward a new sound state (syncing phases or shifting spatial positions) reminds me of soft systems thinking, or how small disturbances in dynamic environments can guide large-scale changes. It’s a musical metaphor for influence and emergence.
Also interesting: the decision to use 3D space—not just as visual flair, but as a functional parameter in the sound synthesis. The angle between an agent’s position and the swarm center is mapped to modulation (LFO phase), which adds spatial logic to the sonic texture. This connection between location, rhythm, and pitch expands the expressive range of the system without overwhelming the user with complexity.
Critique or Question…
While I found the system architecture well thought out (especially the modular design between Unity and Max), I do wonder how accessible the musical outcomes really are for performers or audiences who aren’t already embedded in experimental sound practices.
- What does “sound swarming” feel like to someone who isn’t reading the underlying equations?There’s an assumption that emergent sonic behavior is interesting in itself—which is often true—but I’d be curious about perceived musicality or narrative structure. How does the user know when something meaningful is happening?
- How intuitive is the control? The interactive swarmalator is a smart concept, but its influence seems subtle and potentially slow. In a live performance context, would that control feel satisfying? Or would it feel like poking a beehive and waiting to see what happens?
- Sonification or Composition? I’m torn between seeing this as a sonification project (data → sound) or a compositional tool. It seems to sit between both, but I’d love to see clearer articulation on whether the goal is to represent something through sound, or to compose emergent music through interaction.
What I Would Like to Explore Further
This system opens a door to interesting possibilities for multisensory representation, especially when combined with haptics or extended reality (which the authors mention as future work). Imagine if you could feel vibrations from nearby swarmalators, or use your hand in an AR space to guide sound clusters around you.
It also made me think about accessibility: how could this system be made tangible for someone who doesn’t rely on visual interfaces? Could you “hear” the swarm’s shape or “feel” its convergence? Maybe adding another sensory layer could help bridge that gap.
Finally, I’d love to see this concept applied to non-musical data—for example, using environmental or physiological data as inputs to control the swarm behavior. That could transform this into an ambient, perceptual feedback tool rather than just a sound art piece.
This paper definitely broadened how I think about interactive systems, sonic feedback, and emergence. While the sonic aesthetics may lean toward experimental music, the design principles offer insight into how complex systems can be explored through sound—not just explained, but felt.
Reference:
P. Lucas, S. Fasciani, A. Szorkovszky, and K. Glette, “Interactive Sonification of 3D Swarmalators,” in Proc. Int. Conf. New Interfaces for Musical Expression (NIME), Utrecht, The Netherlands, Sep. 2024. [Online]. Available: https://doi.org/10.5281/zenodo.10948289
https://artsengine.engin.umich.edu/previous_projects/unnatural-nature-eco-art/