by Gregor Schmitz -

Opera meets code: Philippe Manoury’s Die letzten Tage der Menschheit

At our visit at the IRCAM-institute during our Paris-excursion I visited a panel talk, that described the workflow in creating a multi-media opera, that lies at the intersection of traditional opera and contemporary music technology and that struck me: Die letzten Tage der Menschheit (The Last Days of Mankind) by French composer Philippe Manoury. Based on the extensive anti-war drama by Austrian writer Karl Kraus, the work premiered at the Cologne Opera in June 2025 and reflects on themes of conflict, media, and societal collapse.

The Material

Karl Kraus wrote Die letzten Tage der Menschheit during and after World War I. The text consists of over 220 short scenes, depicting fragments of daily life, political rhetoric, and journalistic distortion that led to the chaos of the war. Due to its scale and structure, Kraus himself considered the piece impossible to stage in its entirety.

Manoury’s adaptation condenses the material into a three-hour opera. Rather than present a straightforward narrative, the production offers a layered and often disjointed sequence of impressions and reflections. Manoury and director Nicolas Stemann refer to the result as a “Thinkspiel”, a hybrid of the German Spiel (play) and the English “think”, suggesting a theatre of ideas rather than linear storytelling.

Blending Acoustic and (digital)Electronic Practice

Manoury, known for his work with live electronics, collaborated closely with IRCAM (Institut de Recherche et Coordination Acoustique/Musique) in developing this opera. He used tools such as Antescofo, a real-time score-following system that syncs live instrumental input with preprogrammed electronic components, and PureData, a visual programming environment designed for audio synthesis and spatial control.

The system enables audio to follow performers in real time, allowing electronics to respond to spoken text, instrumental timing, and stage movement. Manoury worked with Miller Puckette, the creator of PureData, to develop new modules tailored to the opera’s needs, including a granular speech-processing system that tracks vocal input spatially on stage.

This setup allowed for integration of a full orchestra, live electronics, spoken word, and multimedia, with a focus on flexibility and performer interaction during rehearsals and live performance.

Structure and Staging

The opera is divided into two distinct parts. The first presents loosely chronological scenes from the First World War, focusing on figures such as politicians, journalists, and ordinary citizens. The second part is meant to be a reflection and takes a more abstract and philosophical tone, exploring themes such as violence, historical memory, and self-destruction.

A newly introduced character, Angelus Novus acts as an observer throughout the piece. Performed by mezzo-soprano Anne Sofie von Otter, the character provides continuity and commentary across the fragmented scenes.

The staging involves video projections, live camera feeds, war imagery, and a modular stage design. The visual components are used not for spectacle but to support the opera’s shifting focus and tonal contrasts.

A Contemporary Approach to Historical Events

Die letzten Tage der Menschheit does not aim for easy accessibility. Its structure, sound design, and subject matter are complex and at times demanding. However, the production reflects current interests in combining artistic disciplines and using digital tools to reinterpret historical works.

Rather than retell World War I history, the opera focuses on atmosphere and fragmentation, using both musical and technological language to examine how war, media, and misinformation interact, which in my opinion is as relevant as ever in the face of current events.

Sources:

https://antescofo-doc.ircam.fr

https://www.oper.koeln/de/produktionen/die-letzten-tage-der-menschheit/1018

https://www.philippemanoury.org/8584-2/

https://de.wikipedia.org/wiki/Die_letzten_Tage_der_Menschheit

https://www.youtube.com/watch?v=yG9OFe2IE7A

by David Adlberger -

Explore II: Image Extender – Image sonification tool for immersive perception of sounds from images and new creation possiblities

The Image Extender project bridges accessibility and creativity, offering an innovative way to perceive visual data through sound. With its dual-purpose approach, the tool has the potential to redefine auditory experiences for diverse audiences, pushing the boundaries of technology and human perception.

The project is designed as a dual-purpose tool for immersive perception and creative sound design. By leveraging AI-based image recognition and sonification algorithms, the tool will transform visual data into auditory experiences. This innovative approach is intended for:

1. Visually Impaired Individuals
2. Artists and Designers

The tool will focus on translating colors, textures, shapes, and spatial arrangements into structured soundscapes, ensuring clarity and creativity for diverse users.

  • Core Functionality: Translating image data into sound using sonification frameworks and AI algorithms.
  • Target Audiences: Visually impaired users and creative professionals.
  • Platforms: Initially desktop applications with planned mobile deployment for on-the-go accessibility.
  • User Experience: A customizable interface to balance complexity, accessibility, and creativity.

Working Hypotheses and Requirements

  • Hypotheses:
    1. Cross-modal sonification enhances understanding and creativity in visual-to-auditory transformations.
    2. Intuitive soundscapes improve accessibility for visually impaired users compared to traditional methods.
  • Requirements:
    • Develop an intuitive sonification framework adaptable to various images.
    • Integrate customizable settings to prevent sensory overload.
    • Ensure compatibility across platforms (desktop and mobile).

    Subtasks

    1. Project Planning & Structure

    • Define Scope and Goals: Clarify key deliverables and objectives for both visually impaired users and artists/designers.
    • Research Methods: Identify research approaches (e.g., user interviews, surveys, literature review).
    • Project Timeline and Milestones: Establish a phased timeline including prototyping, testing, and final implementation.
    • Identify Dependencies: List libraries, frameworks, and tools needed (Python, Pure Data, Max/MSP, OSC, etc.).

    2. Research & Data Collection

    • Sonification Techniques: Research existing sonification methods and metaphors for cross-modal (sight-to-sound) mapping and research different other approaches that can also blend in the overall sonification strategy.
    • Image Recognition Algorithms: Investigate AI image recognition models (e.g., OpenCV, TensorFlow, PyTorch).
    • Psychoacoustics & Perceptual Mapping: Review how different sound frequencies, intensities, and spatialization affect perception.
    • Existing Tools & References: Study tools like Melobytes, VOSIS, and BeMyEyes to understand features, limitations, and user feedback.
    object detection from python yolo library

    3. Concept Development & Prototyping

    • Develop Sonification Mapping Framework: Define rules for mapping visual elements (color, shape, texture) to sound parameters (pitch, timbre, rhythm).
    • Simple Prototype: Create a basic prototype that integrates:
      • AI content recognition (Python + image processing libraries).
      • Sound generation (Pure Data or Max/MSP).
      • Communication via OSC (e.g., using Wekinator).
    • Create or collect Sample Soundscapes: Generate initial soundscapes for different types of images (e.g., landscapes, portraits, abstract visuals).
    example of puredata with rem library (image to sound in pure data by Artiom
    Constantinov)

    4. User Experience Design

    • UI/UX Design for Desktop:
      • Design intuitive interface for uploading images and adjusting sonification parameters.
      • Mock up controls for adjusting sound complexity, intensity, and spatialization.
    • Accessibility Features:
      • Ensure screen reader compatibility.
      • Develop customizable presets for different levels of user experience (basic vs. advanced).
    • Mobile Optimization Plan:
      • Plan for responsive design and functionality for smartphones.

    5. Testing & Feedback Collection

    • Create Testing Scenarios:
      • Develop a set of diverse images (varying in content, color, and complexity).
    • Usability Testing with Visually Impaired Users:
      • Gather feedback on the clarity, intuitiveness, and sensory experience of the sonifications.
      • Identify areas of overstimulation or confusion.
    • Feedback from Artists/Designers:
      • Assess the creative flexibility and utility of the tool for sound design.
    • Iterate Based on Feedback:
      • Refine sonification mappings and interface based on user input.

    6. Implementation of Standalone Application

    • Develop Core Application:
      • Integrate image recognition with sonification engine.
      • Implement adjustable parameters for sound generation.
    • Error Handling & Performance Optimization:
      • Ensure efficient processing for high-resolution images.
      • Handle edge cases for unexpected or low-quality inputs.
    • Cross-Platform Compatibility:
      • Ensure compatibility with Windows, macOS, and plan for future mobile deployment.

    7. Finalization & Deployment

    • Finalize Feature Set:
      • Balance between accessibility and creative flexibility.
      • Ensure the sonification language is both consistent and adaptable.
    • Documentation & Tutorials:
      • Create user guides for visually impaired users and artists.
      • Provide tutorials for customizing sonification settings.
    • Deployment:
      • Package as a standalone desktop application.
      • Plan for mobile release (potentially a future phase).

    Technological Basis Subtasks:

    1. Programming: Develop core image recognition and processing modules in Python.
    2. Sonification Engine: Create audio synthesis patches in Pure Data/Max/MSP.
    3. Integration: Implement OSC communication between Python and the sound engine.
    4. UI Development: Design and code the user interface for accessibility and usability.
    5. Testing Automation: Create scripts for automating image-sonification tests.

    Possible academic foundations for further research and work:

    Chatterjee, Oindrila, and Shantanu Chakrabartty. “Using Growth Transform Dynamical Systems for Spatio-Temporal Data Sonification.” arXiv preprint, 2021.

    Chion, Michel. Audio-Vision. New York: Columbia University Press, 1994.

    Görne, Tobias. Sound Design. Munich: Hanser, 2017.

    Hermann, Thomas, Andy Hunt, and John G. Neuhoff, eds. The Sonification Handbook. Berlin: Logos Publishing House, 2011.

    Schick, Adolf. Schallwirkung aus psychologischer Sicht. Stuttgart: Klett-Cotta, 1979.

    Sigal, Erich. “Akustik: Schall und seine Eigenschaften.” Accessed January 21, 2025. mu-sig.de.

    Spence, Charles. “Crossmodal Correspondences: A Tutorial Review.” Attention, Perception, Psychophysics, 2011.

    Ziemer, Tim. Psychoacoustic Music Sound Field Synthesis. Cham: Springer International Publishing, 2020.

    Ziemer, Tim, Nuttawut Nuchprayoon, and Holger Schultheis. “Psychoacoustic Sonification as User Interface for Human-Machine Interaction.” International Journal of Informatics Society, 2020.

    Ziemer, Tim, and Holger Schultheis. “Three Orthogonal Dimensions for Psychoacoustic Sonification.” Acta Acustica United with Acustica, 2020.