INTERACTIVE AUDIO · MAX/MSP · SPATIAL SOUND

Interactive Multichannel Installation

Gesture-controlled musical layering through Max/MSP and real-time visual tracking.

ROLE
Interaction Design · Max/MSP Programming · Audio Production

FOCUS
Gesture Tracking · Real-time Control · Musical Layering

STATUS
Functional Prototype · 2025–2026

PROJECT OVERVIEW

This interactive installation allows participants to control the presence of individual instrumental layers within a multitrack composition through physical gestures. Real-time visual input is processed in Max/MSP and mapped to gradual fade-in and fade-out behaviors, enabling the audience to reshape the density, balance, and progression of the music without directly touching a controller. The project explores how intuitive bodily movement can function as a musical interface while preserving the continuity and structure of the original composition.

INTERACTION CONCEPT

INPUT
Camera-based gesture and facial-movement tracking

MAPPING
Detected movement is translated into continuous control values inside Max/MSP

OUTPUT
Individual instrumental layers gradually fade in and out within the multitrack composition

Participant → Camera Input → Gesture Detection → Max/MSP Mapping → Instrument Layer Control → Multichannel Playback

VISUAL DOCUMENTATION

A participant uses physical gestures to control the musical arrangement in real time.

Real-time visual tracking converts detected movement into control data.

Max/MSP processes the tracking data and maps it to the gain of individual instrumental layers.

Prototype demonstration integrating visual tracking, audio control, and spatial playback.

MUSICAL CONTROL STRATEGY
Rather than switching tracks on and off abruptly, the system uses continuous fade behaviors to preserve musical coherence. Each gesture influences the gain of a specific instrumental layer, allowing the arrangement to expand or contract gradually. This approach keeps the interaction perceptible while avoiding sudden transitions that would disrupt the composition.

Controlled musical layers:
Drums
Bass
Guitar
Pad
Texture
Voice

SYSTEM STRUCTURE

VISUAL INPUT
Camera feed
Gesture and facial tracking
Continuous movement data

AUDIO SYSTEM
Max/MSP control mapping
Multitrack gain automation
Multichannel audio playback

CURRENT OUTCOME

The prototype successfully demonstrated real-time control of multiple musical layers through visual gesture input. The next stage focuses on refining gesture recognition, improving the responsiveness and predictability of the mappings, and adapting the system for a larger multichannel installation environment.

Current status — Functional prototype and ongoing development

ONGOING THESIS DEVELOPMENT

Expanding the Prototype into a Larger Interactive Installation

This prototype serves as an early technical and conceptual foundation for a developing thesis project focused on interactive multichannel sound. The larger installation will explore how gesture-based control, spatial audio, and layered composition can create a shared experience in which audience movement influences the sonic environment without requiring conventional controllers.

← Previous Project

Home↗

Martain Ma