“Constellation No. 4 (for mobile device and headphones)” explores non-linear, immersive, and interactive music by utilizing the listener’s movement in space to change and manipulate the music they hear. The core of this piece consists of a series of audio channels placed in a virtual space like a Venn diagram, overlapping to create different sound zones that listeners can walk through, hearing various combinations of channels and variations of the music.
Click here for the live experience
Unlike other sound walks, this project does not use GPS positioning; instead, it relies on the mobile device’s orientation and acceleration data. This means the experience isn’t site-specific and can be enjoyed anywhere and without the need for the listener to cover great distances. Using this data, the application approximates the listener’s position in the virtual Venn diagram and manipulates the music accordingly. The application uses JavaScript along with the P5.js and Tone.js libraries, making it accessible on the web without requiring an app download.
The listener’s placement and orientation in the virtual space determine the volume of each channel, ranging from -∞ (no sound) to 0 (full volume), as well as the panning from 1 (hard right) to -1 (hard left). This creates an immersive experience in which every movement can entirely alter the composition.
Although primarily designed for mobile, this app also has a desktop version that allows the listener to navigate the virtual space by dragging the listener icon or setting it to “autopilot,” enabling it to wander randomly.
This project has inspired the creation of “vennWave” – an extended web application that allows musicians to upload their audio files and create their own sound maps, enabling users to walk through the music and share it with the world. This project responds to the emerging community of musicians interested in interactive, immersive, and non-linear music.
The “vennWave” editor utilizes the same technology but communicates with a database. The app employs Google Firebase to manage users (via Google login), allowing them to upload their music to storage and save their composition data. Once saved, the editor provides a shareable link that enables the audience to experience the composition on both mobile and desktop.
