Research Projects: Sonic Panoramas
This work was motivated largely by two areas of inquiry. The first is in developing compositional techniques for real-time interactive sound environments, such as those required in immersive art and VR experiences. A second area of investigation in this work concerns the ways in which humans perceive, understand, and represent physical landscapes. The objective is to enrich a participant’s experience of space through sonic interpretations of visual landscapes, providing a multi-modal interface for data exploration. The user’s physical movement through the immersive projection space is tracked in real-time and used to generate a position-specific visual and auditory representation.
The history of formalism in music composition reflects interest in the application of formal structures to generate innovative and variable works of art from an abstracted analytical aesthetic. These aesthetic and formal structures have included liturgical chants, the rolling of dice, and serial pitch sequences among others.
The use of natural and manmade landscapes as compositional source material is explored in this work. Countless composers such as Debussy and Ravel have incorporated inspiration from nature in their work. Most commonly, composers have attempted to mimic sounds of nature in their compositions as opposed to representing visual perception of space. The World Soundscape Project was a study initiated by Shafer in the late 1960’s, whose main activity was the collection of recorded “found” sounds as a sonic mapping of the physical environment [Shafer]. Their approach was largely to gain perspective on the physical world and social change by becoming an objective listener to naturally occurring sounds, including less natural forms of noise pollution.
The sonification of visual landscape representations poses a unique set of challenges. An observation commonly made is that the influence of sound or “musicality” in painting is not matched by equal or compelling reflection of painting in music. The various color organ experiments of the early 20th Century and Oskar Fischinger’s Lumigraph are often pointed to as examples of experiments in synaesthesia. However, these examples demonstrate the desire to create visual displays that are “musical” in nature or influenced by accompanying music. We explore the reverse: creating sound from visual source material.
Our data representation employs the mapping of visual information into sound, coupled with a visual representation of that sound’s relationship to the source material. In addition, social and collaborative understandings of physical-auditory space are important considerations. This project focuses on the representation of an abstracted space through the user’s navigation of that abstraction, a panoramic digital image. The goal is to develop new techniques to communicate and understand the complex physical and ephemeral qualities of landscape through the synergy of motion, sound and vision.
We would like to thank Elizabeth Chattin, Christopher Dobrian, Robert Nideffer and Kevin Ponto for their consultation, support and input through the course of this project.
- Kabish, E. and Kuester, F. (2005). Sonic panoramas: Experiments with interactive landscape image sonification. In Proceedings of the 15th International Conference on Artificial Reality and Telexistence (ICAT 05).
Related Resarch Projects
This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by the author's copyright. This work may not be reposted without the explicit permission of the copyright holder.