Research and Development
Synthecology's primary foundation is the CAVE
, but the project presented a number of new technical challenges.
Some of this work was specifically oriented toward this specific artwork, though much of it was done with the
additional goal of advancing the expressive capabilities of immersive virtual environments for artists generally. Synthecology
showcased the ATS VRLab's recent research in spatialized audio for immersive environments, as well as powerful and reusable
components ranging from particle systems to web interfaces.
The core goal of Synthecology was to create a multi-user immersive virtual environment for creating collaborative,
evolving spatialized soundscapes. Beginning with the CAVE as display system and the Ygdrasil programming language, we
identified several key technologies that needed to be developed or integrated into Synthecology:
- Spatialized audio and real-time synthesis
Synthecology is essentially an environment for building virtual soundscapes, in which the illusion of spatial location of
sounds is crucial. For a large public installation, a multiple-speaker array approach offers advantages over binaural headphone
playback. Additional requirements included both sample playback and realtime synthesis, reasonable performance (approx 30 to 50 spatialized sound sources at any one time), usability on reasonably common off-the-shelf hardware, easy configuration for variable
size speaker arrays, support for real-time DSP (such as reverb), integration into the Ygdrasil and CAVE system, and reusability.
While the research on multichannel spatialization is substantial, a review of existing solutions found nothing that met our needs.
The solution, building on ongoing research at the ATS VRLab in virtual audio, was to develop an extensible sound server for the CAVE
using SuperCollider, controlled using Open Sound Control and with a simple node interface in Ygdrasil and a testing GUI in Max/MSP.
- Database integration
Many virtual environments use sound by loading a set of defined sounds at startup, or by defining relationships between sounds and
objects or events. In Synthecology, the set of sounds is not predefined but is constantly evolving and growing as new sounds are
contributed over the web. Participants sending sounds over the web can provide additional information such as a category or visual
association for the sound when it appears in the virtual environment; we needed a way to get all this information from the web into the CAVE. We used a MySQL database to store the sounds and all their metadata, and wrote Ygdrasil nodes for reading from the database, while using the existing CAVERN framework for distributing information between networked CAVE's.
- Database and soundserver management layer
To aid in integrating Ygdrasil, SuperCollider, and MySQL, we developed an additional software layer in Python. This centralized much
of the most complex control and logic in the system, and improved performance by removing tasks such as time-costly SQL queries from
both the SuperCollider and Ygdrasil code.
- Ygdrasil Dynamics
There were several important aspects to Synthecology that were difficult to implement using existing capabilities in the Ygdrasil language. Wind was particularly important. Since the primary activities involved planting, growing, arranging, handling, and sharing
objects, it was crucial for the objects to exhibit characteristics such as weight and gravity. Additionally, the objects had to
change the way they behaved as they grew through different states. To solve these challenges, we developed a rudimentary system for
physics simulation, a general-purpose particle system, and a new multiplexor node that can be used to easily build complex
state machines in Ygdrasil.
- Flash front end
While Synthecology is centered on the idea of group interaction and collaboration, the specialized hardware of the immersive display itself limits the number of participants. As a way of addressing this issue, we created an additional way to enter the Synthecology world using a simplified web interface in Flash. The Flash interface and the primary CAVE world were connected using the MySQL database, raising the intriguing possibility of creating shared virtual worlds with interfaces ranging from the CAVE to cellphones.
For the full paper on the sound server development for Synthecology, see
- CAVE is used as shorthand here to additionally refer to the larger class of projection-based immersive display systems.
The public display system for Synthecology was a single-wall passive-stereo rear-screen projection system with eye and hand tracking,
often called a C-Wall or GeoWall. Some of the participants in Synthecology connected using standard desktop PC's with
just a DSL internet connection - but the same CAVE software is used on all these systems.