Open Standards for Real-Time 3D Communication

X3D/VRML has made great strides as standard for creating complex synthesized 3D worlds and visualizations. However it is a different story when it comes to the audio and sound manipulation using X3D. In general, the 3D community has struggled with how to link X3D entities and environments to interactive audio control. We've started a project to integrate FreeWRL with ReWire. This project has the potential to introduce X3D to those who are actually into making or manipulating music, beyond the traditional X3D coder base!

X3D/VRML is a proven standard for creating complex synthesized 3D worlds and visualizations. The spec has evolved to include increasingly sophisticated visual capabilities including particle systems, physics, realistic motion, enhance UI and more. However it is a different story when it comes to the audio and sound manipulation using X3D. The capabilities of the X3D AudioClip Node have not evolved beyond the DirectedSound and PointSound nodes of VRML Version 1.1.

In general, the 3D community has struggled with how to link X3D entities and environments to interactive audio control.

At CRC we’ve previously done some experimenting with shared interactive audio streaming. A few years ago we created a protocol that lets the FreeWRL VRML/X3D web browser control an audio application. It simply sent orientation and position values, and the application did the actual audio. We used a Multicast audio tool, called RAT from UCL in London. The protocol was one we created, MVIP-II. With MVIP, dynamic audio groups would form and disperse; no explicit join required. It was a very interesting experiment as the audio increased realism, without adding complexity to the user interface. (See the papers from the 2000 and 2003 Web3D conference proceedings.)

What I’m doing with audio right now is to make FreeWRL a “ReWire” client, allowing it to interact with Propellerhead’s Reason.  (Reason is a very popular virtual studio system. ReWire is the associated communications protocol for sharing timing and control/audio data).

In “ReWire” terms - FreeWRL is becoming a Mixer application, allowing it to directly interact with pro audio tools for sound (sampler) data, and for using musicians control surfaces for I/O control. By offloading the audio processing to a ReWire device, we enable X3D entities to use state-of-the-art real-time, audio processing in X3D environments.  So for example, a creator of a virtual world could trigger self-modulating audio based on user location and orientation in a virtual world.

It’s pretty neat - the flexibility is amazing, and the audio tools are mind-boggling. This FreeWRL-ReWire code has the potential to introduce X3D to those who are actually into making or manipulating music, beyond the traditional X3D coder base!

I’ll be presenting a short paper for the Web3D 2007 conference on this work.