Re: [x3d-public] Heilan X3D Browser Released

[Niall Moody <niallmoody@yahoo.co.uk>]

I think the obvious omission from my point of view is the lack of a realtime audio input node.  That would pave the way for VOIP and, with something like Jack (http://jackaudio.org/) on Linux and OSX (I believe there's even a version for Windows now) allow for separate apps to pipe audio into scenes.  I have to admit I have very little experience with doing this kind of thing over the internet though, so I've no idea how hard it would be to actually implement the streaming of audio to/from multiple clients.

Related to this, though probably not as important to most people, is the ability to stream control data from audio apps.  Heilan uses OSC (http://www.cnmat.berkeley.edu/OpenSoundControl/) for this, and with software such as Max/MSP (http://www.cycling74.com/products/maxmsp) there are huge possibilities for live performance, particularly if that software is also streaming audio into the scene.  A simple example would be to just move sounds around the 3d space, but you could take it further and have the performer also manipulating the (visual) scene itself.  And of course, programs like Max have a lot of support already for alternative hardware interfaces, as well as numerous ways of manipulating the data from them.

Another thing that strikes me as fairly obvious is reverb/ambience.  It seems to me that X3D should specify some way of defining how sound interacts with the various surfaces in a scene.  A simplistic implementation could just specify bounding boxes within which certain reverb parameters apply (this box sounds like a cave, that one sounds like a cathedral...).  Or, taking the idea to its logical conclusion, you could have a system where you can specify the acoustic properties of different surfaces (this wall damps the sound, that floor reflects it...).  That's probably far too ambitious, and definitely overkill for what most people would want to use X3D for, but from a composer's point of view, the ability to actually create spaces with their own acoustic character within which sounds can be placed and moved (as opposed to fiddling with the parameters of a reverb plugin and a panpot, say) would be incredibly useful.

Personally, I'd also like ways of creating audiovisual mappings in an X3D scene (say you have a node that takes the amplitude envelope of an AudioClip, and constantly outputs events with the current state of the envelope - you could then route that to the size of a Box node, and have a simplistic audio visualisation).  I'd guess most people would really have no use for such a feature though, and with realtime audio input and OSC support, it could be easily faked with outside software.

Oh, and low latency audio is a must if realtime performance is involved, though obviously if you're doing this over the internet, there are other factors which will pretty much negate any benefits from choosing to use ASIO over DirectSound drivers etc.


- Niall.


----- Original Message ----
From: Len Bullard <cbullard@hiwaay.net>
To: giles@oz.net; Niall Moody <niallmoody@yahoo.co.uk>
Cc: x3d-public@web3d.org
Sent: Saturday, 2 December, 2006 11:27:55 AM
Subject: RE: [x3d-public] Heilan X3D Browser Released

++1.

Although, off the top of my head, I'm not finding any particular rendering
deficiencies other than no support for mp3 or some other advanced
compression that is supported by middle tier toolkits such as Adobe
Audition.  I find that the size barrier is the most basic problem.  Then the
availability which is a chops, market, and copyright combination and can't
be solved by X3D.  Audio rendering is best achieved by using decent sample
sources and sizes.  Again, for a complex sound, going below 22khz is deadly.

There are some basic VRML/X3D building techniques that can be better
proselytized so graphics artists who are not audio specialists can build
faster.  For example, I've been working on simple techniques for making the
ambient background sounds hush when a room is entered and using the audio
isActive events to trigger animation effects to get around
startTime/stopTime unreliability.  These are just practice though.

Streaming definitely.  Server support for VOIP naturally.

The use of audio creates so much serendipity and potential for VR that this
is not in doubt.  For increasing the reach into other markets such as the
immersive album, this is definitely possible and actually pretty simple to
do.  It is a genre that can be developed quickly and pushed out cheaply.
This one is really a matter of access to the market moguls.

len



From: owner-x3d-public@web3d.org [mailto:owner-x3d-public@web3d.org] On
Behalf Of Alan Hudson

Audio rendering should be as expressive as the graphics.







Send instant messages to your online friends http://uk.messenger.yahoo.com 

-------------------------------------------------------------------------
for list subscription/unsubscription,
go to http://www.web3d.org/cgi-bin/public_list_signup/lwgate/listsavail.html