[x3d-public] X3D Audio and Sound minutes 7 OCT 2020: ChannelSplitter/Selector
Don Brutzman
brutzman at nps.edu
Wed Oct 7 12:19:49 PDT 2020
1. We discussed arity of ChannelSplitter/ChannelSelector nodes, i.e. number of inputs/outputs for each.
[1.1] arity
https://en.wikipedia.org/wiki/Arity
[1.2] MIMO (multiple input multiple output)
https://en.wikipedia.org/wiki/Arity
[1.3] Systems Analysis, Characterization of systems
SIMO, MIMO etc.
https://en.wikipedia.org/wiki/System_analysis#Characterization_of_systems
[1.4] W3C Web Audio API CR, 1.15. The ChannelSplitterNode Interface
https://www.w3.org/TR/webaudio/#channelsplitternode
*ChannelSplitter*
- W3C Audio API: single input, multiple output (SIMO) stream
- X3D4 Draft: single input, multiple output (SIMO) stream
*ChannelSelector*
- no corresponding W3C Audio API node
- Motivation: we are trying to define ChannelSelector as an X3D utility node
as a subset of ChannelSplitter functionality,
in order to avoid the need for Script node operations on MFNode arrays.
option a is simplest:
- X3D4 Draft: single input, single output (SISO) stream
SFInt32 [in,out] channelNumber 0 [0,∞)
currently implicit:
SFNode [output] outputChannel
which requires multiple instances for different channels.
Venturing further:
Note each SFNode input node might contain multiple channels. Do we want to expose them individually?
or option b:
- X3D4 Draft: single input, multiple output (SIMO)
MFInt32 [in,out] channelNumbers [] [0,∞)
MFNode [output] outputChannels
which produces an MFNode output.
OR option c:
ROUTE one element from an MFNode array, not currently possible in X3D, see next section.
OR a possible variation, option d:
- Single input, "multiple" outputs but each is the same to facilitate copying channels.
Possibly more (and more) confusion that it is worth - certainly it was confusing for us to go through all the variations! Also array operations are not necessarily needed if we can use DEF/USE to duplicate individual nodes.
We think that option A is best approach. Repeating/summarizing from above:
==============================
*ChannelSelector*
- no corresponding W3C Audio API node
- Motivation: we are trying to define ChannelSelector as an X3D utility node
as a subset of ChannelSplitter functionality,
in order to avoid the need for Script node operations on MFNode arrays.
- X3D4 Draft: single input, single output (SISO) stream
SFInt32 [in,out] channelNumber 0 [0,∞)
# Mechanisms for parent-child input-output graph design remain under review
SFNode [in out] inputs NULL [X3DSoundAnalysisNode,X3DSoundChannelNode,X3DSoundProcessingNode,X3DSoundSourceNode]
SFNode [in out] outputs NULL [X3DSoundAnalysisNode,X3DSoundChannelNode,X3DSoundProcessingNode,X3DSoundSourceNode]
==============================
---
2. adding ROUTE connections
We discussed how to add ROUTE capability. It seems possible for single nodes.
If we add the following to each sound-component node, then we could say that any node is either usable as a child itself, or as a ROUTE-able source.
SFNode [in out] outputs NULL
"The /outputs/ field can be used to ROUTE the output of a given node, which may have one or more channels."
If we do other than option a, we would not be able to do this for ChannelSplitter without changing one of the field names.
Example adapted from Efi's document:
<!--functionally equivalent to parent-child field relationship -->
<ROUTE fromNode=’ChannelMerger2’ fromField=’outputChannel’ toNode=’Gain20’ toField=’inputs’/>
---
3. ROUTE solely one element from within an MFNode array.
We discussed this possibility. Changing the nature of ROUTE is something that affects all of X3D. If it is to be considered, it should be thought about separately from Sound component.
Current approach to selecting any node from within an MFNode array is to ROUTE it to a Script, which can inspect the size of the input array and select one node for delivery to an output field and subsequent additional ROUTE connections.
If we want this kind of new expressive power, that is beyond scope of Sound component.
---
4. Stream sources and channels.
Striving for better clarity and crispness in our use of terminology:
- Nodes represent a stream source.
- Any stream source might have multiple channels.
Examples:
"The output destination stream has stereo channels."
"The input microphone source is monophonic and provides one channel."
"The stereo MIDI file is an input stream source with two channels, loaded in a single AudioClip node."
"The SIMO ChannelSplitter receives an input source and provides multiple output channels."
"The SISO ChannelSelector receives an input source and selects one and provides a single output channel."
etc. especially
"Each X3D4 Sound component node represents a stream source which may have one or more channels."
---
5. Name of 'gain field'
Do we want to use
SFFloat [in,out] gain 1 [-∞,∞)
or
SFFloat [in,out] gainOutput 1 [-∞,∞)
or
SFFloat [in,out] outputGain 1 [-∞,∞)
gain is name used by W3C Web Audio API but is also overloaded with X3D4 Gain node.
gainOutput/outputGain is more descriptive and avoids name collisions. It also makes it very clear that any amplification only occurs after all other node operations.
outputGain is easier to use in a sentence, "the outputGain is an amplification factor of 12 for this node."
Preferred choice: outputGain is our choice. We will update the specification.
---
6. controlling signal flow: enabled FALSE and muting
Discussion of enabled FALSE and (disable effect vs. muting), following up on last Sunday's email.
- disable effect for most nodes or mute?
- for Sound or SpatialSound, no enabled field is (yet) present.
For X3D nodes, might be simplest as follows:
- source: mute FALSE for silencing a sound source node (with no change in start/stop/pause/resume etc.),
- processing: enabled FALSE disables functions of a processing node, but does not block signal,
- destination: mute FALSE added as default to X3DSoundNode (Sound and SpatialSound)
TODO: updated working-draft X3D4 appropriately.
---
7. Unnecessary X3D fields.
X3DSoundChannelNode (ChannelMerger, ChannelSplitter, ChannelSelector) does not need any of the X3DTimeDependentNode fields.
16.3.2 X3DSoundChannelNode
X3DSoundChannelNode : X3DTimeDependentNode {
SFString [in,out] description ""
SFBool [in,out] enabled TRUE
SFFloat [in,out] gain 1 [-∞,∞)
SFBool [in,out] loop FALSE
SFNode [in,out] metadata NULL [X3DMetadataObject]
SFTime [in,out] pauseTime 0 (-∞,∞)
SFTime [in,out] resumeTime 0 (-∞,∞)
SFTime [in,out] startTime 0 (-∞,∞)
SFTime [in,out] stopTime 0 (-∞,∞)
SFTime [out] elapsedTime
SFBool [out] isActive
SFBool [out] isPaused
SFInt32 [ out] channelCount 0 [0,∞)
SFString [in,out] channelCountMode "max" ["max", "clamped-max", "explicit"]
SFString [in,out] channelInterpretation "speakers" ["speakers", "discrete"]
# Mechanisms for parent-child input-output graph design remain under review
MFNode [in out] inputs NULL [X3DSoundAnalysisNode,X3DSoundChannelNode,X3DSoundProcessingNode,X3DSoundSourceNode]
}
Instead we will go minimalist, adopting X3DSoundNode instead.
16.3.5 X3DSoundNode
X3DSoundNode : X3DChildNode {
SFString [in,out] description ""
SFFloat [in,out] gain 1 [-∞,∞)
SFNode [in,out] metadata NULL [X3DMetadataObject]
}
Updated:
4.4.2.3 Interface hierarchy
+- X3DSoundNode -+- Sound
| |
| +- SpatialSound
| |
| +- X3DSoundChannelNode -+- ChannelMerger
| +- ChannelSelector
| +- ChannelSplitter
|
+- X3DTimeDependentNode -+- TimeSensor (X3DSensorNode)*
| |
| +- X3DSoundAnalysisNode -+- Analyser
| |
| +- X3DSoundDestinationNode -+- AudioDestination
| | +- StreamAudioDestination
| |
| +- X3DSoundProcessingNode -+- BiquadFilter
| | +- Convolver
| | +- Delay
| | +- DynamicsCompressor
| | +- Gain
| | +- PeriodicWave
| | +- WaveShaper
| |
| +- X3DSoundSourceNode -+- AudioBufferSource
| +- AudioClip (X3DUrlObject)*
| +- MicrophoneSource
| +- MovieTexture (X3DTexture2DNode, X3DUrlObject)*
| +- OscillatorSource
| +- StreamAudioSource
resulting in following change, also in (ChannelMerger, ChannelSelector, ChannelSplitter)
16.3.2 X3DSoundChannelNode
X3DSoundChannelNode : X3DSoundNode {
SFString [in,out] description ""
SFBool [in,out] enabled TRUE
SFFloat [in,out] gain 1 [-∞,∞)
SFNode [in,out] metadata NULL [X3DMetadataObject]
SFInt32 [ out] channelCount 0 [0,∞)
SFString [in,out] channelCountMode "max" ["max", "clamped-max", "explicit"]
SFString [in,out] channelInterpretation "speakers" ["speakers", "discrete"]
# Mechanisms for parent-child input-output graph design remain under review
MFNode [in out] inputs NULL [X3DSoundAnalysisNode,X3DSoundChannelNode,X3DSoundProcessingNode,X3DSoundSourceNode]
}
Quick followup after teleconference completion: discussed with Dick, we should similarly apply to elimination of those fields by moving interface) to the following:
X3DSoundAnalysisNode (Analyser)
X3DSoundDestinationNode (AudioDestination, StreamAudioDestination)
Result:
+- X3DSoundNode -+- Sound
| |
| +- SpatialSound
| |
| +- X3DSoundChannelNode -+- ChannelMerger
| | +- ChannelSelector
| | +- ChannelSplitter
| |
| +- X3DSoundAnalysisNode -+- Analyser
| |
| +- X3DSoundDestinationNode -+- AudioDestination
| | +- StreamAudioDestination
|
+- X3DTimeDependentNode -+- TimeSensor (X3DSensorNode)*
| |
| +- X3DSoundProcessingNode -+- BiquadFilter
| | +- Convolver
| | +- Delay
| | +- DynamicsCompressor
| | +- Gain
| | +- PeriodicWave
| | +- WaveShaper
| |
| +- X3DSoundSourceNode -+- AudioBufferSource
| +- AudioClip (X3DUrlObject)*
| +- MicrophoneSource
| +- MovieTexture (X3DTexture2DNode, X3DUrlObject)*
| +- OscillatorSource
| +- StreamAudioSource
---
7. Tutorial preparation
Next week we will focus on our proposed Web3D 2020 tutorial and examples.
* https://web3d.siggraph.org/?s=tutorial
We will also plan when to record the tutorial video.
---
8. Deep breaths, are we finished??
Issues keep popping up - but we keep resolving them with high confidence. Impressive team efforts!
We meet again next week.
Thank everyone for continued steady efforts. Have fun with X3D4 Audio and Sound! 8)
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list