]>
Extensible 3D (X3D) encodings
Part 3: Compressed binary encoding
4 Concepts
This clause describes key concepts in this part of ISO/IEC 19776. This includes how the X3D constructs defined in ISO/IEC 19775-1 X3D Architecture are encoded as a compressed binary file.
Table 4.1 lists the topics is this clause.
Table 4.1 — Topics in this clause
The following conventions are used throughout this part of ISO/IEC 19776:
Italics are used for event and field names, and are also used when new terms are introduced and equation variables are referenced.
A fixed-space font is used for URL addresses and source code examples. ISO/IEC 19776 Compressed binary encoding examples appear in bold, fixed-space font.
Node type names are appropriately capitalized ( e.g., "The Billboard node is a grouping node..."). However, the concept of the node is often referred to in lower case in order to refer to the semantics of the node, not the node itself ( e.g., "To rotate the billboard...").
The form "0xhh" expresses a byte as a hexadecimal number representing the bit configuration for that byte.
Normative references and informative Bibliography entries are linked via document identifiers.
EXAMPLE
Functional requirements are defined by
ISO/IEC 19775-1 X3D Architecture.
This X3D encoding provides a compact transmission format that minimizes delivery size and maximizes parsing speed while following the precepts of XML (see XML). Privacy and ownership protections are optionally provided using XML security mechanisms.
There is an essential one-to-one match between the nodes and fields of the abstract X3D scene graph and corresponding constructs in the Compressed binary encoding. This section describes the design patterns that govern the correspondence.
The following compression steps (as depicted in Figure 4.1) are performed as part of the Compressed binary encoding. The actual processing order and methods of accomplishment may vary between applications. An implementation may skip any step so long as the resultant compressed binary file is the same as if these steps were followed.
The following decompression steps (as depicted in Figure 4.2) are performed as part of the Compressed binary encoding. The actual processing order and methods of accomplishment may vary between applications. An implementation may skip any step so long as the resultant .X3D encoded file is the same as if these steps were followed.
TODO: has ISO finally restored the registry?
Need to confirm and correct the following references and links
(see Mantis 1439).
A registry of node compression methods is maintained under, and using the established procedures of, the ISO International Registration Authority for Items. A file conforming to this part of ISO/IEC 19776 includes only compressors that have been standardized or have been approved for inclusion in the registry. See Annex B Node compressor registry for more information on registered node compressors.
NOTE Contact information for the ISO-designated Registration Authority for Items registered under the ISO/IEC 9973 procedures is available at the ISO Maintenance Agencies and Registration Authorities web site: https://www.iso.ch/iso/standards_development/maintenance_agencies.htm.
ISO/IEC 24824-1 Fast Infoset specifies the technique used to encode a binary stream representing an X3D scene graph. This technique has the name Fast infoset. Conceptually, an XML-encoded canonical X3D document (possibly further modified by node compressors) is passed to a Fast infoset processor to write the binary stream.
An X3D file is structured as specified in ISO/IEC 19775-1 X3D Architecture.
Conceptually, the X3D scene input to the Fast infoset encoder is an XML-encoded document with certain restrictions.
X3D canonical form eliminates file ambiguities that have no impact on the 3D content but which otherwise would negatively impact security issues, information-theoretic compression, or parsing performance.
X3D canonical form is based on Canonical XML (see XML_CANONICALIZATION) which specifically allows modification to the default XML canonicalization rules. This provides the ability to establish equivalence between differently formatted (but functionally identical) XML documents. This approach also supports X3D models that are multi-namespace documents, including elements and attributes from other XML vocabularies (see XML_NAMESPACES). This capability is required for the application of XML Encryption (see XML_ENCRYPTION) or XML Signature (see XML_SIGNATURE) syntax and processing techniques.
The following X3D canonicalization restrictions are applied to an X3D scene (or scene fragment) prior to encryption, signature or compression:
EXAMPLE 1
<NavigationInfo type='"WALK" "EXAMINE" "ANY"'/>
′ character entity.
" character entity (and escaped by a leading backslash
"\" character).
EXAMPLE 1
<Text string=' "\"Hello, quotation marks\"" "Line 2
has no quotation marks" '/> displays the following
two lines:
“Hello, quotation marks”
Line 2 has no quotation marks
A default or substitute DOCTYPE statement
(as specified in Annex A of
ISO/IEC 19776-1) is
often
included following the
<?xml version="1.0" encoding="UTF-8"?>
header in canonical XML form.
The DOCTYPE statement is not included in the Compressed binary encoding.
Default or substitute
X3D schema attributes (see Annex B of ISO/IEC 19776-1) are
often
included in the root
<X3D> element.
X3D schema attributes are not included in the Compressed binary encoding.
EXAMPLE 2
2.004e3(equal to 2004.0).
DEF, USE and (non-default)
containerField
shall appear before other attributes, which then follow in alphabetic order.
This ordering typically provides higher parsing performance during subsequent decoding.
This step supersedes the XML canonicalization
rule that all attribute values are provided in alphabetic order.
All MFNode content for a child field shall be provided in a contiguous block with no intermixed containerField usage.
EXAMPLE 3 The following code exhibits X3D not in this form:
<Collision>
<Shape containerField="children" />
<Shape containerField="proxy" />
<Shape containerField="children" />
</Collision>
The proper child-element grouping for canonical form is:
<Collision>
<Shape containerField="proxy" />
<Shape containerField="children" />
<Shape containerField="children" />
</Collision>EXAMPLE 4 The construct:
<Group DEF="someDEF" class="someClass"></Group>
is converted to:
<Group DEF="someDEF" class="someClass"/>
The header follows the Fast infoset rules for header declaration as defined in 12.3 of ISO/IEC 24824-1 Fast Infoset.
A valid Compressed binary encoded file shall contain either an XML file declaration (see XML) followed by the Fast infoset header bits '11100000000000000000000000000000000' or just the Fast infoset header bits.
EXAMPLE 1 <?xml encoding='finf'?>'11100000000000000000000000000000000'
EXAMPLE 2 <?xml version='1.0' encoding='finf' standalone='yes'> '11100000000000000000000000000000000'
Other XML header elements like the DTD or schema definition are encoded using standard Fast infoset mechanisms.Two additional fields are appended to the X3D tag to denote Compressed binary version information. These attributes are:
<X3D type accessType binaryVersion="4.0"
serializerVersion="Fast infoset 2004"
>
</X3D>SFString
SFString
[initializeOnly]
[initializeOnly]
All Prototype-related elements are treated in a manner consistent with the representations of the X3D XML encoding. Thus, Prototype instances are treated as ProtoInstance node constructions in XML.
EXAMPLE
<ProtoDeclare name="ExampleSensor">
<ProtoInterface>
<field accessType="initializeOnly" name="enabled" type="SFBool" value="true"/>
</ProtoInterface>
<ProtoBody>
<TouchSensor description="touch to activate">
<IS>
<connect nodeField="enabled" protoField="enabled"/>
</IS>
</TouchSensor>
</ProtoBody>
</ProtoDeclare>
<ProtoInstance name="ExampleSensor">
<fieldValue name="enabled" value="false"/>
</ProtoInstance>
The following "native-tag" representation is not legal and shall not be used within the Compressed binary encoding:
<ExampleSensor enabled="false"/>
Node compressors are responsible for compressing a node and all its children. Typically, this compressor has domain-specific knowledge about how to compress a node's data.
EXAMPLE Geometric compression techniques may be used to store an IndexedFaceSet more compactly.
The compressor shall restore the exact same scenegraph structure as was compressed, including any ProtoInstance usage. In addition, no reordering of data is allowed; i.e., vertex values in a Coordinate node point array shall not change ordinal position.
A compressor shall only be assigned to handle a single node and all its children. Other constructs, like Prototypes and Routes, may not be compressed using node compressors. Furthermore, nodes that implement the X3DScriptNode or X3DProgrammableShaderObject shall not be assigned a node compressor. A node compressor can be used to compress the child nodes of these node types.
Node compressors shall not be nested. A provided node compressor shall handle all of the children nodes for that node type. A compressor may leave a node or field unaltered in the compressed stream.
A node compressor transforms a scenegraph fragment using the following steps:
EXAMPLE The following illustrates compressing an IndexedFaceSet
An IndexedFaceSet containing a MetadataString and a Coordinate node are to be compressed using a geometric compression algorithm (encoding 1). The input scene graph looks like this:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataString name="myData" value="myValue">
<Coordinate DEF="COORD" point="1.0 2.0 3.0, 4.0 5.0 6.0">
</IndexedFaceSet>
The geometric encoder compresses and encodes the coordIndex and point fields into the payload section. The resulting form sent to Fast infoset looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value= " the compressed and encoded coordIndex and point fields">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="rate" value="12:1"/>
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="1.0 2.0 3.0 4.0 5.0 6.0">
</IndexedFaceSet>
Then the final form parsed by a decoder engine of X3D files encoded in the Compressed binary encoding is:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataString name="myData" value="myValue"/>
<Coordinate DEF="COORD" point="1.0 2.0 3.0, 4.0 5.0 6.0">
</IndexedFaceSet>
An optional MetadataString may be specified before the encoding MetadataInteger name using the keyword "preserved". If the corresponding "preserved" keyword is specified, the decompressed form shall retain all ".x3db" MetadataSet data including the payload in addition to the uncompressed values in the original node. The user's metadata is retained but will not be returned to its original location.
EXAMPLE The following example illustrates compressing an IndexedFaceSet with preserved parameters
An IndexedFaceSet containing a MetadataString and a Coordinate node is to be compressed using a geometric-compression algorithm (encoding 1). The input scene graph looks like this:
<IndexedFaceSet coordIndex="...">
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="preserved" value="true">
<MetadataInteger name="encoding" value="1"/>
<MetadataSet name="Encoder Metadata">
<MetadataString name="lossy" value="true"/>
<MetadataString name="coordinateBits" value="15"/>
<MetadataString name="normalBits" value="6"/>
<MetadataString name="colorBits" value="9"/>
<MetadataString name="textureBits" value="9"/>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
The compressed X3D canonical form sent to Fast infoset looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value=" the compressed and encoded coordIndex and point fields">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="lossy" value="true"/>
<MetadataString name="coordinateBits" value="15"/>
<MetadataString name="normalBits" value="6"/>
<MetadataString name="colorBits" value="9"/>
<MetadataString name="textureBits" value="9"/>
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
The final form parsed by the X3D engine is:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="encoding" value="1">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="lossy" value="true"/>
<MetadataString name="coordinateBits" value="15"/>
<MetadataString name="normalBits" value="6"/>
<MetadataString name="colorBits" value="9"/>
<MetadataString name="textureBits" value="9"/>
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="1.0 2.0 3.0 4.0 5.0 6.0"/>
</IndexedFaceSet>
An X3D Compressed binary processor compressing a node that already contains a MetadataSet with the name ".x3db" and reference "Web3D" shall generate the payload block using the specified encoder and Encoder Metadata parameters. Any unknown or incorrect values may be ignored for compression purposes.
Further type-specific field compression is specified in 5 Encoding of fields. Table A.2.5 specifies the indices to use for each algorithm.
This part of ISO/IEC 19776 uses Fast infoset to serialize and compress an X3D document. It uses several techniques that reduce the size of an X3D document and that increase the speed of creating and processing such documents. These techniques are based on the use of vocabulary tables that allow small (typically) integer values (vocabulary table indexes) to be used instead of character strings.
EXAMPLE 1 Character strings that form the names of nodes or fields in an X3D document.
The tables that apply to this part of ISO/IEC 19776 are specified in Annex A Fast infoset tables.
A further optimization uses the encoding algorithm vocabulary table. This table identifies specialized encodings that can be employed for commonly occurring strings, again with a number of built-in algorithms.
EXAMPLE 2 If there is a string that looks like the decimal representation of an integer in the range -32768 to 32767, that string can be encoded by identifying that this vocabulary table is being used, giving the vocabulary table index, and then encoding the integer as a two-octet signed integer. Floating-point numbers and arrays of such numbers can be supported in the same way.
This part of ISO/IEC 19776 defines several unique encoders for X3D that utilize type-aware knowledge to further compress data. Details are specified in 5 Encoding of fields.
Use of XML Signature and XML Encryption is documented within the context of examples in Annex C Examples.
This part of ISO/IEC 19776 uses Fast infoset to deserialize and uncompress an X3D document. The tables that apply to this part of ISO/IEC 19776 are specified in Annex A Fast infoset tables.
The encoding algorithm vocabulary table is used to reference built-in algorithms to decompress field values.
This part of ISO/IEC 19776 defines several unique encoders for X3D that utilize type-aware knowledge to further compress data. Details are specified in 5 Encoding of fields.
Type-specific field decompression is handled by the registered field encoders specified in 5 Encoding of fields.
Type-specific field decompression is handled by the registered field encoders specified in 5 Encoding of fields.
Node compressors are responsible for decompressing a node and all its children. Typically, this decompressor has domain-specific knowledge about how to decompress a node's data.
The decompressor shall restore the exact same scenegraph structure as was compressed, including any ProtoInstance usage. In addition, no reordering of data is allowed; e.g., vertex values in a Coordinate node point array shall not change ordinal position.
A decompressor shall only be assigned to handle a single node and all its children. Other constructs, like Prototypes and Routes, may not be compressed using node compressors. Furthermore, nodes that implement the X3DScriptNode or X3DProgrammableShaderObject shall not be assigned a node compressor. A node decompressor can be used to decompress the child nodes of these node types.
Node decompressors shall not be nested. A provided node decompressor shall handle all of the children nodes for that node type.
A node decompressor transforms a compressed scenegraph fragment using the following steps:
If MetadataSet preservation was specified, see 4.4.3.3 Preservation.
Example The following illustrates decompressing an IndexedFaceSet:
An IndexedFaceSet node containing a MetadataString node and a Coordinate node are to be compressed using a geometric compression algorithm (encoding 1). The input scenegraph from FastInfoSet looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value ="myValue">
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value=" the compressed and encoded coordIndex and point fields as a binary payload block">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="rate"
value="12:1">
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD">
</IndexedFaceSet>
The .x3db MetadataSet is removed and the user's metadata placed back in its original position. Then the final form parsed by a decoder engine of X3D files encoded in the Compressed binary encoding is:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataString name="myData" value="myValue">
<Coordinate DEF="COORD" point="1.0 2.0 3.0, 4.0 5.0 6.0">
</IndexedFaceSet>
If the preserved keyword was used when encoding, the .x3db MetadataSet node will be retained in the decompressed stream. The user's metadata will be retained but will not be returned to its original position.
EXAMPLE The decompressed X3D form returned from Fast infoset looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value ="myValue">
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value=" the compressed and encoded coordIndex and point fields as a binary payload block">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="Binary Params">
<MetadataString name="lossy"
value="true">
<MetadataString name="coordinateBits"
value="15">
<MetadataString name="normalBits"
value="6">
<MetadataString name="colorBits"
value="9">
<MetadataString name="textureBits"
value="9">
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
After the geometry decompressor converts the payload into X3D, the final form parsed by the X3D engine is:
<IndexedFaceSet coordIndex="...">
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value ="myValue">
<MetadataInteger name="encoding" value="1">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="Binary Params">
<MetadataString name="lossy"
value="true">
<MetadataString name="coordinateBits"
value="15">
<MetadataString name="normalBits"
value="6">
<MetadataString name="colorBits"
value="9">
<MetadataString name="textureBits"
value="9">
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
The original user metadata is retained but now resides as a child of the MetadataSet with the name ".x3db".
An optional final step is gzip compression of the entire compressed scene as specified in RFC1952. This step supports global compression, while the other compression techniques described in this part of ISO/IEC 19776 are localized to individual chunks of data.
NOTE The application of gzip or zip compression to compressed binary encoded scenes can increase overall file size, and may have a negative impact on parser processing performance during decompression.
Compressed binary encoded files shall use the file extension ".x3db". The MIME-type associated with that binary compressed X3D encoded file is "model/x3d+binary".
Compressed binary encoded files that have also been "gzipped" shall use a file extension of ".x3db.gz". The MIME type is "model/x3d+binary". The content-encoding value is "gzip".
The concept of MIME types is defined in RFC2077.
X3D Inline nodes are allowed to reference glTF files in the url field, with expected file extensions of .gltf and .glb (glTF binary). Using .glb or other compressed file formats can sometimes reduce the overall file size of multiple models composed into a single world.
NOTE. Of related interest is that .glb files offer additional geometric compression that is independent of X3D techniques.