Difference between revisions of "X3D Binary Compression Capabilities and Plans"

Revision as of 08:49, 29 March 2013

Motivation: Lots of work has been accomplished using the X3D Compressed Binary Encoding (CBE) standard for a wide variety of Web-based 3D graphics usages. X3D has coherent solutions already available that currently meets numerous requirements for a general 3D transmission format. We estimate that we are currently at 80% of the way there... more progress is possible. Further improvements and standards-based partnerships are possible for achieving broader industry interoperability.

Call For Contributions

We are building a draft 2013 Call For Contributions to further advance these technical capabilities for X3D.

• The previous RFP can serve as a baseline
• We will request public review of past comments
• We will issue a Call For Contributions in early April, then review submissions at the Web3D Conference in Juene

RFP, IPR, Requirements and Use Cases

• The original August 2003 X3D Compressed Binary Encoding Request For Proposals (RFP).
  • Intellectual Property Rights (IPR) protections for X3D specification.
• Patented technologies can be considered, but only when eventual use will be royalty free for X3D use (if eventually accepted).
  • Submitters can restrict access to patented submissions during member-only working group review, if desired.

Defining example usages and authoring requirements for compressed X3D scenes is helpful, providing useful criteria for technical capabilities. Use cases enable us to determine whether each possible improvement effectively meets a declared need.

• TODO Web3D members will review past work and provided an updated set of use cases
• TODO Consider whether we also want to include a call for further use cases in our forthcoming public Call for Contributions

Existing Compression Usage for X3D and VRML97

A solid foundation exists for continued progress.

• Approved ISO standard Compressed Binary Encoding (CBE) for X3D.
  • Based on ISO standard Fast Infoset (FI) for XML compression.
• Optional, alternative gzip compression and MIME Type definitions for X3D.
  • XML encoding (.x3dz/.x3d.gz), ClassicVRML encoding (.x3dvz/.x3dv.gz) and Compressed Binary encoding (.x3db.gz) file extensions.
• Optional, alternative gzip compression for VRML97.
  • Original compression technique of applying gzip to .wrl compressed VRML97 files was called .wrz.
  • This emerged as a common practice when gzip was originally used. No formal specification of .wrz or corresponding mime type was produced.
  • Occasionally authors might also gzip .wrl files while retaining the .wrl file extension.

Polygon Reduction and Geometric Compression

Formal specifications:

• The X3D Compressed Binary Encoding defines extensible, repeatable compression and decompression algorithms for data-flow production chains. Multiple type-specific field encoders and geometric compression algorithms can be used in concert, but no specific geometric compression algorithms are referenced or required.
• X3D Canonicalization (C14N) provides standardized formatting so that digital signatures are not thwarted by whitespace variations.

Best practices and examples:

• CAD Distillation Filter (CDF) technique allowing successive refinement of large X3D scenes into tighter X3D scenes.
• A highly effective exemplar algorithm for Coding Polygon Meshes as Compressable ASCII is demonstrated in the Experimental Binary Compression examples.
• Multiple CAD Distillation Filter (CDF) algorithms implemented in Xj3D and X3D-Edit as open-source.
• The Chisel VRML Optimisation Tool has an excellent set of data-reduction tools for VRML that are worth repeating for X3D.
• Multiple other conversion and translation tools available with supporting capabilities.
• Following submission of candidate technologies, we will build a table of 3D graphics compression benchmark results, providing a full comparison of existing tools (compression ratio, computational complexity, memory usage, etc.)
• TODO: MPEG4 effort
• TODO: Fraunhofer effort

Data-Centric Binary Encodings

The X3D Compressed Binary Encoding (CBE) uses the ISO Fast Infoset (FI) standard to compress XML information, which is how the CBE maintains equivalent expressive power with other X3D scene encodings.

• Plan to add a further-improved X3D Compressed Binary Encoding using now-approved W3C Recommendation for Efficient XML Interchange (EXI).
• Web3D contributed to EXI working group and XML Binary Characterization working group efforts.
  • 3D Model Compression, Serialization and Transmission use-case requirements are defined and met by EXI.
• Design includes compatibility with CDF techniques, XML Encryption, and XML Digital Signature for author authentication.
  • Relevant example scenes maintained as part of X3D Basic Examples Archive - Security.
• These capabilities meets most needs of digital authors for digital rights management (DRM).
• Following submission of candidate technologies, we will build a table of data compression benchmark results, providing a full comparison of existing tools (compression ratio, computational complexity, memory usage, etc.)

Network Streaming

• Multiple capabilities are already available in X3D for flexible network transmission.
  • Anchor, Inline, LOD, LoadSensor, Script and Prototype nodes support successive retrieval of content once initial model is displayed.
• TODO: Willing to consider further addition of progressive-mesh geometric streaming technologies, perhaps by a public Call for Contributions. Note however that an effective progressive-mesh approach is likely dependent on the availability of a corresponding mesh compression algorithm.

• TODO: Use cases and test examples can demonstrate whether http/https and local-file url retrieval are sufficient for a network protocol.
  • Other network protocols (Web sockets, P2P channels, etc.) might be possible, but only if security restrictions can be handled satisfactorily.
  • Javascript Object Notation (JSON) might be suitable for simple streaming of X3D scene-graph data via Script node or external HTML page, which can be useful for progressive mesh and other incremental network-update approaches. Such possibilities are being actively discussed on the x3d-public mailing list.
  • JSON might have other uses as well, see Khronos work on Collada2JSON and glTF.
  • Further investigation needed.

X3D Implementations

The following tools implement the X3D Compressed Binary Encoding (CBE) standard.

• Two independent open-source implementations available.
  • C++ codebase: XIOT X3D Input Output Tool library.
  • Java codebase: Xj3D.
• Ongoing status is maintained on Web3D-member wiki pages:
  • Player support for X3D components
  • Tool support for X3D components
• Several thousand X3D Examples are available in .x3db form, encoded by Xj3D.

We do not currently have a compression benchmark suite. We plan to automate conversions and comparisons, using Xj3D XIOT and other submissions, for cross-check interoperability and validation testing. We intend to automate such a suite using our many example scenes and nightly build (continuous integration) processes. This will automate the production of results summarized in the compression performance tables constructed as part of the call for technologies.

Looking Ahead

• Web3D's X3D and CAD Working Groups each have member commitments to pursue this continued innovative work in 2013.
• We are keen to consider common, sharable technical strategies with MPEG-4 and Collada. Web3D Consortium has been waiting for a response from The Khronos Group and MPEG standards SC 29 committee since a joint meeting at SIGGRAPH conference August 2012. To date, formal MPEG4 communications have indicated that a royalty-free solution is not possible.
• TODO: An open meeting on 3D Transmission Formats might be a useful way to compare candidate technologies to complete these capabilities. We are preparing to hold a collaboration workshop immediately preceding the Web3D 2013 conference on Wednesday 19 June 2013 in San Sebastian Spain.