43.1 Introduction

43.1.1 Name

The name of this component is "GaussianSplats". This name shall be used when referring to this component in the COMPONENT statement (see 7.2.5.4 Component statement).

43.1.2 Overview

This clause describes the Gaussian splats component of this document. This includes how Gaussian splats are specified and how they are presented in the 3D scene. Table 43.1 provides links to the major topics in this clause.

43.2 Concepts

43.2.1 Summary

This component integrates Gaussian splats extensions that provide support for direct real-time radiance-field rendering of volume data, without first converting into surface or line primitives.

3D Gaussian splatting is a rasterization-based technique used in the field of real-time radiance fields for representing and rendering photorealistic 3D scenes from a sparse set of 2D images. It enables the creation of high-quality real-time scenes by combining multiple photos or videos, computationally producing new data structures for 3D representation (see GAUSSIAN_SPLATS).

The method represents scenes that retain properties of continuous volumetric radiance fields, integrating sparse points produced during camera calibration. It introduces an anisotropic representation using 3D Gaussian splats to model radiance fields, along with an interleaved optimization and density control of the Gaussian splats. These complex mathematical representations are suitable for implementation using a fast visibility-aware rendering algorithm supporting anisotropic splatting, tuned for efficient real-time rendering.

NOTE   Gaussian splats are based on radiance fields, not voxels. Thus this component is distinct from the Volume rendering component.

43.3 Abstract types

43.3.1 X3DGaussianSplatsNode

X3DGaussianSplatsNode : X3DChildNode, X3DBoundedObject {
  SFBool   [in,out] bboxDisplay      FALSE
  SFBool   [in,out] castShadow       FALSE
  SFString [in,out] colorSpace       "SRGB_REC709_DISPLAY" [ "SRGB_REC709_DISPLAY" | "LIN_REC709_DISPLAY" ]
  SFNode   [in,out] metadata         NULL                  [X3DMetadataObject]
  SFBool   [in,out] pointerEvents    FALSE
  SFBool   [in,out] visible          TRUE
  SFVec3f  []       bboxCenter       0 0 0                 (-∞,∞)
  SFVec3f  []       bboxSize         -1 -1 -1              [0,∞) or −1 −1 −1
}

This abstract node type is the base type for all node types that specify Gaussian splats. It is based on the glTF 2.0 KHR_gaussian_splatting extension (see GLTF_GAUSSIAN_SPLAT) for storing 3D Gaussian splat information, treating data as point clouds for purposes of storage.

The castShadow field defines whether this Shape casts shadows as produced by lighting nodes. If the visible field is FALSE, then the Shape does not cast any shadows, regardless of the castShadow value.

TODO do we need to addcastShadow in Volume rendering component?

The colorSpace field specifies the color space of the reconstructed 3D Gaussian Splat color values. The color space is typically determined by the training process for the splats. This color space value only applies to the 3D Gaussian splatting data and does not affect any other color data (see GLTF_GAUSSIAN_SPLAT).

The pointerEvents field indicates whether this GaussianSplats node is a target for pointer events, if FALSE then it is ignored during pointer picking.
(TODO Editors note: should this new field also appear elsewhere? or is TouchSensor sufficient?

EXAMPLE:  GaussianSplats

GaussianSplats {
  // TODO
}

43.4 Node reference

43.4.1 GaussianSplats

GaussianSplats : X3DGaussianSplatsNode {
  SFBool       [in,out] bboxDisplay      FALSE
  SFBool       [in,out] castShadow                     FALSE
  SFString     [in,out] colorSpace                     "SRGB_REC709_DISPLAY" [ "SRGB_REC709_DISPLAY" | "LIN_REC709_DISPLAY" ]
  SFNode       [in,out] metadata                       NULL                  [X3DMetadataObject]   
  MFFloat      [in,out] opacities                      [] 
  MFQuaternion [in,out] orientations                   []
  SFBool       [in,out] pointerEvents                  FALSE
  MFVec3f      [in,out] positions                      []
  MFVec3f      [in,out] scales                         []                    [0,∞)
  MFVec3f      [in,out] sphericalHarmonicsDegree0Coef0 []
  MFVec3f      [in,out] sphericalHarmonicsDegree1Coef0 []
  MFVec3f      [in,out] sphericalHarmonicsDegree1Coef1 []
  MFVec3f      [in,out] sphericalHarmonicsDegree1Coef2 []
  MFVec3f      [in,out] sphericalHarmonicsDegree2Coef0 []
  MFVec3f      [in,out] sphericalHarmonicsDegree2Coef1 []
  MFVec3f      [in,out] sphericalHarmonicsDegree2Coef2 []
  MFVec3f      [in,out] sphericalHarmonicsDegree2Coef3 []
  MFVec3f      [in,out] sphericalHarmonicsDegree2Coef4 []
  MFVec3f      [in,out] sphericalHarmonicsDegree3Coef0 []
  MFVec3f      [in,out] sphericalHarmonicsDegree3Coef1 []
  MFVec3f      [in,out] sphericalHarmonicsDegree3Coef2 []
  MFVec3f      [in,out] sphericalHarmonicsDegree3Coef3 []
  MFVec3f      [in,out] sphericalHarmonicsDegree3Coef4 []
  MFVec3f      [in,out] sphericalHarmonicsDegree3Coef5 []
  MFVec3f      [in,out] sphericalHarmonicsDegree3Coef6 []
  SFBool       [in,out] visible                        TRUE
  SFVec3f      []       bboxCenter                     0 0 0                 (-∞,∞)
  SFVec3f      []       bboxSize                       -1 -1 -1              [0,∞) or −1 −1 −1
}

The GaussianSplats node provides support for direct real-time radiance field rendering of volume data, without converting into surface or line primitives.

The orientations field values correspond to the orientation of each splat value in local space. Orientation values are stored as unit quaternions in the usual order (x,y,z,w).

The positions field defines the position each individual Gaussian splat. (TODO do we need to discuss mean vector??) The mean vector for the Gaussian splat is provided by the positions of the mesh primitive. This defines the center of the Gaussian splat ellipsoid in local space. The effective global mean vector for the Gaussian splat is derived from the positions field value and the global transformation matrix of the X3D node that instantiates the mesh containing the splat primitive as defined in the glTF specification.

The scales field correspond to the spread of the Gaussian along its local principal axes. Scale values are linear and nonnegative.

The opacities field defines the opacity of each individual Gaussian splat. Values are normalized between 0.0 (transparent) and 1.0 (opaque).

The sphericalHarmonicsDegreeNCoefN fields provide spherical harmonics data defining the Gaussian splats values. Note that lower degrees of data must be present if higher degrees of data are defined.

43.5 Support levels

The Texture projection component defines levels of support as specified in Table 43.2.