[x3d-public] Support for loading glTF in X3D players

Sun Jan 31 05:00:20 PST 2021

Titania supports conversion from gltf to X3D native nodes where the PBR shader is implemented in a proto for Appearance node.X_ITE does unfortunately not yet support gltf.Holger
-------- Ursprüngliche Nachricht --------Von: Andreas Plesch <andreasplesch at gmail.com> Datum: 29.01.21  17:31  (GMT+01:00) An: Michalis Kamburelis <michalis.kambi at gmail.com> Cc: Don Brutzman <brutzman at nps.edu>, Holger Seelig <holger.seelig at googlemail.com>, "Sturm, Timo" <timo.sturm at igd.fraunhofer.de>, doug sanden <highaspirations at hotmail.com>, vmarchetti at kshell.com, Nicholas Polys <npolys at vt.edu>, Anita Havele <anita.havele at web3d.org> Betreff: Re: Support for loading glTF in X3D players Good point about SHCubeMapTexture. SH in computer graphics really isonly used and useful for IBL, it appears.And 27 * 4 is 108 not 112. It was too late, something misfired.Cheers, -AndreasOn Fri, Jan 29, 2021 at 9:13 AM Michalis Kamburelis<michalis.kambi at gmail.com> wrote:>> Andreas,>> Thank you -- I agree with everything :) In particular I also saw the> spherical harmonics (instead of a cubemap texture) used for "diffuse"> in glTF IBL extension as something a bit non-standard. Most IBL> implementations I saw take a cubemap texture as input (whether it> means 1x DDS/KTX or 6x 2D image files). But as you say. this also> probably performs very well.>> This strenghtens my idea that we should support both options in X3D> EnvironmentLight.>> As for your approach of SHCubeMapTexture:>> - on one hand,I like it a lot, because it makes these 2 problems> independent :) I mean, then EnvironmentLight could just refer to> X3DEnvironmentTextureNode, and it's an independent decision whether we> add SHCubeMapTexture.>> - OTOH, it makes implementation harder. Right now all> X3DEnvironmentTextureNode descendants result in the same thing for GPU> - a cubemap sampler. So e.g. shaders don't care about the difference> between ImageCubeMapTexture, GeneratedCubeMapTexture,> ComposedCubeMapTexture . If we add SHCubeMapTexture as new> X3DEnvironmentTextureNode, then it has to be supported everywhere> where we allow a cubemap (not only at EnvironmentLight), and browsers> have to be prepared that this "different cubemap implementation"> requires different handling in shaders.>> Thank you for the references, reading :)>> Regards,> Michalis>> pt., 29 sty 2021 o 02:15 Andreas Plesch <andreasplesch at gmail.com> napisał(a):> >> > Yes, x3dom has Timo's PhysicalEnvironmentLight. It only accepts> > cubemaps in the form of DDS files.> >> > This has the advantage that it is simple and allows MIP levels and HDR.> >> > X3DEnvironmentTextureNode as in Michalis' proposal also allows DDS> > through ImageCubeMapTexture. This signature is more flexible> > (requiring more implementation work), and more similar to glTF which> > expects separate images for each side of the cube map.> > An x3dom implementation would require first implementing> > ImageCubeMapTexture and quite a bit of replumbing.> >> > The coefficients for irradiance are for spherical harmonics which> > allow for an extremely efficient representation of sampling levels of> > a spherical image. Apparently, these coefficients are easy to use in> > the shader and require only a few lines of computation. There are> > almost certainly sample implementations. It sounds like a nice option> > but I wonder how common the use of spherical harmonic coefficients for> > IBL really is. Maybe it is the standard now ?> >> > The main motivation for the glTF spec.extension to prefer 27> > coefficients over a cubemap must be efficiency, eg. savings in file> > size. 27 * 4  bytes is only 112 bytes. However, a comparable cubemap> > image should be also pretty small, relative to other content. 3 levels> > of spherical harmonics may correspond to only a 8x8 pixel image, times> > 6 for each face. 384 pixels maybe times 12 bytes is only 4.5 kbytes or> > so, uncompressed. So I am not sure if the added complexity is worth> > the savings.> >> > While generating the coefficients from a cube map seems a bit> > involved, using them in the shader to find a color for a given> > direction is apparently pretty straightforward. So these probably> > should be supported. The harder part may be how to exactly define> > these coefficients in the spec.> >> > Although coefficients are probably only useful for IBL, they are from> > a logical standpoint an implementation of X3DEnvironmentTextureNode.> > So one option would be to add a field MFFloat "SHcoefficients" to> > Image or ComposedCubeMapTexture or have a new node "SHCubeMapTexture"> >> > Hm, searching the github repo found this:> >> > https://github.com/KhronosGroup/glTF/pull/1850> >> > which is discussed as version 2 of the extension. Probably worth> > studying. KTX2 as container image format may need to be supported.> >> > https://github.com/KhronosGroup/glTF/pull/1377 is the original discussion.> >> > https://github.com/KhronosGroup/glTF/pull/1612#issuecomment-495737070> > has the math for SH.> >> > More generally, I am not sure how strongly the glTF extension should> > guide an X3D EnvironmentLight node. It is only an extension and most> > glTF models do not use it expecting the renderer to provide their own> > idea of IBL.> >> > -Andreas> >> >> > On Thu, Jan 28, 2021 at 7:30 AM Michalis Kamburelis> > <michalis.kambi at gmail.com> wrote:> > >> > > X3DOM, as far as I know, implements> > > https://doc.x3dom.org/developer/x3dom/nodeTypes/PhysicalEnvironmentLight.html> > > . Which follows the original design of Timo Sturm, and it was the> > > first approach to IBL in glTF (it was also used in first glTF sample> > > viewer implementation of IBL), with "diffuse" being a cubemap.> > >> > > The new glTF approach to IBL is a bit different (> > > https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based> > > ). In particular in new glTF approach, the "diffuse" (which is now> > > called "irradiance") is defined by coefficients, not a cubemap> > > texture.> > >> > > This is an important change, and it's exactly what I describe on> > > https://github.com/michaliskambi/x3d-tests/wiki/Image-Based-Lighting-(EnvironmentLight-node)> > > :) We should find (and test!) a way to reconcile these approaches, or> > > just use the new (from glTF extension) one. So that the new> > > EnvironmentLight> > >> > > 1. is comfortable for authors,> > > 2. matches glTF extension for IBL,> > > 3. is implementable in X3D browsers without much pain.> > >> > > And note that we should do more than glTF. EnvironmentLight  in X3D> > > should work with both PhysicalMaterial and Material (because it can,> > > and because other lights also do), on both normal X3D shapes and on> > > X3D shapes imported from glTF.> > >> > > I welcome Andreas (and other implementors) comments about this. Maybe> > > we can design a new EnvironmentLight and make it work in e.g. X3DOM> > > sooner?> > >> > > I emphasize, I do *not* require that EnvironmentLight must be> > > implemented in CGE/view3dcene before it is added to the spec. I am not> > > special, I don't want you to wait for me with something :)> > >> > > But I *do* require that EnvironmentLight (in the form matching X3D> > > spec, and matching glTF IBL extension) must be implemented in *some*> > > browser, and we must have *some* testcases.> > >> > > Regards,> > > Michalis> > >> > > czw., 28 sty 2021 o 06:57 Don Brutzman <brutzman at nps.edu> napisał(a):> > > >> > > > Thanks for status updates Andreas, very helpful and promising.> > > >> > > > Michalis, given this additional interesting status:> > > >> > > > On 1/27/2021 6:44 PM, Andreas Plesch wrote:> > > > > X3DOM also has a working EnvironmentLight node for image based> > > > > lighting (IBL). I believe it only affects PhysicalMaterial nodes which> > > > > are used internally for glTF models.> > > >> > > > If you think we can have a reasonable "first draft" approximation of EnvironmentLight field signature and functional definitions, it is likely preferable to get this into the CD specification (thereby inviting continued comment and implementation discussions) rather than waiting until you are able to pursue your implementation.> > > >> > > > The glTF definition looks reasonably well defined and stable, and if they change we will match.  If you want, I'll try to draft it.> > > >> > > > Seems like there is little downside and some good positives if we put that draft node in...> > > >> > > > Please advise what you think, thanks for reconsidering it.> > > >> > > > all the best, Don> > > > --> > > > Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu> > > > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149> > > > X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman> >> >> >> > --> > Andreas Plesch> > Waltham, MA 02453-- Andreas PleschWaltham, MA 02453
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20210131/7971f097/attachment.html>