[x3d-public] X3D and AR

[GPU Group]

I've done VR HMD on android phone with x3d and a Google Cardboard headset.
AR is different -- Background is live stream from front facing camera?
That might need a special Background node?
Otherwise X3D doesn't usually interfere with custom navigation.
For the math to orient, in theory it takes 3 external points to be aligned
to solve for 7 degrees of freedom (3 translations, 3 rotations, 1 scale),
but with some known constraints such as which way is up, or scale, can
shave a few degrees off and 2 points or 1 point and some distance
adjustment can solve.
For example when orienting the scene to the real world, somehow you would
press something to 'freeze' the x3d scene on the screen while the
background is still moving,  then once you have the x3d scene aligned with
the real world at that point release the button. Do that with one or 2 more
points depending on how many degrees of freedom you ar solving.
-Doug

On Wed, May 11, 2022 at 11:15 AM Andreas Plesch <andreasplesch at gmail.com>
wrote:

> There is some interest in using WebXR for augmented reality (AR) on
> phones. WebXR is browser built in functionality to help with VR/AR and
> provides pose information (for phone/headset), or hit testing
> (locating a real object in the look direction).
>
> What are your thoughts on if X3D is useful for AR as it is and if
> there is opportunity or need for AR specific X3D nodes such as
> sensors, or navigation modes ?
>
> AR on the phone is meant as an overlay over the real world shown as a
> video feed background from the frontal camera. The overlay could be an
> X3D scene. The phone would act as a viewpoint, eg. one moves and looks
> around by moving and rotating the phone. This could become a new first
> person, non-mouse, non-touch, navigation mode ? Or is it just Walk
> mode for AR ?
>
> The primary AR modality requires registration of the virtual X3D world
> spatially to the video feed (the real world). This registration occurs
> naturally for geospatial X3D scenes. But for regular X3D scenes, an
> additional step is needed. Should this additional registration step
> become part of or facilitated by X3D ? WebXR has anchor objects which
> I think play a role here.
>
> A typical AR scenario is that you put virtual objects (a new sofa) in
> real space (your living room). That requires hit testing of real space
> to find locations. This capability is provided by AR devices. Since
> Touchsensor only works with the virtual scene, and not the real space,
> do we need another sensor ? For X3D, the real world feed would be
> equivalent to the X3D Background. So perhaps a BackgroundTouchSensor ?
> That only would make sense for AR. For desktop or VR such a sensor
> could switch to a "sense anything in the scene" mode, perhaps.
>
> Let's stop here. Can augmented, or mixed reality work with X3D in some
> standard way ?
>
> Cheers, Andreas
>
>
>
> --
> Andreas Plesch
> Waltham, MA 02453
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220511/f60faa38/attachment.html>