[x3d-public] X3D JSON schema, identifying X3D type for items within an array. (was element)

[John Carlson]

Good feedback!  Here's some feedback on the feedback.  Would it satisfy 
you if we put appinfo from X3DUOM FieldType in an items' $comment?  If 
that's okay, you can probably skip most of the rest of this message.  
There is potentially some possibility of parsing ContentModel for 
ordering stuff in "head" and "X3D"--let me know if ContentModel is the 
right place to look for order. I don't think that accessType is doable 
due to potential shared fields (see below).

On 4/6/21 7:07 PM, Don Brutzman wrote:
> thanks for review and progress
>
> On 4/6/2021 4:08 PM, John Carlson wrote:
>>
>>
>> Decision should be made whether array X3D type to array item X3D type
>> map should be in X3D JSON schema generator or X3DUOM.
>
> all types (array or singleton) already defined in X3DUOM so no 
> decision needed.
>
> what question do you have?


For example, take FieldType SFVec3f in X3DUOM.    FieldType SFVec3f says 
it's an array, but does not specify the X3D type of the items in the 
array, except in appinfo (SFFloat), and I don't have a desire to parse 
through that.   Perhaps we could add "itemType" to each FieldType in 
X3DUOM which is an array--or is "itemType" too JSONic?  It's okay with 
me if I put the proposed map in the JSON schema generator, I just figure 
you would complain about hard-coded things that we're trying to make 
explicit in X3DUOM, like the order of objects in head and X3D.

If you've got the order expressed in X3DUOM, then we can do something 
like this in JSON schema: 
https://stackoverflow.com/questions/31691247/how-can-i-define-the-sequence-of-properties-in-json-schema 
I am really surprised at how easy some stuff is in JSON Schema! As for 
order, is that done in ContentModel in X3DUOM?  Just confirming before I 
run off to program.  I am not familiar with how ContentModel works at 
this time.   Do you have documentation on it?


What I feel we're trying to do is add "X3D" information to the X3D JSON 
schema, so it would be good to remain "X3D-like" for item types, to 
maintain historical information, and so we can avoid creating extra 
files from X3DUOM for my JS programs--see fieldTypes.js and 
mapToMethod.js in X3DJSONLD/src/main/node  (the JavaScript remains out 
of date unless a full build is run).   Also it would be good to look at 
mapToMethod2.js as these were the things I found missing from 
mapToMethod.js (but I need to run through that with the newer X3DUOM, 
sorry).

>
>> As discussed in our meeting today, I am adding a $comment with X3D type
>> before "type": occurs in the X3D JSON schema.
>>
>> This will require more work than initially required, although the
>> changes so far have been fairly straightforward in the somewhat messy 
>> code.
>>
>> For example:
>>
>>              "@url": {
>>                "pattern": "^(\\s|\\S)*$",
>> +              "$comment": "X3D type: MFString",
>>                "type": "array",
>>                "minItems": 1,
>>                "items": {
>>                  "format": "uri-reference",
>> +                "$comment": "X3D type: MFString",
>>                  "type": "string"
>>                }
>>              },
>>
>>
>> The X3D type for the items object, should be SFString one would think.
>> So I'll have to check to see if I'm in an array.
>
> Great.  No need to say "X3D type: "

See suggestion on including appinfo in some $comment's.

>
> I would add accessType as well please.
>
> example: "$comment": "MFString inputOutput",

I'm a little bit fuzzy on adding an accessType, because shared field 
definitions may have different accessTypes per node/statement.  If we're 
not sharing field definitions, then this would be OK.

I like to write self-documenting code (note my lack of actual 
documentation except for emails :().  Sometimes I do better than 
others.   I was frankly puzzled when someone commented on the amount of 
documentation in my repositories.  I was pretty shocked, and I wasn't 
sure if they were kidding or not.

Someday, I'm going to write an web crawler that reads all my emails and 
constructs documentation :)  Or I'll just let a startup, OpenAI or 
Google do it.

>
>> Probably by looking for "items".  Then I'll look for MF in the field
>> type, and replace the first M with an S.
>>
>>
>> But here are added problems looking at arrays:
>>
>>
>>              "@bboxSize": {
>>                "pattern":
>> "^\\s*(([+-]?((0|[1-9][0-9]*)(\\.[0-9]*)?|\\.[0-9]+)([Ee][+-]?[0-9]+)?)\\s+){2}([+-]?((0|[1-9][0-9]*)(\\.[0-9]*)?|\\.[0-9]+)([Ee][+-]?[0-9]+)?)\\s*$", 
>>
>> +              "$comment": "X3D type: SFVec3f",
>>                "type": "array",
>>                "minItems": 3,
>>                "maxItems": 3,
>>                "items": {
>>                  "default": -1,
>> +                "$comment": "X3D type: SFVec3f",
>>                  "type": "number"
>>                }
>>              },
>>
>>
>> In this case, the SFVec3f would be found, and a mapping between SFVec3f
>> to SFFloat for individual items should be made.
>
> I think your actual mapping is SFVec3f to Javascript "type": "number" 
> with min/maxItems 3  which appears above.


Yes, we could leave off the $comment for native JavaScript.  I don't 
actually know what the items of a SFVec3f are except that the appinfo in 
X3DUOM says the numbers are SFFloat's.  That's the relationship I'm 
trying to directly put in the X3DUOM so I don't have to build the map 
myself.  I believe it will be more informative to declare the types of 
the items of an array in a $comment in the schema (and in X3DUOM too).   
If we're in JSON, we pretty much know it's a number, since there are no 
quotes around it.

>
>> So perhaps a simple map could be constructed which maps array type to
>> item type.
>>
>>
>> Does this sound pretty feasible, essentially a hard coded map from array
>> type to item type in the generator code?
>
> sounds trivially simple and workable


See above where I attempt a design for putting the map into X3DUOM, or 
just copying appinfo into the $comment value for items.

>
>
>> I will look for additional issues now. I'm not seeing much right away.
>> I know I ignore some of the "type" fields right now, particularly in
>> hard coded definitions.
>>
>> Note that I'm inserting documentation.  This shouldn't affect the
>> running of the generator.
>>
>> My main question is, where does the information reside?  Should it be in
>> X3DUOM?
>
> (on loudspeaker) "I Say Again" all of the needed information is 
> already in X3DUOM.


Yes, but perhaps not easily accessible? I don't want you to have to say 
something twice, and I can certainly attempt to parse appinfo.  I don't 
particularly want you to have to parse appinfo either.  I can egrep 
'FieldType|appinfo' to collect information that is relevant from X3DUOM 
to start--actually gathering the info by hand might be easier and then 
you can verify the information I present and then put it into X3DUOM.  
Then I'll start querying FieldType (or whatever you prefer) in the X3D 
JSON schema generator. I currently only query for regex in FieldType, it 
should be easy to add more features!  Hurrah!

>
>> Does anyone mind if I start calling my generators "synthesizers"?  I'm
>> on a new kick!
>
> yes, please don't kick us with mixed terms...


Okay!

>
>> Note that we are moving towards XSL for our primary JSON schema
>> synthesizer, lessening requirements on future maintainers. We're
>> currently using python to suss out patterns and requirements for
>> X3DUOM.   The python pretty much is impossible to maintain (even the guy
>> that wrote it admits that).  If you know any "younguns" who love XSL,
>> encourage them to join and contribute to X3D!
>
> when you have a good pattern in a good draft-07 schema, i will create 
> an XSLT autogenerator that converts X3DUOM XML to build an X3D JSON 
> schema.


Think about how to add JSON schemas for various profiles. That was one 
thing I hadn't gotten to yet, and I know you added that to X3DUOM a 
while ago.   Sorry for not keeping up-to-date.

>
>> Building holistic schema synthesizers.
>>
>> My next job will be to build holistic spaceship synthesizers.
>
> please leave me behind


:)  maybe we'll take a saliva sample if we need an expert on XSL 
programming.

I think later JSON schema versions are adding stuff like "how to show on 
a GUI" so it would be good to keep a handle on what they are adding to 
the JSON schema when we want to upgrade.  Perhaps we could just put 
appinfo in $comment and call it a day?

It seems like there's still a lot of work surrounding X3D JSON. I enjoy 
that everyone puts up with me, but I don't know of any customers for X3D 
JSON!

Respectfully (with a bit of tongue in check).

John