[x3d-public] xml:lang language identification, X3D metadata patterns for XMP and multilingual support
Brutzman, Donald (Don) (CIV)
brutzman at nps.edu
Thu Nov 25 08:00:05 PST 2021
Thanks for your insights and helpful summary Jakub.
With respect to xml:lang support in X3D XML encoding, am looking at ways we might effectively support it in validation tools and documentation as an allowed XML attribute that is not a preferred approach, emphasizing declarative MetadataString representations instead. Have started variation testing on an example scene.
With regard to mapping XMP structures (essentially embedding Semantic Web RDF declarations), your suggestion to look at all of the defined terms & relationships there for correspondences to Dublin Core and other known terms is a good one. Thanks for sharing the XMP specification reference so everyone can look at it (I believe that XMP was later codified as an ISO standard too). This will also help us in future work when mapping glTF JSON-LD (linked-data) constructs, if those folks are actually following such practices.
With regard to changing X3D4 XML default containerField for Metadata* nodes as ‘value’ for much terser Metadata structures, initial tests look good. Am considering how to best add a corresponding X3D3 XML example that clearly shows necessary compatible verbose forms for X3D versions 3.0 through 3.3. In combination with good diagnostics and documentation, this should give us a solid path forward for backward/forward compatibility without problems.
More on language representation: here is an interesting excerpt from RFC 5646.
* RFC 5646, Tags for Identifying Languages, Best Current Practice BCP 47
* https://datatracker.ietf.org/doc/html/rfc5646
For markup languages, such as HTML and XML, language information
can be added to each part of the document identified by the markup
structure (including the whole document itself). For example, one
could write <span lang="fr">C'est la vie.</span> inside a German
document; the German-speaking user could then access a French-
German dictionary to find out what the marked section meant. If
the user were listening to that document through a speech
synthesis interface, this formation could be used to signal the
synthesizer to appropriately apply French text-to-speech
pronunciation rules to that span of text, instead of applying the
inappropriate German rules.
Am wondering if we can extend this pattern to allow defining compatible multilingual expressions, as often occurs in XML and Semantic Web. Perhaps something like this:
<MetadataSet name='description' reference='https://datatracker.ietf.org/doc/html/rfc5646'>
<MetadataString name='description' value='Hello World' reference='lang=EN'/>
<MetadataString name='description' value='Witaj świecie' reference='lang=PL'/>
<MetadataString name='description' value='Bonjour le monde' reference='lang=FR'/>
<MetadataString name='description' value='Hola Mundo' reference='lang=ES'/>
</MetadataSet>
Nesting the 'lang=??' construct as an overload on the reference field is a bit awkward, and also limits proper use of the reference field (instead of pointing to a semantic reference as intended). Possible simpler alternative follows, which looks appealing and would also facilitate X3D parsing/conversions to various programming languages and file encodings:
<MetadataSet name='description' reference='https://datatracker.ietf.org/doc/html/rfc5646'>
<MetadataString name='description' value='Hello World' lang='EN'/>
<MetadataString name='description' value='Witaj świecie' lang='PL'/>
<MetadataString name='description' value='Bonjour le monde' lang='FR'/>
<MetadataString name='description' value='Hola Mundo' lang='ES'/>
</MetadataSet>
Since multilingual support is already a goal for X3D (but we only support one language at a time), this second approach is worth considering. In addition to usage in metadata structures like XMP, it seems useful across the rest of X3D as well. For example an author might put such a construct within an Anchor or TouchSensor or Text node, and an X3D player might then offer the corresponding description matching user (or HTML page, or Web browser) language preferences.
Note that it would require an additional field for MetadataString nodes. Candidate specification addition follows:
* https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/components/core.html#MetadataString
7.4.6 MetadataString
MetadataString : X3DNode, X3DMetadataObject {
SFNode [in,out] metadata NULL [X3DMetadataObject]
SFString [in,out] name ""
SFString [in,out] reference ""
MFString [in,out] value []
SFString [in,out] lang ""
}
The lang field identifies corresponding human language for the provided value strings in accordance with [RFC5646].
Since this additional lang field is only suggested for X3D4 MetadataString node, not seeing any negative impact on existing X3D content or implementations.
Opinions please?
(Dick, just found a specification erratum: current X3D4 draft refers to [RFC4646] which is now obsoleted by [RFC5656] as above.)
Incidentally there are also some Mantis issues regarding the internal interface hierarchy defined for Metadata nodes defined by X3D4 Architecture. I will look at whether they are resolvable without impacting any content and review findings with Dick. Am expecting to either find a functionally equivalent satisfactory resolution (with zero impact on authors/implementers) or else recommend closing without specification change. As ever, X3D community and working group will be asked to review prior to final issue resolution.
Finally, today in USA is Thanksgiving Day, a time to give thanks with family and count our blessings. All of us building on X3D around the world have a lot to be grateful for. Thanks to everybody helping advance these capabilities for everyone.
* https://en.wikipedia.org/wiki/Thanksgiving
So, steady improvement as we explore… Have fun with X3D! 8)
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics https:// faculty.nps.edu/brutzman
From: Jakub Flotyński <flotynski at kti.ue.poznan.pl>
Sent: Tuesday, November 23, 2021 9:00 AM
To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>; John Carlson <yottzumm at gmail.com>
Cc: vmarchetti at kshell.com; X3D-Public <x3d-public at web3d.org>
Subject: Re: [x3d-public] xml:lang language identification
Hi All,
we had a short discussion with Don about the transformation of different XMP nodes and attributes to RDF. A general idea is to have an overview of the XMP specification available at:
* https://wwwimages2.adobe.com/content/dam/acom/en/devnet/xmp/pdfs/XMP%20SDK%20Release%20cc-2016-08/XMPSpecificationPart1.pdf
and determine groups of XMP nodes and attributes that would require different transformation to RDF nodes and attributes. In such a case, we could have a pretty uniform approach to various metadata entities, while maintaining possibility to extend the transformation with new nodes and attributes.
Best regards
Jakub
W dniu 19.11.2021 o 11:03, Brutzman, Donald (Don) (CIV) pisze:
We had a useful discussion about xml:lang in Thursday weekly CAD DPS call, had similar thinking going on.
I will be confirming with Jakub next week (using our XMP test model) whether <MetatadaString name=”xml:lang” value=”EN”/> is always preferable to attribute xml:lang=’EN’ because it is portable across all varieties of X3D.
Whether or not to include attribute xml:lang=’EN’ in DOCTYPE/Schema: either add it everywhere since it is legal XML, or else not at all so that validation warnings are indeed thrown (since it is likely not the recommended approach).
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu<mailto:brutzman at nps.edu>
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics https:// faculty.nps.edu/brutzman
From: John Carlson <yottzumm at gmail.com><mailto:yottzumm at gmail.com>
Sent: Wednesday, November 17, 2021 5:58 PM
To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu><mailto:brutzman at nps.edu>
Cc: vmarchetti at kshell.com<mailto:vmarchetti at kshell.com>; X3D-Public <x3d-public at web3d.org><mailto:x3d-public at web3d.org>; Jakub Flotyński <flotynski at kti.ue.poznan.pl><mailto:flotynski at kti.ue.poznan.pl>
Subject: Re: [x3d-public] xml:lang language identification
I suggest getting xml:lang into X3DUOM before JSON Schema, but we can do some hard coding into the schema generator if pretesting is desired. An example JSON schema snippet would be required.
Also consider “subclassing” xml:lang for our own purposes, including supporting locales in MFStrings perhaps.
I think full testing of existing JSON schema in python,Java, and JavaScript would be desired before proceeding.
Sent from my iPad
On Nov 17, 2021, at 9:12 AM, Brutzman, Donald (Don) (CIV) <brutzman at nps.edu<mailto:brutzman at nps.edu>> wrote:
Thanks for review comments.
I agree that using MetadataString for xml:lang definitions is a good approach for portable X3D mappings within MetadataSet collections. Indeed that was the original approach we tried in the XMP mapping. Further scrutiny will be needed to see if it can support multiple xml:lang definitions for a given value, that might be a separate design pattern. Jakub and I will continue to look closely at this.
Meanwhile, the xml:lang attribute is allowed by XML. Thus am still looking to add it to X3D XML Schema and DOCTYPE (and possibly X3D JSON schema) as an attribute allowed during XML validation, at least for the handful of X3D use cases where it might make sense. Flagging presence of xml:lang values as erroneous would be a “false negative,” meaning an incorrect error diagnostic.
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu<mailto:brutzman at nps.edu>
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics https:// faculty.nps.edu/brutzman
From: x3d-public <x3d-public-bounces at web3d.org<mailto:x3d-public-bounces at web3d.org>> On Behalf Of John Carlson
Sent: Wednesday, November 17, 2021 6:53 AM
To: vmarchetti at kshell.com<mailto:vmarchetti at kshell.com>
Cc: X3D-Public <x3d-public at web3d.org<mailto:x3d-public at web3d.org>>
Subject: Re: [x3d-public] xml:lang language identification
Thanks, Vince. That’s pretty much what I wanted to say. I don’t know if schemas can apply xml:lang to an attribute in all XML documents guided by the schema. I don’t think that would make sense. I am not arguing a position, because I have not studied.
I’ve had problems with namespaced attribute names in the past. I suggest we go through a thorough test after any updates to X3DUOM.
John
Sent from my iPad
On Nov 16, 2021, at 6:24 PM, vmarchetti at kshell.com<mailto:vmarchetti at kshell.com> wrote:
On Nov 16, 2021, at 6:00 PM, Brutzman, Donald (Don) (CIV) <brutzman at nps.edu<mailto:brutzman at nps.edu>> wrote:
XML allows identification of language via the reserved xml:lang attribute on any attribute.
1. XML Recommendation, fifth edition, W3C, section2.12 Language Identification
2. https://www.w3.org/TR/REC-xml/#sec-lang-tag
and
1. W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes, W3C Recommendation 5 April 2012 section 3.4.3 language
2. https://www.w3.org/TR/xmlschema11-2/#language
These excerpts from the XML documents assert that the xml:lang attribute is not as granular as applying to individual attributes, but applies at
the XML element level, and so to all the attributes and subelements , with some clear rules for overriding the xml:lang attribute in sub-elements.
With that in mind, I think that the function of the xml:lang attribute can, in the X3D abstract specification, be performed by a MetadataString node which can be applied to any X3D node, including
other Metadata nodes. The advantage of this approach would be that the language attribution can be readily transferred between different encodings of the same X3D content.
Today while working on XMP metadata transcription from RDF/XML (or TTL) form into X3D/XML form, Jakub and I found that we needed to utilize xml:lang attribute for lossless two-way conversion.
I can think of the following X3D elements that might need to indicate which identification of human language might be appropriate:
1. Text node (corresponding to ‘string’ field)
2. MetadataString
3. <field> and <fieldValue> declarations, if type is SFString/MFString
Conceivably we might also permit this whenever a description field is present also (Viewpoint, sensors, etc.).
Not looking to change X3D expressive power but think that xml:lang information ought to be permissible in X3D XML encoding since it is an allowed part of XML.
Thus am planning to add xml:lang as indicated above in the X3D XML Schema and X3D XML DOCTYPE to avoid incorrect, misleading validation errors.
Any other considerations appropriate? Will enter Mantis issue when ready to proceed. All comments welcome.
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu<mailto:brutzman at nps.edu>
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics https:// faculty.nps.edu/brutzman<http://faculty.nps.edu/brutzman>
_______________________________________________
x3d-public mailing list
x3d-public at web3d.org<mailto:x3d-public at web3d.org>
http://web3d.org/mailman/listinfo/x3d-public_web3d.org
_______________________________________________
x3d-public mailing list
x3d-public at web3d.org<mailto:x3d-public at web3d.org>
http://web3d.org/mailman/listinfo/x3d-public_web3d.org
--
dr inż. Jakub Flotyński, prof. UEP
Katedra Technologii Informacyjnych
Uniwersytet Ekonomiczny w Poznaniu
Department of Information Technology
Poznań University of Economics and Business
al. Niepodległości 10, 61-875 Poznań
tel./fax: (+48)(61)639-2758
e-mail: flotynski at kti.ue.poznan.pl<mailto:flotynski at kti.ue.poznan.pl>
http://www.kti.ue.poznan.pl/flotynski<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kti.ue.poznan.pl%2Fflotynski&data=04%7C01%7Cbrutzman%40nps.edu%7C8e0e32665c4e44a1803508d9aea2ae25%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637732838019438091%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=32J0hPYXRxHIYdJgyjFOp1wsnvtI%2FxSmhyQc1GVj56E%3D&reserved=0>
Nadawcą korespondencji i jednocześnie administratorem Państwa danych osobowych jest Uniwersytet Ekonomiczny w Poznaniu, al. Niepodległości 10, 61-875 Poznań, z którym można skontaktować się listownie lub poprzez wiadomość e-mail (rodo at ue.poznan.pl<mailto:rodo at ue.poznan.pl>). Państwa dane osobowe przetwarzane będą w celu prowadzenia niniejszej korespondencji i przez czas jej trwania. Przysługuje Państwu prawo dostępu do danych, ich sprostowania, ograniczenia przetwarzania, usunięcia, żądania sprzeciwu wobec przetwarzania oraz wniesienia skargi do Prezesa Urzędu Ochrony Danych Osobowych. Szczegółowe informacje na temat ochrony danych osobowych dostępne są w Polityce prywatności<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fue.poznan.pl%2Fpl%2Fpolityka-prywatnosci%2Cc12767%2F&data=04%7C01%7Cbrutzman%40nps.edu%7C8e0e32665c4e44a1803508d9aea2ae25%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637732838019448022%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Ubr7LyWiRBVPBeT6tsXYJqm%2B5fvidFG37lpEqOJ0Gcc%3D&reserved=0>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20211125/ec800262/attachment-0001.html>
More information about the x3d-public
mailing list