xrfragment-haxe/doc/RFC_XR_Fragments.txt

Internet Engineering Task Force                          L.R. van Kammen
Internet-Draft                                         18 September 2023
Intended status: Informational                                          



                              XR Fragments
                   draft-XRFRAGMENTS-leonvankammen-00

Abstract

   This draft is a specification for 4D URLs & navigation, which links
   together space, time & text together, for hypermedia browsers with-
   or without a network-connection.
   The specification promotes spatial addressibility, sharing,
   navigation, query-ing and annotating interactive (text)objects across
   for (XR) Browsers.
   XR Fragments allows us to enrich existing dataformats, by recursive
   use of existing proven technologies like URI Fragments
   (https://en.wikipedia.org/wiki/URI_fragment) and BibTags notation.

   Almost every idea in this document is demonstrated at
   https://xrfragment.org (https://xrfragment.org)

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 21 March 2024.

Copyright Notice

   Copyright (c) 2023 IETF Trust and the persons identified as the
   document authors.  All rights reserved.






van Kammen                Expires 21 March 2024                 [Page 1]

Internet-Draft                XR Fragments                September 2023


   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Core principle  . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Conventions and Definitions . . . . . . . . . . . . . . . . .   4
     3.1.  XR Fragment URI Grammar . . . . . . . . . . . . . . . . .   4
   4.  List of URI Fragments . . . . . . . . . . . . . . . . . . . .   5
   5.  List of metadata for 3D nodes . . . . . . . . . . . . . . . .   5
   6.  Navigating 3D . . . . . . . . . . . . . . . . . . . . . . . .   6
   7.  Embedding 3D content  . . . . . . . . . . . . . . . . . . . .   6
   8.  XR Fragment queries . . . . . . . . . . . . . . . . . . . . .   7
     8.1.  including/excluding . . . . . . . . . . . . . . . . . . .   8
     8.2.  Query Parser  . . . . . . . . . . . . . . . . . . . . . .   9
   9.  Embedding content (src-instancing)  . . . . . . . . . . . . .   9
     9.1.  Referencing content (href portals)  . . . . . . . . . . .  11
     9.2.  Scaling instanced content . . . . . . . . . . . . . . . .  12
   10. Text in XR (tagging,linking to spatial objects) . . . . . . .  13
     10.1.  Default Data URI mimetype  . . . . . . . . . . . . . . .  16
     10.2.  URL and Data URI . . . . . . . . . . . . . . . . . . . .  17
     10.3.  XR Text example parser . . . . . . . . . . . . . . . . .  18
   11. Security Considerations . . . . . . . . . . . . . . . . . . .  20
   12. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  21
   13. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  21
   14. Appendix: Definitions . . . . . . . . . . . . . . . . . . . .  21

1.  Introduction

   How can we add more features to existing text & 3D scenes, without
   introducing new dataformats?
   Historically, there's many attempts to create the ultimate
   markuplanguage or 3D fileformat.
   Their lowest common denominator is: (co)authoring using plain text.
   XR Fragments allows us to enrich/connect existing dataformats, by
   recursive use of existing technologies:

   1.  addressibility and navigation of 3D scenes/objects: URI Fragments
       (https://en.wikipedia.org/wiki/URI_fragment) + src/href spatial
       metadata




van Kammen                Expires 21 March 2024                 [Page 2]

Internet-Draft                XR Fragments                September 2023


   2.  Interlinking text/& 3D by collapsing space into a Word Graph
       (XRWG) (and augmenting text with bibs
       (https://github.com/coderofsalvation/tagbibs) / BibTags
       (https://en.wikipedia.org/wiki/BibTeX) appendices (see visual-
       meta (https://visual-meta.info) e.g.)
   3.  extend the hashtag-to-browser-viewport paradigm beyond 2D
       documents (XR documents)

   |  NOTE: The chapters in this document are ordered from highlevel to
   |  lowlevel (technical) as much as possible

2.  Core principle

   XR Fragments strives to serve (nontechnical/fuzzy) humans first, and
   machine(implementations) later, by ensuring hasslefree text-vs-
   thought feedback loops.
   This also means that the repair-ability of machine-matters should be
   human friendly too (not too complex).
   XR Fragments tries to seek to connect the world of text (semantical
   web / RDF), and the world of pixels.
   Instead of combining them (in a game-editor e.g.), XR Fragments is
   opting for a more integrated path *towards* them, by describing how
   to make browsers *4D URL-ready*:

      +===========+============================+====================+
      | principle | XR 4D URL                  | HTML 2D URL        |
      +===========+============================+====================+
      | the XRWG  | wordgraph (collapses 3D    | Ctrl-F (find)      |
      |           | scene to tags)             |                    |
      +-----------+----------------------------+--------------------+
      | the       | hashtags map to camera/    | hashtags map to    |
      | hashbus   | scene-projections          | document positions |
      +-----------+----------------------------+--------------------+
      | spacetime | positions camera, triggers | jumps/scrolls to   |
      | hashtags  | scene-preset/time          | chapter            |
      +-----------+----------------------------+--------------------+

                                  Table 1

   |  XR Fragments does not look at XR (or the web) thru the lens of
   |  HTML.
   |  But approaches things from a higherlevel browser-perspective:









van Kammen                Expires 21 March 2024                 [Page 3]

Internet-Draft                XR Fragments                September 2023


 +----------------------------------------------------------------------------------------------+
 |                                                                                              |
 |   the soul of any URL:       ://macro        /meso            ?micro      #nano              |
 |                                                                                              |
 |                2D URL:       ://library.com  /document        ?search     #chapter           |
 |                                                                                              |
 |                4D URL:       ://park.com     /4Dscene.fbx --> ?search --> #view ---> hashbus |
 |                                                │                                      |      |
 |                                               XRWG <---------------------<------------+      |
 |                                                │                                      |      |
 |                                                ├─ objects --------------->------------|      |
 |                                                └─ text    --------------->------------+      |
 |                                                                                              |
 |                                                                                              |
 +----------------------------------------------------------------------------------------------+

   Traditional webbrowsers can become 4D document-ready by:

   *  loading 3D assets (gltf/fbx e.g.) natively (not thru HTML).
   *  allowing assets to publish hashtags to themselves (the scene)
      using the hashbus (like hashtags controlling the scrollbar).
   *  collapsing the 3D scene to an wordgraph (for essential navigation
      purposes) controllable thru a hash(tag)bus

   XR Fragments itself is HTML-agnostic, though pseudo-XR Fragment
   browsers *can* be implemented on top of HTML/Javascript.

3.  Conventions and Definitions

   See appendix below in case certain terms are not clear.

3.1.  XR Fragment URI Grammar

   reserved    = gen-delims / sub-delims
   gen-delims  = "#" / "&"
   sub-delims  = "," / "="

   |  Example: ://foo.com/my3d.gltf#pos=1,0,0&prio=-5&t=0,100

     +=============================+=================================+
     | Demo                        | Explanation                     |
     +=============================+=================================+
     | pos=1,2,3                   | vector/coordinate argument e.g. |
     +-----------------------------+---------------------------------+
     | pos=1,2,3&rot=0,90,0&q=.foo | combinators                     |
     +-----------------------------+---------------------------------+

                                  Table 2



van Kammen                Expires 21 March 2024                 [Page 4]

Internet-Draft                XR Fragments                September 2023


   |  this is already implemented in all browsers

4.  List of URI Fragments

   +==========+=========+==============+==============================+
   | fragment | type    | example      | info                         |
   +==========+=========+==============+==============================+
   | #pos     | vector3 | #pos=0.5,0,0 | positions camera (or XR      |
   |          |         |              | floor) to xyz-coord 0.5,0,0, |
   +----------+---------+--------------+------------------------------+
   | #rot     | vector3 | #rot=0,90,0  | rotates camera to xyz-coord  |
   |          |         |              | 0.5,0,0                      |
   +----------+---------+--------------+------------------------------+
   | #t       | vector2 | #t=500,1000  | sets animation-loop range    |
   |          |         |              | between frame 500 and 1000   |
   +----------+---------+--------------+------------------------------+
   | #......  | string  | #.cubes      | predefined views, XRWG       |
   |          |         | #cube        | fragments and ID fragments   |
   +----------+---------+--------------+------------------------------+

                                 Table 3

   |  xyz coordinates are similar to ones found in SVG Media Fragments

5.  List of metadata for 3D nodes

      +======+========+================+============================+
      | key  | type   | example (JSON) | info                       |
      +======+========+================+============================+
      | name | string | "name": "cube" | available in all 3D        |
      |      |        |                | fileformats & scenes       |
      +------+--------+----------------+----------------------------+
      | tag  | string | "tag": "cubes  | available through custom   |
      |      |        | geo"           | property in 3D fileformats |
      +------+--------+----------------+----------------------------+
      | href | string | "href":        | available through custom   |
      |      |        | "b.gltf"       | property in 3D fileformats |
      +------+--------+----------------+----------------------------+
      | src  | string | "src": "#cube" | available through custom   |
      |      |        |                | property in 3D fileformats |
      +------+--------+----------------+----------------------------+

                                  Table 4

   Popular compatible 3D fileformats: .gltf, .obj, .fbx, .usdz, .json
   (THREE.js), .dae and so on.





van Kammen                Expires 21 March 2024                 [Page 5]

Internet-Draft                XR Fragments                September 2023


   |  NOTE: XR Fragments are file-agnostic, which means that the
   |  metadata exist in programmatic 3D scene(nodes) too.

6.  Navigating 3D

   Here's an ascii representation of a 3D scene-graph which contains 3D
   objects &#9723; and their metadata:

  +--------------------------------------------------------+
  |                                                        |
  |  index.gltf                                            |
  |    │                                                   |
  |    ├── ◻ buttonA                                       |
  |    │      └ href: #pos=1,0,1&t=100,200                 |
  |    │                                                   |
  |    └── ◻ buttonB                                       |
  |           └ href: other.fbx                            |   <-- file-agnostic (can be .gltf .obj etc)
  |                                                        |
  +--------------------------------------------------------+

   An XR Fragment-compatible browser viewing this scene, allows the end-
   user to interact with the buttonA and buttonB.
   In case of buttonA the end-user will be teleported to another
   location and time in the *current loaded scene*, but buttonB will
   *replace the current scene* with a new one, like other.fbx.

7.  Embedding 3D content

   Here's an ascii representation of a 3D scene-graph with 3D objects
   &#9723; which embeds remote & local 3D objects &#9723; with/out using
   queries:




















van Kammen                Expires 21 March 2024                 [Page 6]

Internet-Draft                XR Fragments                September 2023


  +--------------------------------------------------------+  +-------------------------+
  |                                                        |  |                         |
  |  index.gltf                                            |  | ocean.com/aquarium.fbx  |
  |    │                                                   |  |   │                     |
  |    ├── ◻ canvas                                        |  |   └── ◻ fishbowl        |
  |    │      └ src: painting.png                          |  |         ├─ ◻ bass       |
  |    │                                                   |  |         └─ ◻ tuna       |
  |    ├── ◻ aquariumcube                                  |  |                         |
  |    │      └ src: ://rescue.com/fish.gltf#bass%20tuna   |  +-------------------------+
  |    │                                                   |
  |    ├── ◻ bedroom                                       |
  |    │      └ src: #canvas                               |
  |    │                                                   |
  |    └── ◻ livingroom                                    |
  |           └ src: #canvas                               |
  |                                                        |
  +--------------------------------------------------------+

   An XR Fragment-compatible browser viewing this scene, lazy-loads and
   projects painting.png onto the (plane) object called canvas (which is
   copy-instanced in the bed and livingroom).
   Also, after lazy-loading ocean.com/aquarium.gltf, only the queried
   objects bass and tuna will be instanced inside aquariumcube.
   Resizing will be happen accordingly to its placeholder object
   aquariumcube, see chapter Scaling.

   |  Instead of cherrypicking objects with #bass&tuna thru src, queries
   |  can be used to import the whole scene (and filter out certain
   |  objects).  See next chapter below.

8.  XR Fragment queries

   Include, exclude, hide/shows objects using space-separated strings:

   +========================+==========================================+
   | example                | outcome                                  |
   +========================+==========================================+
   | #q=-sky                | show everything except object named sky  |
   +------------------------+------------------------------------------+
   | #q=-.language          | hide everything with tag language, but   |
   | .english               | show all tag english objects             |
   +------------------------+------------------------------------------+
   | #q=price:>2            | of all objects with property price, show |
   | price:<5               | only objects with value between 2 and 5  |
   +------------------------+------------------------------------------+

                                  Table 5




van Kammen                Expires 21 March 2024                 [Page 7]

Internet-Draft                XR Fragments                September 2023


   It's simple but powerful syntax which allows <b>css</b>-like tag/id-
   selectors with a searchengine prompt-style feeling:

   1.  queries are a way to traverse a scene, and filter objects based
       on their tag- or property-values.
   2.  words starting with . like .german match tag-metadata of 3D
       objects like "tag":"german"
   3.  words starting with . like .german match tag-metadata of (BibTeX)
       tags in XR Text objects like @german{KarlHeinz, ... e.g.

   |  *For example*: #q=.foo is a shorthand for #q=tag:foo, which will
   |  select objects with custom property tag:foo.  Just a simple
   |  #q=cube will simply select an object named cube.

   *  see an example video here
      (https://coderofsalvation.github.io/xrfragment.media/queries.mp4)

8.1.  including/excluding

      +==========+=================================================+
      | operator | info                                            |
      +==========+=================================================+
      | -        | removes/hides object(s)                         |
      +----------+-------------------------------------------------+
      | :        | indicates an object-embedded custom property    |
      |          | key/value                                       |
      +----------+-------------------------------------------------+
      | .        | alias for "tag" :".foo" equals tag:foo          |
      +----------+-------------------------------------------------+
      | > <      | compare float or int number                     |
      +----------+-------------------------------------------------+
      | /        | reference to root-scene.                        |
      |          | Useful in case of (preventing) showing/hiding   |
      |          | objects in nested scenes (instanced by src) (*) |
      +----------+-------------------------------------------------+

                                 Table 6

   |  * = #q=-/cube hides object cube only in the root-scene (not nested
   |  cube objects)
   |  #q=-cube hides both object cube in the root-scene <b>AND</b>
   |  nested skybox objects |









van Kammen                Expires 21 March 2024                 [Page 8]

Internet-Draft                XR Fragments                September 2023


   &#187; example implementation
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/q.js) &#187; example 3D asset
   (https://github.com/coderofsalvation/xrfragment/blob/main/example/
   assets/query.gltf#L192) &#187; discussion
   (https://github.com/coderofsalvation/xrfragment/issues/3)

8.2.  Query Parser

   Here's how to write a query parser:

   1.   create an associative array/object to store query-arguments as
        objects
   2.   detect object id's & properties foo:1 and foo (reference regex:
        /^.*:[><=!]?/ )
   3.   detect excluders like -foo,-foo:1,-.foo,-/foo (reference regex:
        /^-/ )
   4.   detect root selectors like /foo (reference regex: /^[-]?\// )
   5.   detect tag selectors like .foo (reference regex: /^[-]?tag$/ )
   6.   detect number values like foo:1 (reference regex: /^[0-9\.]+$/ )
   7.   expand aliases like .foo into tag:foo
   8.   for every query token split string on :
   9.   create an empty array rules
   10.  then strip key-operator: convert "-foo" into "foo"
   11.  add operator and value to rule-array
   12.  therefore we we set id to true or false (false=excluder -)
   13.  and we set root to true or false (true=/ root selector is
        present)
   14.  we convert key '/foo' into 'foo'
   15.  finally we add the key/value to the store like store.foo =
        {id:false,root:true} e.g.

   |  An example query-parser (which compiles to many languages) can be
   |  found here
   |  (https://github.com/coderofsalvation/xrfragment/blob/main/src/
   |  xrfragment/Query.hx)

9.  Embedding content (src-instancing)

   src is the 3D version of the <a target="_blank"
   href="https://www.w3.org/html/wiki/Elements/iframe">iframe</a>.
   It instances content (in objects) in the current scene/asset.









van Kammen                Expires 21 March 2024                 [Page 9]

Internet-Draft                XR Fragments                September 2023


    +==========+==================================+===================+
    | fragment | type                             | example value     |
    +==========+==================================+===================+
    | src      | string (uri or [[predefined view | predefined_view]] |
    |          |                                  | or [[query        |
    +----------+----------------------------------+-------------------+

                                  Table 7

   1.  local/remote content is instanced by the src (query) value (and
       attaches it to the placeholder mesh containing the src property)
   2.  <b>local</b> src values (URL *starting* with #, like #cube&foo)
       means *only* the mentioned objectnames will be copied to the
       instanced scene (from the current scene) while preserving their
       names (to support recursive selectors). (example code) (https://g
       ithub.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/
       xrf/src.js)
   3.  <b>local</b> src values indicating a query (#q=), means that all
       included objects (from the current scene) will be copied to the
       instanced scene (before applying the query) while preserving
       their names (to support recursive selectors). (example code) (htt
       ps://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
       three/xrf/src.js)
   4.  the instanced scene (from a src value) should be <b>scaled
       accordingly</b> to its placeholder object or <b>scaled
       relatively</b> based on the scale-property (of a geometry-less
       placeholder, an 'empty'-object in blender e.g.).  For more info
       see Chapter Scaling.
   5.  <b>external</b> src (file) values should be served with
       appropriate mimetype (so the XR Fragment-compatible browser will
       now how to render it).  The bare minimum supported mimetypes are:
   6.  when the placeholder object is a 2D plane, but the mimetype is
       3D, then render the spatial content on that plane via a stencil
       buffer.
   7.  when only one object was cherrypicked (#cube e.g.), set its
       position to 0,0,0

   *  model/gltf+json
   *  image/png
   *  image/jpg
   *  text/plain;charset=utf-8;bib=^@










van Kammen                Expires 21 March 2024                [Page 10]

Internet-Draft                XR Fragments                September 2023


   &#187; example implementation
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/src.js)
   &#187; example 3D asset
   (https://github.com/coderofsalvation/xrfragment/blob/main/example/
   assets/src.gltf#L192)
   &#187; discussion (https://github.com/coderofsalvation/xrfragment/
   issues/4)

9.1.  Referencing content (href portals)

   navigation, portals & mutations

       +==========+==================+============================+
       | fragment | type             | example value              |
       +==========+==================+============================+
       | href     | string (uri or   | #pos=1,1,0                 |
       |          | predefined view) | #pos=1,1,0&rot=90,0,0      |
       |          |                  | ://somefile.gltf#pos=1,1,0 |
       +----------+------------------+----------------------------+

                                 Table 8

   1.  clicking an ''external''- or ''file URI'' fully replaces the
       current scene and assumes pos=0,0,0&rot=0,0,0 by default (unless
       specified)

   2.  relocation/reorientation should happen locally for local URI's
       (#pos=....)

   3.  navigation should not happen ''immediately'' when user is more
       than 2 meter away from the portal/object containing the href (to
       prevent accidental navigation e.g.)

   4.  URL navigation should always be reflected in the client (in case
       of javascript: see [here
       (https://github.com/coderofsalvation/xrfragment/blob/dev/src/3rd/
       js/three/navigator.js) for an example navigator).

   5.  In XR mode, the navigator back/forward-buttons should be always
       visible (using a wearable e.g., see [here
       (https://github.com/coderofsalvation/xrfragment/blob/dev/example/
       aframe/sandbox/index.html#L26-L29) for an example wearable)








van Kammen                Expires 21 March 2024                [Page 11]

Internet-Draft                XR Fragments                September 2023


   6.  in case of navigating to a new [[pos)ition, ''first'' navigate to
       the ''current position'' so that the ''back-button'' of the
       ''browser-history'' always refers to the previous position (see
       [here (https://github.com/coderofsalvation/xrfragment/blob/main/s
       rc/3rd/js/three/xrf/href.js#L97))

   7.  portal-rendering: a 2:1 ratio texture-material indicates an
       equirectangular projection

   &#187; example implementation
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/href.js)
   &#187; example 3D asset
   (https://github.com/coderofsalvation/xrfragment/blob/main/example/
   assets/href.gltf#L192)
   &#187; discussion (https://github.com/coderofsalvation/xrfragment/
   issues/1)

9.2.  Scaling instanced content

   Sometimes embedded properties (like [[href|href]] or [[src|src]])
   instance new objects.
   But what about their scale?
   How does the scale of the object (with the embedded properties)
   impact the scale of the referenced content?

   |  Rule of thumb: visible placeholder objects act as a '3D canvas'
   |  for the referenced scene (a plane acts like a 2D canvas for images
   |  e, a cube as a 3D canvas e.g.).

   1.  <b>IF</b> an embedded property (src e.g.) is set on an non-empty
       placeholder object (geometry of >2 vertices):

   *  calculate the <b>bounding box</b> of the ''placeholder'' object
      (maxsize=1.4 e.g.)
   *  hide the ''placeholder'' object (material e.g.)
   *  instance the src scene as a child of the existing object
   *  calculate the <b>bounding box</b> of the instanced scene, and
      scale it accordingly (to 1.4 e.g.)

   |  REASON: non-empty placeholder object can act as a protective
   |  bounding-box (for remote content of which might grow over time
   |  e.g.)

   2.  ELSE multiply the scale-vector of the instanced scene with the
       scale-vector of the <b>placeholder</b> object.

   |  TODO: needs intermediate visuals to make things more obvious



van Kammen                Expires 21 March 2024                [Page 12]

Internet-Draft                XR Fragments                September 2023


10.  Text in XR (tagging,linking to spatial objects)

   How does XR Fragments interlink text with objects?

   |  The XR Fragments does this by collapsing space into a *Word Graph*
   |  (the *XRWG*), augmented by Bib(s)Tex.

   Instead of just throwing together all kinds media types into one
   experience (games), what about the intrinsic connections between
   them?
   Why is HTML adopted less in games outside the browser?  Through the
   lens of game-making, ideally metadata must come *with* that text, but
   not *obfuscate* the text, or *spawning another request* to fetch it.
   XR Fragments does this by detecting Bib(s)Tex, without introducing a
   new language or fileformat

   |  Why Bib(s)Tex?  Because its seems to be the lowest common
   |  denominator for an human-curated XRWG (extendable by
   |  speech/scanner/writing/typing e.g, see further motivation here
   |  (https://github.com/coderofsalvation/hashtagbibs#bibs--bibtex-
   |  combo-lowest-common-denominator-for-linking-data))

   Hence:

   1.  XR Fragments promotes (de)serializing a scene to the XRWG
   2.  XR Fragments primes the XRWG, by collecting words from the tag
       and name-property of 3D objects.
   3.  XR Fragments primes the XRWG, by collecting words from *optional*
       metadata *at the end of content* of text (see default mimetype &
       Data URI)
   4.  Bib's (https://github.com/coderofsalvation/hashtagbibs) and
       BibTex are first tag citizens for priming the XRWG with words
       (from XR text)
   5.  Like Bibs, XR Fragments generalizes the BibTex author/title-
       semantics (author{title}) into *this* points to *that*
       (this{that})
   6.  The XRWG should be recalculated when textvalues (in src) change
   7.  HTML/RDF/JSON is still great, but is beyond the XRWG-scope (they
       fit better in the application-layer)
   8.  Applications don't have to be able to access the XRWG
       programmatically, as they can easily generate one themselves by
       traversing the scene-nodes.
   9.  The XR Fragment focuses on fast and easy-to-generate end-user
       controllable word graphs (instead of complex implementations that
       try to defeat word ambiguity)

   Example:




van Kammen                Expires 21 March 2024                [Page 13]

Internet-Draft                XR Fragments                September 2023


  http://y.io/z.fbx                                                           | Derived XRWG (shown as BibTex)
  ----------------------------------------------------------------------------+--------------------------------------
                                                                              | @house{castle,
  +-[src: data:.....]----------------------+   +-[3D mesh]-+                  |   url = {https://y.io/z.fbx#castle}
  | Chapter one                            |   |    / \    |                  | }
  |                                        |   |   /   \   |                  | @baroque{castle,
  | John built houses in baroque style.    |   |  /     \  |                  |   url = {https://y.io/z.fbx#castle}
  |                                        |   |  |_____|  |                  | }
  | #john@baroque                          |   +-----│-----+                  | @baroque{john}
  |                                        |         │                        |
  |                                        |         ├─ name: castle          |
  |                                        |         └─ tag: house baroque    |
  +----------------------------------------+                                  |
                                               [3D mesh ]                     |
                                               |    O   ├─ name: john         |
                                               |   /|\  |                     |
                                               |   / \  |                     |
                                               +--------+                     |

   |  the #john@baroque-bib associates both text John and objectname
   |  john, with tag baroque

   Another example:

  http://y.io/z.fbx                                                           | Derived XRWG (printed as BibTex)
  ----------------------------------------------------------------------------+--------------------------------------
                                                                              |
  +-[src: data:.....]----------------------+   +-[3D mesh]-+                  | @house{castle,
  | Chapter one                            |   |    / \    |                  |   url = {https://y.io/z.fbx#castle}
  |                                        |   |   /   \   |                  | }
  | John built houses in baroque style.    |   |  /     \  |                  | @baroque{castle,
  |                                        |   |  |_____|  |                  |   url = {https://y.io/z.fbx#castle}
  | #john@baroque                          |   +-----│-----+                  | }
  | @baroque{john}                         |         │                        | @baroque{john}
  |                                        |         ├─ name: castle          |
  |                                        |         └─ tag: house baroque    |
  +----------------------------------------+                                  | @house{baroque}
                                               [3D mesh ]                     | @todo{baroque}
  +-[remotestorage.io / localstorage]------+   |    O   + name: john          |
  | #baroque@todo@house                    |   |   /|\  |                     |
  | ...                                    |   |   / \  |                     |
  +----------------------------------------+   +--------+                     |

   |  both #john@baroque-bib and BibTex @baroque{john} result in the
   |  same XRWG, however on top of that 2 tages (house and todo) are now
   |  associated with text/objectname/tag 'baroque'.





van Kammen                Expires 21 March 2024                [Page 14]

Internet-Draft                XR Fragments                September 2023


   As seen above, the XRWG can expand bibs
   (https://github.com/coderofsalvation/hashtagbibs) (and the whole
   scene) to BibTeX.
   This allows hasslefree authoring and copy-paste of associations *for
   and by humans*, but also makes these URLs possible:

         +===================+===================================+
         | URL example       | Result                            |
         +===================+===================================+
         | https://my.com/   | highlights mesh john, 3D mesh     |
         | foo.gltf#.baroque | castle, text John built(..)       |
         +-------------------+-----------------------------------+
         | https://my.com/   | highlights mesh john, and the     |
         | foo.gltf#john     | text John built (..)              |
         +-------------------+-----------------------------------+
         | https://my.com/   | highlights mesh castle, and other |
         | foo.gltf#house    | objects with tag house or todo    |
         +-------------------+-----------------------------------+

                                  Table 9

   |  hashtagbibs (https://github.com/coderofsalvation/hashtagbibs)
   |  potentially allow the enduser to annotate text/objects by
   |  *speaking/typing/scanning associations*, which the XR Browser
   |  saves to remotestorage (or localStorage per toplevel URL).  As
   |  well as, referencing BibTags per URI later on: https://y.io/
   |  z.fbx#@baroque@todo e.g.

   The XRWG allows XR Browsers to show/hide relationships in realtime at
   various levels:

   *  wordmatch *inside* src text
   *  wordmatch *inside* href text
   *  wordmatch object-names
   *  wordmatch object-tagnames

   Spatial wires can be rendered, words/objects can be highlighted/
   scaled etc.
   Some pointers for good UX (but not necessary to be XR Fragment
   compatible):

   9.  The XR Browser needs to adjust tag-scope based on the endusers
       needs/focus (infinite tagging only makes sense when environment
       is scaled down significantly)
   10. The XR Browser should always allow the human to view/edit the
       metadata, by clicking 'toggle metadata' on the 'back'
       (contextmenu e.g.) of any XR text, anywhere anytime.




van Kammen                Expires 21 March 2024                [Page 15]

Internet-Draft                XR Fragments                September 2023


   11. respect multi-line BiBTeX metadata in text because of the core
       principle (#core-principle)
   12. Default font (unless specified otherwise) is a modern monospace
       font, for maximized tabular expressiveness (see the core
       principle (#core-principle)).
   13. anti-pattern: hardcoupling an XR Browser with a mandatory
       *markup/scripting-language* which departs from onubtrusive plain
       text (HTML/VRML/Javascript) (see the core principle (#core-
       principle))
   14. anti-pattern: limiting human introspection, by abandoning plain
       text as first tag citizen.

   |  The simplicity of appending metadata (and leveling the metadata-
   |  playfield between humans and machines) is also demonstrated by
   |  visual-meta (https://visual-meta.info) in greater detail.

   Fictional chat:

<John> Hey what about this: https://my.com/station.gltf#pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> I'm checking it right now
<Sarah> I don't see everything..where's our text from yesterday?
<John> Ah wait, that's tagged with tag 'draft' (and hidden)..hold on, try this:
<John> https://my.com/station.gltf#.draft&pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> how about we link the draft to the upcoming YELLO-event?
<John> ok I'm adding #draft@YELLO
<Sarah> Yesterday I also came up with other usefull assocations between other texts in the scene:
#event#YELLO
#2025@YELLO
<John> thanks, added.
<Sarah> Btw. I stumbled upon this spatial book which references station.gltf in some chapters:
<Sarah> https://thecommunity.org/forum/foo/mytrainstory.txt
<John> interesting, I'm importing mytrainstory.txt into station.gltf
<John> ah yes, chapter three points to trainterminal_2A in the scene, cool

10.1.  Default Data URI mimetype

   The src-values work as expected (respecting mime-types), however:

   The XR Fragment specification bumps the traditional default browser-
   mimetype

   text/plain;charset=US-ASCII

   to a hashtagbib(tex)-friendly one:

   text/plain;charset=utf-8;bib=^@

   This indicates that:



van Kammen                Expires 21 March 2024                [Page 16]

Internet-Draft                XR Fragments                September 2023


   *  utf-8 is supported by default
   *  lines beginning with @ will not be rendered verbatim by default
      (read more (https://github.com/coderofsalvation/
      hashtagbibs#hashtagbib-mimetypes))
   *  the XRWG should expand bibs to BibTex occurring in text
      (#contactjohn@todo@important e.g.)

   By doing so, the XR Browser (applications-layer) can interpret
   microformats (visual-meta (https://visual-meta.info) to connect text
   further with its environment ( setup links between textual/spatial
   objects automatically e.g.).

   |  for more info on this mimetype see bibs
   |  (https://github.com/coderofsalvation/hashtagbibs)

   Advantages:

   *  auto-expanding of hashtagbibs
      (https://github.com/coderofsalvation/hashtagbibs) associations
   *  out-of-the-box (de)multiplex human text and metadata in one go
      (see the core principle (#core-principle))
   *  no network-overhead for metadata (see the core principle (#core-
      principle))
   *  ensuring high FPS: HTML/RDF historically is too 'requesty'/'parsy'
      for game studios
   *  rich send/receive/copy-paste everywhere by default, metadata being
      retained (see the core principle (#core-principle))
   *  netto result: less webservices, therefore less servers, and
      overall better FPS in XR

   |  This significantly expands expressiveness and portability of human
   |  tagged text, by *postponing machine-concerns to the end of the
   |  human text* in contrast to literal interweaving of content and
   |  markupsymbols (or extra network requests, webservices e.g.).

   For all other purposes, regular mimetypes can be used (but are not
   required by the spec).

10.2.  URL and Data URI












van Kammen                Expires 21 March 2024                [Page 17]

Internet-Draft                XR Fragments                September 2023


  +--------------------------------------------------------------+  +------------------------+
  |                                                              |  | author.com/article.txt |
  |  index.gltf                                                  |  +------------------------+
  |    │                                                         |  |                        |
  |    ├── ◻ article_canvas                                      |  | Hello friends.         |
  |    │    └ src: ://author.com/article.txt                     |  |                        |
  |    │                                                         |  | @book{greatgatsby      |
  |    └── ◻ note_canvas                                         |  |   ...                  |
  |           └ src:`data:welcome human\n@book{sunday...}`       |  | }                      |
  |                                                              |  +------------------------+
  |                                                              |
  +--------------------------------------------------------------+

   The enduser will only see welcome human and Hello friends rendered
   verbatim (see mimetype).  The beauty is that text in Data URI
   automatically promotes rich copy-paste (retaining metadata).  In both
   cases, the text gets rendered immediately (onto a plane geometry,
   hence the name '_canvas').  The XR Fragment-compatible browser can
   let the enduser access visual-meta(data)-fields after interacting
   with the object (contextmenu e.g.).

   |  additional tagging using bibs
   |  (https://github.com/coderofsalvation/hashtagbibs): to tag spatial
   |  object note_canvas with 'todo', the enduser can type or speak
   |  #note_canvas@todo

10.3.  XR Text example parser

   To prime the XRWG with text from plain text src-values, here's an
   example XR Text (de)multiplexer in javascript (which supports inline
   bibs & bibtex):

xrtext = {

  expandBibs: (text) => {
    let bibs   = { regex: /(#[a-zA-Z0-9_+@\-]+(#)?)/g, tags: {}}
    text.replace( bibs.regex , (m,k,v) => {
       tok   = m.substr(1).split("@")
       match = tok.shift()
       if( tok.length ) tok.map( (t) => bibs.tags[t] = `@${t}{${match},\n}` )
       else if( match.substr(-1) == '#' )
          bibs.tags[match] = `@{${match.replace(/#/,'')}}`
       else bibs.tags[match] = `@${match}{${match},\n}`
    })
    return text.replace( bibs.regex, '') + Object.values(bibs.tags).join('\n')
  },

  decode: (str) => {



van Kammen                Expires 21 March 2024                [Page 18]

Internet-Draft                XR Fragments                September 2023


    // bibtex:     ↓@   ↓<tag|tag{phrase,|{ruler}>  ↓property  ↓end
    let pat    = [ /@/, /^\S+[,{}]/,                /},/,      /}/ ]
    let tags   = [], text='', i=0, prop=''
    let lines  = xrtext.expandBibs(str).replace(/\r?\n/g,'\n').split(/\n/)
    for( let i = 0; i < lines.length && !String(lines[i]).match( /^@/ ); i++ )
        text += lines[i]+'\n'

    bibtex = lines.join('\n').substr( text.length )
    bibtex.split( pat[0] ).map( (t) => {
        try{
           let v = {}
           if( !(t = t.trim())         ) return
           if( tag = t.match( pat[1] ) ) tag = tag[0]
           if( tag.match( /^{.*}$/ )   ) return tags.push({ruler:tag})
           if( tag.match( /}$/ )       ) return tags.push({k: tag.replace(/}$/,''), v: {}})
           t = t.substr( tag.length )
           t.split( pat[2] )
           .map( kv => {
             if( !(kv = kv.trim()) || kv == "}" ) return
             v[ kv.match(/\s?(\S+)\s?=/)[1] ] = kv.substr( kv.indexOf("{")+1 )
           })
           tags.push( { k:tag, v } )
        }catch(e){ console.error(e) }
    })
    return {text, tags}
  },

  encode: (text,tags) => {
    let str = text+"\n"
    for( let i in tags ){
      let item = tags[i]
      if( item.ruler ){
          str += `@${item.ruler}\n`
          continue;
      }
      str += `@${item.k}\n`
      for( let j in item.v ) str += `  ${j} = {${item.v[j]}}\n`
      str += `}\n`
    }
    return str
  }
}

   The above functions (de)multiplexe text/metadata, expands bibs,
   (de)serialize bibtex and vice versa

   |  above can be used as a startingpoint for LLVM's to translate/
   |  steelman to a more formal form/language.



van Kammen                Expires 21 March 2024                [Page 19]

Internet-Draft                XR Fragments                September 2023


str = `
hello world
here are some hashtagbibs followed by bibtex:

#world
#hello@greeting
#another-section#

@{some-section}
@flap{
  asdf = {23423}
}`

var {tags,text} = xrtext.decode(str)          // demultiplex text & bibtex
tags.find( (t) => t.k == 'flap{' ).v.asdf = 1 // edit tag
tags.push({ k:'bar{', v:{abc:123} })          // add tag
console.log( xrtext.encode(text,tags) )       // multiplex text & bibtex back together

   This expands to the following (hidden by default) BibTex appendix:

   hello world
   here are some hashtagbibs followed by bibtex:

   @{some-section}
   @flap{
     asdf = {1}
   }
   @world{world,
   }
   @greeting{hello,
   }
   @{another-section}
   @bar{
     abc = {123}
   }

   |  when an XR browser updates the human text, a quick scan for
   |  nonmatching tags (@book{nonmatchingbook e.g.) should be performed
   |  and prompt the enduser for deleting them.

11.  Security Considerations

   Since XR Text contains metadata too, the user should be able to set
   up tagging-rules, so the copy-paste feature can :

   *  filter out sensitive data when copy/pasting (XR text with
      tag:secret e.g.)




van Kammen                Expires 21 March 2024                [Page 20]

Internet-Draft                XR Fragments                September 2023


12.  IANA Considerations

   This document has no IANA actions.

13.  Acknowledgments

   *  NLNET (https://nlnet.nl)
   *  Future of Text (https://futureoftext.org)
   *  visual-meta.info (https://visual-meta.info)

14.  Appendix: Definitions

     +===============+===============================================+
     | definition    | explanation                                   |
     +===============+===============================================+
     | human         | a sentient being who thinks fuzzy, absorbs,   |
     |               | and shares thought (by plain text, not        |
     |               | markuplanguage)                               |
     +---------------+-----------------------------------------------+
     | scene         | a (local/remote) 3D scene or 3D file          |
     |               | (index.gltf e.g.)                             |
     +---------------+-----------------------------------------------+
     | 3D object     | an object inside a scene characterized by     |
     |               | vertex-, face- and customproperty data.       |
     +---------------+-----------------------------------------------+
     | metadata      | custom properties of text, 3D Scene or        |
     |               | Object(nodes), relevant to machines and a     |
     |               | human minority (academics/developers)         |
     +---------------+-----------------------------------------------+
     | XR fragment   | URI Fragment with spatial hints like          |
     |               | #pos=0,0,0&t=1,100 e.g.                       |
     +---------------+-----------------------------------------------+
     | the XRWG      | wordgraph (collapses 3D scene to tags)        |
     +---------------+-----------------------------------------------+
     | the hashbus   | hashtags map to camera/scene-projections      |
     +---------------+-----------------------------------------------+
     | spacetime     | positions camera, triggers scene-preset/time  |
     | hashtags      |                                               |
     +---------------+-----------------------------------------------+
     | placeholder   | a 3D object which with src-metadata (which    |
     | object        | will be replaced by the src-data.)            |
     +---------------+-----------------------------------------------+
     | src           | (HTML-piggybacked) metadata of a 3D object    |
     |               | which instances content                       |
     +---------------+-----------------------------------------------+
     | href          | (HTML-piggybacked) metadata of a 3D object    |
     |               | which links to content                        |
     +---------------+-----------------------------------------------+



van Kammen                Expires 21 March 2024                [Page 21]

Internet-Draft                XR Fragments                September 2023


     | query         | an URI Fragment-operator which queries        |
     |               | object(s) from a scene like #q=cube           |
     +---------------+-----------------------------------------------+
     | visual-meta   | visual-meta (https://visual.meta.info) data   |
     |               | appended to text/books/papers which is        |
     |               | indirectly visible/editable in XR.            |
     +---------------+-----------------------------------------------+
     | requestless   | metadata which never spawns new requests      |
     | metadata      | (unlike RDF/HTML, which can cause framerate-  |
     |               | dropping, hence not used a lot in games)      |
     +---------------+-----------------------------------------------+
     | FPS           | frames per second in spatial experiences      |
     |               | (games,VR,AR e.g.), should be as high as      |
     |               | possible                                      |
     +---------------+-----------------------------------------------+
     | introspective | inward sensemaking ("I feel this belongs to   |
     |               | that")                                        |
     +---------------+-----------------------------------------------+
     | extrospective | outward sensemaking ("I'm fairly sure John is |
     |               | a person who lives in oklahoma")              |
     +---------------+-----------------------------------------------+
     | &#9723;       | ascii representation of an 3D object/mesh     |
     +---------------+-----------------------------------------------+
     | (un)obtrusive | obtrusive: wrapping human text/thought in     |
     |               | XML/HTML/JSON obfuscates human text into a    |
     |               | salad of machine-symbols and words            |
     +---------------+-----------------------------------------------+
     | BibTeX        | simple tagging/citing/referencing standard    |
     |               | for plaintext                                 |
     +---------------+-----------------------------------------------+
     | BibTag        | a BibTeX tag                                  |
     +---------------+-----------------------------------------------+
     | (hashtag)bibs | an easy to speak/type/scan tagging SDL (see   |
     |               | here (https://github.com/coderofsalvation/    |
     |               | hashtagbibs) which expands to BibTex/JSON/XML |
     +---------------+-----------------------------------------------+

                                  Table 10













van Kammen                Expires 21 March 2024                [Page 22]