xrfragment/doc/RFC_XR_Fragments.txt

Internet Engineering Task Force                          L.R. van Kammen
Internet-Draft                                           6 December 2023
Intended status: Informational                                          



                              XR Fragments
                   draft-XRFRAGMENTS-leonvankammen-00

Abstract

   This draft is a specification for 4D URLs & hypermediatic
   (https://github.com/coderofsalvation/hypermediatic) navigation, which
   links together space, time & text together, for hypermedia browsers
   with- or without a network-connection.
   The specification promotes spatial addressibility, sharing,
   navigation, filtering and databinding objects for (XR) Browsers.
   XR Fragments allows us to better use existing metadata inside 3D
   scene(files), by connecting it to proven technologies like URI
   Fragments (https://en.wikipedia.org/wiki/URI_fragment).

   Almost every idea in this document is demonstrated at
   https://xrfragment.org (https://xrfragment.org)

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 8 June 2024.

Copyright Notice

   Copyright (c) 2023 IETF Trust and the persons identified as the
   document authors.  All rights reserved.






van Kammen                 Expires 8 June 2024                  [Page 1]

Internet-Draft                XR Fragments                 December 2023


   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.  Code Components
   extracted from this document must include Revised BSD License text as
   described in Section 4.e of the Trust Legal Provisions and are
   provided without warranty as described in the Revised BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Core principle  . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Conventions and Definitions . . . . . . . . . . . . . . . . .   5
     3.1.  XR Fragment URI Grammar . . . . . . . . . . . . . . . . .   5
   4.  List of URI Fragments . . . . . . . . . . . . . . . . . . . .   6
     4.1.  List of metadata for 3D nodes . . . . . . . . . . . . . .   6
     4.2.  vector datatypes  . . . . . . . . . . . . . . . . . . . .   7
     4.3.  Dynamic XR Fragments (+databindings)  . . . . . . . . . .   7
   5.  Spatial Referencing 3D  . . . . . . . . . . . . . . . . . . .   8
   6.  Navigating 3D . . . . . . . . . . . . . . . . . . . . . . . .   9
   7.  Top-level URL processing  . . . . . . . . . . . . . . . . . .  10
   8.  Embedding XR content using src  . . . . . . . . . . . . . . .  11
   9.  Navigating content href portals . . . . . . . . . . . . . . .  13
     9.1.  Walking surfaces  . . . . . . . . . . . . . . . . . . . .  15
     9.2.  UX spec . . . . . . . . . . . . . . . . . . . . . . . . .  15
     9.3.  Scaling instanced content . . . . . . . . . . . . . . . .  15
   10. XR Fragment: pos  . . . . . . . . . . . . . . . . . . . . . .  16
   11. XR Fragment: rot  . . . . . . . . . . . . . . . . . . . . . .  16
   12. XR Fragment: t  . . . . . . . . . . . . . . . . . . . . . . .  16
   13. XR audio/video integration  . . . . . . . . . . . . . . . . .  17
   14. XR Fragment filters . . . . . . . . . . . . . . . . . . . . .  17
     14.1.  including/excluding  . . . . . . . . . . . . . . . . . .  18
     14.2.  Filter Parser  . . . . . . . . . . . . . . . . . . . . .  18
   15. Visible links . . . . . . . . . . . . . . . . . . . . . . . .  19
   16. Text in XR (tagging,linking to spatial objects) . . . . . . .  19
     16.1.  Default Data URI mimetype  . . . . . . . . . . . . . . .  23
     16.2.  URL and Data URI . . . . . . . . . . . . . . . . . . . .  24
     16.3.  XR Text example parser . . . . . . . . . . . . . . . . .  25
   17. Transclusion (broken link) resolution . . . . . . . . . . . .  27
   18. Topic-based index-less Webrings . . . . . . . . . . . . . . .  28
   19. Security Considerations . . . . . . . . . . . . . . . . . . .  29
   20. FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  29
   21. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  30
   22. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  30
   23. Appendix: Definitions . . . . . . . . . . . . . . . . . . . .  30





van Kammen                 Expires 8 June 2024                  [Page 2]

Internet-Draft                XR Fragments                 December 2023


1.  Introduction

   How can we add more control to existing text & 3D scenes, without
   introducing new dataformats?
   Historically, there's many attempts to create the ultimate
   markuplanguage or 3D fileformat.
   The lowest common denominator is: designers describing/tagging/naming
   things using *plain text*.
   XR Fragments exploits the fact that all 3D models already contain
   such metadata:

   *XR Fragments allows controlling of metadata in 3D scene(files) using
   URLs*

   Or more detailed:

   1.  addressibility and hypermediatic
       (https://github.com/coderofsalvation/hypermediatic) navigation of
       3D scenes/objects: URI Fragments (https://en.wikipedia.org/wiki/
       URI_fragment) + src/href spatial metadata
   2.  Interlinking (text)objects by collapsing space into a Word Graph
       (XRWG) to show visible links (#visible-links)
   3.  unlocking spatial potential of the (originally 2D) hashtag (which
       jumps to a chapter) for navigating XR documents

   |  NOTE: The chapters in this document are ordered from highlevel to
   |  lowlevel (technical) as much as possible

2.  Core principle

   *XR Fragments allows controlling of metadata in 3D scene(files) using
   URLs*

   XR Fragments tries to seek to connect the world of text (semantical
   web / RDF), and the world of pixels.
   Instead of combining them (in a game-editor e.g.), XR Fragments
   *integrates all*, by collecting metadata into an XRWG and control it
   via URL:













van Kammen                 Expires 8 June 2024                  [Page 3]

Internet-Draft                XR Fragments                 December 2023


    +===========+===============================+====================+
    | principle | XR 4D URL                     | HTML 2D URL        |
    +===========+===============================+====================+
    | the XRWG  | wordgraph (collapses 3D scene | Ctrl-F (find)      |
    |           | to tags)                      |                    |
    +-----------+-------------------------------+--------------------+
    | the       | hashtags alter camera/scene/  | hashtags alter     |
    | hashbus   | object-projections            | document positions |
    +-----------+-------------------------------+--------------------+
    | src       | renders content and offers    | renders content    |
    | metadata  | sourceportation               |                    |
    +-----------+-------------------------------+--------------------+
    | href      | teleports to other XR         | jumps to other     |
    | metadata  | document                      | HTML document      |
    +-----------+-------------------------------+--------------------+
    | href      | triggers predefined view      | Media fragments    |
    | metadata  |                               |                    |
    +-----------+-------------------------------+--------------------+
    | href      | triggers camera/scene/object/ | n/a                |
    | metadata  | projections                   |                    |
    +-----------+-------------------------------+--------------------+
    | href      | draws visible connection(s)   | n/a                |
    | metadata  | for XRWG 'tag'                |                    |
    +-----------+-------------------------------+--------------------+
    | href      | filters certain (in)visible   | n/a                |
    | metadata  | objects                       |                    |
    +-----------+-------------------------------+--------------------+

                                 Table 1

   |  XR Fragments does not look at XR (or the web) thru the lens of
   |  HTML.
   |  But approaches things from a higherlevel feedbackloop/hypermedia
   |  browser-perspective:

















van Kammen                 Expires 8 June 2024                  [Page 4]

Internet-Draft                XR Fragments                 December 2023


 +──────────────────────────────────────────────────────────────────────────────────────────────+
 │                                                                                              │
 │   the soul of any URL:       ://macro        /meso            ?micro      #nano              │
 │                                                                                              │
 │                2D URL:       ://library.com  /document        ?search     #chapter           │
 │                                                                                              │
 │                4D URL:       ://park.com     /4Dscene.fbx ──> ?misc  ──>  #view ───> hashbus │
 │                                                │                          #filter     │      │
 │                                                │                          #tag        │      │
 │                                                │                          #material   │      │
 │                                                │                          #animation  │      │
 │                                                │                          #texture    │      │
 │                                                │                          #variable   │      │
 │                                                │                                      │      │
 │                                               XRWG <─────────────────────<────────────+      │
 │                                                │                                      │      │
 │                                                └─ objects  ──────────────>────────────+      │
 │                                                                                              │
 │                                                                                              │
 +──────────────────────────────────────────────────────────────────────────────────────────────+

   Traditional webbrowsers can become 4D document-ready by:

   *  hypermediatic (https://github.com/coderofsalvation/hypermediatic)
      loading 3D assets (gltf/fbx e.g.) natively (with or without using
      HTML).
   *  allowing assets to publish hashtags to themselves (the scene)
      using the hashbus (like hashtags controlling the scrollbar).
   *  collapsing the 3D scene to an wordgraph (for essential navigation
      purposes) controllable thru a hash(tag)bus

   XR Fragments itself are hypermediatic
   (https://github.com/coderofsalvation/hypermediatic) and HTML-
   agnostic, though pseudo-XR Fragment browsers *can* be implemented on
   top of HTML/Javascript.

3.  Conventions and Definitions

   See appendix below in case certain terms are not clear.

3.1.  XR Fragment URI Grammar

   reserved    = gen-delims / sub-delims
   gen-delims  = "#" / "&"
   sub-delims  = "," / "="

   |  Example: ://foo.com/my3d.gltf#pos=1,0,0&prio=-5&t=0,100




van Kammen                 Expires 8 June 2024                  [Page 5]

Internet-Draft                XR Fragments                 December 2023


      +==========================+=================================+
      | Demo                     | Explanation                     |
      +==========================+=================================+
      | pos=1,2,3                | vector/coordinate argument e.g. |
      +--------------------------+---------------------------------+
      | pos=1,2,3&rot=0,90,0&foo | combinators                     |
      +--------------------------+---------------------------------+

                                 Table 2

   |  this is already implemented in all browsers

4.  List of URI Fragments

   +==========+============+==============+===========================+
   | fragment | type       | example      | info                      |
   +==========+============+==============+===========================+
   | #pos     | vector3    | #pos=0.5,0,0 | positions camera (or XR   |
   |          |            |              | floor) to xyz-coord       |
   |          |            |              | 0.5,0,0,                  |
   +----------+------------+--------------+---------------------------+
   | #rot     | vector3    | #rot=0,90,0  | rotates camera to xyz-    |
   |          |            |              | coord 0.5,0,0             |
   +----------+------------+--------------+---------------------------+
   | #t       | timevector | #t=2,2000,1  | play animation-loop range |
   |          |            |              | between frame 2 and 2000  |
   |          |            |              | at (normal) speed 1       |
   +----------+------------+--------------+---------------------------+

                                 Table 3

4.1.  List of metadata for 3D nodes

   +======+========+==========+===================+===================+
   | key  | type   | example  | function          | existing          |
   |      |        | (JSON)   |                   | compatibility     |
   +======+========+==========+===================+===================+
   | href | string | "href":  | XR teleport       | custom property   |
   |      |        | "b.gltf" |                   | in 3D fileformats |
   +------+--------+----------+-------------------+-------------------+
   | src  | string | "src":   | XR embed /        | custom property   |
   |      |        | "#cube"  | teleport          | in 3D fileformats |
   +------+--------+----------+-------------------+-------------------+
   | tag  | string | "tag":   | tag object (for   | custom property   |
   |      |        | "cubes   | filter-use / XRWG | in 3D fileformats |
   |      |        | geo"     | highlighting)     |                   |
   +------+--------+----------+-------------------+-------------------+




van Kammen                 Expires 8 June 2024                  [Page 6]

Internet-Draft                XR Fragments                 December 2023


                                 Table 4

   |  Supported popular compatible 3D fileformats: .gltf, .obj, .fbx,
   |  .usdz, .json (THREE.js), .dae and so on.

4.2.  vector datatypes

   | type | syntax | example | info |

                              +------------+
                              | vector2    |
                              +------------+
                              | vector3    |
                              +------------+
                              | timevector |
                              +------------+
                              +------------+
                              +------------+
                              +------------+
                              +------------+
                              +------------+
                              +------------+
                              +------------+
                              +------------+
                              +------------+

                                 Table 5

   |  NOTE: XR Fragments are optional but also file- and protocol-
   |  agnostic, which means that programmatic 3D scene(nodes) can also
   |  use the mechanism/metadata.

4.3.  Dynamic XR Fragments (+databindings)

   These are automatic fragment-to-metadata mappings, which only trigger
   if the 3D scene metadata matches a specific identifier (aliasname
   e.g.)














van Kammen                 Expires 8 June 2024                  [Page 7]

Internet-Draft                XR Fragments                 December 2023


   +==========================+=======================+=================+=======================+
   |fragment                  |type                   |example          |info                   |
   +==========================+=======================+=================+=======================+
   |#<aliasname>              |string                 |#cubes           |evaluate predefined    |
   |                          |                       |                 |views (#cubes: #foo&bar|
   |                          |                       |                 |e.g.)                  |
   +--------------------------+-----------------------+-----------------+-----------------------+
   |#<tag_or_objectname>      |string                 |#person          |focus object(s) with   |
   |                          |                       |                 |tag: person or name    |
   |                          |                       |                 |person by looking up   |
   |                          |                       |                 |XRWG                   |
   +--------------------------+-----------------------+-----------------+-----------------------+
   |#<cameraname>             |string                 |#cam01           |set camera as active   |
   |                          |                       |                 |camera                 |
   +--------------------------+-----------------------+-----------------+-----------------------+
   |#<objectname>=<material>  |string=string          |#car=metallic    |set material of car to |
   |                          |                       |                 |material with name     |
   |                          |                       |                 |metallic               |
   +--------------------------+-----------------------+-----------------+-----------------------+
   |                          |string=string          |#product=metallic|set material of objects|
   |                          |                       |                 |tagged with product to |
   |                          |                       |                 |material with name     |
   |                          |                       |                 |metallic               |
   +--------------------------+-----------------------+-----------------+-----------------------+
   |#<objectname>=<mediafrag> |string=media frag      |#foo=0,1         |play media src using   |
   |                          |(https://www.w3.org/TR/|                 |media fragment URI     |
   |                          |media-frags/#valid-uri)|                 |(https://www.w3.org/TR/|
   |                          |                       |                 |media-frags/#valid-uri)|
   +--------------------------+-----------------------+-----------------+-----------------------+
   |#<objectname>=<timevector>|string=timevector      |#sky=0,0.5,0.1,0 |sets 1D/2D/3D          |
   |                          |                       |                 |time(line) vectors (uv-|
   |                          |                       |                 |position e.g.) to 0,0.5|
   |                          |                       |                 |(and autoscroll x with |
   |                          |                       |                 |max 0.1 every second)  |
   +--------------------------+-----------------------+-----------------+-----------------------+
   |                          |                       |#music=1,2       |play media of object   |
   |                          |                       |                 |(src: podcast.mp3 e.g.)|
   |                          |                       |                 |from beginning (1) at  |
   |                          |                       |                 |double speed (2)       |
   +--------------------------+-----------------------+-----------------+-----------------------+

                                  Table 6

5.  Spatial Referencing 3D

   XR Fragments assume the following objectname-to-URIFragment mapping:





van Kammen                 Expires 8 June 2024                  [Page 8]

Internet-Draft                XR Fragments                 December 2023


  my.io/scene.fbx
  +─────────────────────────────+
  │ sky                         │  src: http://my.io/scene.fbx#sky          (includes building,mainobject,floor)
  │ +─────────────────────────+ │
  │ │ building                │ │  src: http://my.io/scene.fbx#building     (includes mainobject,floor)
  │ │ +─────────────────────+ │ │
  │ │ │ mainobject          │ │ │  src: http://my.io/scene.fbx#mainobject   (includes floor)
  │ │ │ +─────────────────+ │ │ │
  │ │ │ │ floor           │ │ │ │  src: http://my.io/scene.fbx#floor        (just floor object)
  │ │ │ │                 │ │ │ │
  │ │ │ +─────────────────+ │ │ │
  │ │ +─────────────────────+ │ │
  │ +─────────────────────────+ │
  +─────────────────────────────+

   |  Every 3D fileformat supports named 3D object, and this name allows
   |  URLs (fragments) to reference them (and their children objects).

   Clever nested design of 3D scenes allow great ways for re-using
   content, and/or previewing scenes.
   For example, to render a portal with a preview-version of the scene,
   create an 3D object with:

   *  href: https://scene.fbx
   *  src: https://otherworld.gltf#mainobject

   |  It also allows *sourceportation*, which basically means the
   |  enduser can teleport to the original XR Document of an src
   |  embedded object, and see a visible connection to the particular
   |  embedded object.  Basically an embedded link becoming an outbound
   |  link by activating it.

6.  Navigating 3D

      +====================+=========+=============================+
      | fragment           | type    | functionality               |
      +====================+=========+=============================+
      | <b>#pos</b>=0,0,0  | vector3 | (re)position camera based   |
      |                    | or      | on coordinates directly, or |
      |                    | string  | indirectly using objectname |
      |                    |         | (its worldposition)         |
      +--------------------+---------+-----------------------------+
      | <b>#t</b>=0,100    | vector3 | set playback speed, and     |
      |                    |         | (re)position looprange of   |
      |                    |         | scene-animation or src-     |
      |                    |         | mediacontent                |
      +--------------------+---------+-----------------------------+
      | <b>#rot</b>=0,90,0 | vector3 | rotate camera               |



van Kammen                 Expires 8 June 2024                  [Page 9]

Internet-Draft                XR Fragments                 December 2023


      +--------------------+---------+-----------------------------+

                                 Table 7

   » example implementation
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/pos.js)
   » discussion (https://github.com/coderofsalvation/xrfragment/
   issues/5)

   1.  the Y-coordinate of pos identifies the floorposition.  This means
       that desktop-projections usually need to add 1.5m (average person
       height) on top (which is done automatically by VR/AR headsets).
   2.  set the position of the camera accordingly to the vector3 values
       of #pos
   3.  rot sets the rotation of the camera (only for non-VR/AR headsets)
   4.  t sets the playbackspeed and animation-range of the current scene
       animation(s) or src-mediacontent (video/audioframes e.g., use
       t=0,7,7 to 'STOP' at frame 7 e.g.)
   5.  after scene load: in case an href does not mention any pos-
       coordinate, pos=0,0,0 will be assumed

   Here's an ascii representation of a 3D scene-graph which contains 3D
   objects ◻ and their metadata:

  +────────────────────────────────────────────────────────+
  │                                                        │
  │  index.gltf                                            │
  │    │                                                   │
  │    ├── ◻ buttonA                                       │
  │    │      └ href: #pos=1,0,1&t=100,200                 │
  │    │                                                   │
  │    └── ◻ buttonB                                       │
  │           └ href: other.fbx                            │   <── file─agnostic (can be .gltf .obj etc)
  │                                                        │
  +────────────────────────────────────────────────────────+

   An XR Fragment-compatible browser viewing this scene, allows the end-
   user to interact with the buttonA and buttonB.
   In case of buttonA the end-user will be teleported to another
   location and time in the *current loaded scene*, but buttonB will
   *replace the current scene* with a new one, like other.fbx, and
   assume pos=0,0,0.

7.  Top-level URL processing

   |  Example URL: ://foo/world.gltf#cube&pos=0,0,0




van Kammen                 Expires 8 June 2024                 [Page 10]

Internet-Draft                XR Fragments                 December 2023


   The URL-processing-flow for hypermedia browsers goes like this:

   1.  IF a #cube matches a custom property-key (of an object) in the 3D
       file/scene (#cube: #......) <b>THEN</b> execute that
       predefined_view.
   2.  IF scene operators (pos) and/or animation operator (t) are
       present in the URL then (re)position the camera and/or animation-
       range accordingly.
   3.  IF no camera-position has been set in <b>step 1 or 2</b> update
       the top-level URL with #pos=0,0,0 (example (https://github.com/co
       derofsalvation/xrfragment/blob/main/src/3rd/js/three/
       navigator.js#L31]]))
   4.  IF a #cube matches the name (of an object) in the 3D file/scene
       then draw a line from the enduser('s heart) to that object (to
       highlight it).
   5.  IF a #cube matches anything else in the XR Word Graph (XRWG) draw
       wires to them (text or related objects).

8.  Embedding XR content using src

   src is the 3D version of the <a target="_blank"
   href="https://www.w3.org/html/wiki/Elements/iframe">iframe</a>.
   It instances content (in objects) in the current scene/asset.

   +========+========+===================================================+
   |fragment|type    |example value                                      |
   +========+========+===================================================+
   |src     |string  |#cube                                              |
   |        |(uri,   |#sometag                                           |
   |        |hashtag/|#cube&-ball_inside_cube<br>#-sky&-rain<br>#-       |
   |        |filter) |language&english<br>#price=>5<br>https://linux.org/|
   |        |        |penguin.png` (https://linux.org/penguin.png`)      |
   |        |        |https://linux.world/distrowatch.gltf#t=1,100       |
   |        |        |linuxapp://conference/nixworkshop/apply.gltf#-     |
   |        |        |cta&cta_apply                                      |
   |        |        |androidapp://page1?tutorial#pos=0,0,1&t1,100       |
   |        |        |foo.mp3#0,0,0                                      |
   +--------+--------+---------------------------------------------------+

                                  Table 8

   Here's an ascii representation of a 3D scene-graph with 3D objects ◻
   which embeds remote & local 3D objects ◻ with/out using filters:








van Kammen                 Expires 8 June 2024                 [Page 11]

Internet-Draft                XR Fragments                 December 2023


  +────────────────────────────────────────────────────────+  +─────────────────────────+
  │                                                        │  │                         │
  │  index.gltf                                            │  │ ocean.com/aquarium.fbx  │
  │    │                                                   │  │   ├ room                │
  │    ├── ◻ canvas                                        │  │   └── ◻ fishbowl        │
  │    │      └ src: painting.png                          │  │         ├─ ◻ bass       │
  │    │                                                   │  │         └─ ◻ tuna       │
  │    ├── ◻ aquariumcube                                  │  │                         │
  │    │      └ src: ://rescue.com/fish.gltf#fishbowl      │  +─────────────────────────+
  │    │                                                   │
  │    ├── ◻ bedroom                                       │
  │    │      └ src: #canvas                               │
  │    │                                                   │
  │    └── ◻ livingroom                                    │
  │           └ src: #canvas                               │
  │                                                        │
  +────────────────────────────────────────────────────────+

   An XR Fragment-compatible browser viewing this scene, lazy-loads and
   projects painting.png onto the (plane) object called canvas (which is
   copy-instanced in the bed and livingroom).
   Also, after lazy-loading ocean.com/aquarium.gltf, only the queried
   objects fishbowl (and bass and tuna) will be instanced inside
   aquariumcube.
   Resizing will be happen accordingly to its placeholder object
   aquariumcube, see chapter Scaling.

   |  Instead of cherrypicking a rootobject #fishbowl with src,
   |  additional filters can be used to include/exclude certain objects.
   |  See next chapter on filtering below.

   *Specification*:

   1.   local/remote content is instanced by the src (filter) value (and
        attaches it to the placeholder mesh containing the src property)
   2.   by default all objects are loaded into the instanced src (scene)
        object (but not shown yet)
   3.   <b>local</b> src values (#... e.g.) starting with a non-negating
        filter (#cube e.g.) will (deep)reparent that object (with name
        cube) as the new root of the scene at position 0,0,0
   4.   <b>local</b> src values should respect (negative) filters
        (#-foo&price=>3)
   5.   the instanced scene (from a src value) should be <b>scaled
        accordingly</b> to its placeholder object or <b>scaled
        relatively</b> based on the scale-property (of a geometry-less
        placeholder, an 'empty'-object in blender e.g.).  For more info
        see Chapter Scaling.




van Kammen                 Expires 8 June 2024                 [Page 12]

Internet-Draft                XR Fragments                 December 2023


   6.   <b>external</b> src values should be served with appropriate
        mimetype (so the XR Fragment-compatible browser will now how to
        render it).  The bare minimum supported mimetypes are:
   7.   src values should make its placeholder object invisible, and
        only flush its children when the resolved content can
        succesfully be retrieved (see broken links (#links))
   8.   <b>external</b> src values should respect the fallback link
        mechanism (see broken links (#broken-links)
   9.   when the placeholder object is a 2D plane, but the mimetype is
        3D, then render the spatial content on that plane via a stencil
        buffer.
   10.  src-values are non-recursive: when linking to an external object
        (src: foo.fbx#bar), then src-metadata on object bar should be
        ignored.
   11.  an external src-value should always allow a sourceportation icon
        within 3 meter: teleporting to the origin URI to which the
        object belongs.
   12.  when only one object was cherrypicked (#cube e.g.), set its
        position to 0,0,0
   13.  when the enduser clicks an href with #t=1,0,0 (play) will be
        applied to all src mediacontent with a timeline (mp4/mp3 e.g.)
   14.  a non-euclidian portal can be rendered for flat 3D objects
        (using stencil buffer e.g.) in case ofspatial src-values (an
        object #world3 or URL world3.fbx e.g.).

   *  model/gltf-binary
   *  model/gltf+json
   *  image/png
   *  image/jpg
   *  text/plain;charset=utf-8

   » example implementation
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/src.js)
   » example 3D asset
   (https://github.com/coderofsalvation/xrfragment/blob/main/example/
   assets/src.gltf#L192)
   » discussion (https://github.com/coderofsalvation/xrfragment/
   issues/4)

9.  Navigating content href portals

   navigation, portals & mutations








van Kammen                 Expires 8 June 2024                 [Page 13]

Internet-Draft                XR Fragments                 December 2023


       +==========+==================+============================+
       | fragment | type             | example value              |
       +==========+==================+============================+
       | href     | string (uri or   | #pos=1,1,0                 |
       |          | predefined view) | #pos=1,1,0&rot=90,0,0      |
       |          |                  | ://somefile.gltf#pos=1,1,0 |
       +----------+------------------+----------------------------+

                                 Table 9

   1.  clicking an outbound ''external''- or ''file URI'' fully replaces
       the current scene and assumes pos=0,0,0&rot=0,0,0 by default
       (unless specified)

   2.  relocation/reorientation should happen locally for local URI's
       (#pos=....)

   3.  navigation should not happen ''immediately'' when user is more
       than 2 meter away from the portal/object containing the href (to
       prevent accidental navigation e.g.)

   4.  URL navigation should always be reflected in the client (in case
       of javascript: see [here
       (https://github.com/coderofsalvation/xrfragment/blob/dev/src/3rd/
       js/three/navigator.js) for an example navigator).

   5.  In XR mode, the navigator back/forward-buttons should be always
       visible (using a wearable e.g., see [here
       (https://github.com/coderofsalvation/xrfragment/blob/dev/example/
       aframe/sandbox/index.html#L26-L29) for an example wearable)

   6.  in case of navigating to a new [[pos)ition, ''first'' navigate to
       the ''current position'' so that the ''back-button'' of the
       ''browser-history'' always refers to the previous position (see
       [here (https://github.com/coderofsalvation/xrfragment/blob/main/s
       rc/3rd/js/three/xrf/href.js#L97))

   7.  ignore previous rule in special cases, like clicking an href
       using camera-portal collision (the back-button would cause a
       teleport-loop)











van Kammen                 Expires 8 June 2024                 [Page 14]

Internet-Draft                XR Fragments                 December 2023


   » example implementation
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/href.js)
   » example 3D asset
   (https://github.com/coderofsalvation/xrfragment/blob/main/example/
   assets/href.gltf#L192)
   » discussion (https://github.com/coderofsalvation/xrfragment/
   issues/1)

9.1.  Walking surfaces

   XR Fragment-compatible viewers can infer this data based scanning the
   scene for:

   1.  materialless (nameless & textureless) mesh-objects (without src
       and href)

   |  optionally the viewer can offer thumbstick, mouse or joystick
   |  teleport-tools for non-roomscale VR/AR setups.

9.2.  UX spec

   End-users should always have read/write access to:

   1.  the current (toplevel) <b>URL</b> (an URLbar etc)
   2.  URL-history (a <b>back/forward</b> button e.g.)
   3.  Clicking/Touching an href navigates (and updates the URL) to
       another scene/file (and coordinate e.g. in case the URL contains
       XR Fragments).

9.3.  Scaling instanced content

   Sometimes embedded properties (like src) instance new objects.
   But what about their scale?
   How does the scale of the object (with the embedded properties)
   impact the scale of the referenced content?

   |  Rule of thumb: visible placeholder objects act as a '3D canvas'
   |  for the referenced scene (a plane acts like a 2D canvas for images
   |  e, a cube as a 3D canvas e.g.).

   1.  <b>IF</b> an embedded property (src e.g.) is set on an non-empty
       placeholder object (geometry of >2 vertices):

   *  calculate the <b>bounding box</b> of the ''placeholder'' object
      (maxsize=1.4 e.g.)
   *  hide the ''placeholder'' object (material e.g.)
   *  instance the src scene as a child of the existing object



van Kammen                 Expires 8 June 2024                 [Page 15]

Internet-Draft                XR Fragments                 December 2023


   *  calculate the <b>bounding box</b> of the instanced scene, and
      scale it accordingly (to 1.4 e.g.)

   |  REASON: non-empty placeholder object can act as a protective
   |  bounding-box (for remote content of which might grow over time
   |  e.g.)

   2.  ELSE multiply the scale-vector of the instanced scene with the
       scale-vector (a common property of a 3D node) of the
       <b>placeholder</b> object.

   |  TODO: needs intermediate visuals to make things more obvious

10.  XR Fragment: pos

11.  XR Fragment: rot

12.  XR Fragment: t

   controls the animation(s) of the scene (or src resource which
   contains a timeline)

   | fragment | type | functionality | | <b>#t</b>=1,1,100 |
   [vector3|vector] (default:`#t=1,0,0`) | speed,framestart,framestop |

   *  playposition is reset to framestart, when framestart or framestop
      is greater than 0 |

   | Example Value | Explanation | |-|-| | 1,1,100 | play loop between
   frame 1 and 100 | | 1,1,0 | play once from frame 1 (oneshot) | |
   1,0,0 | play (previously set looprange if any) | | 0,0,0 | pause | |
   1,1,1 | play and auto-loop between begin and end of duration | |
   -1,0,0 | reverse playback speed | | 2.3,0,0 | set (forward) playback
   speed to 2.3 (no restart) | | -2.3,0,0 | set (reverse) playback speed
   to -2.3 ( no restart)| | -2.3,100,0 | set (reverse) playback speed to
   -2.3 restarting from frame 100 |

   [[» example implementation|https://github.com/coderofsalvation/xrfrag
   ment/blob/main/src/3rd/js/three/xrf/t.js]
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/t.js])]
   [[» discussion|https://github.com/coderofsalvation/xrfragment/
   issues/10] (https://github.com/coderofsalvation/xrfragment/
   issues/10])]







van Kammen                 Expires 8 June 2024                 [Page 16]

Internet-Draft                XR Fragments                 December 2023


13.  XR audio/video integration

   To play global audio/video items:

   1.  add a src: foo.mp3 or src: bar.mp4 metadata to a 3D object (cube
       e.g.)
   2.  to disable auto-play and global timeline ([[#t=|t]]) control:
       hardcode a [[#t=|t]] XR Fragment: (src: bar.mp3#t=0,0,0 e.g.)
   3.  to play it, add href: #cube somewhere else
   4.  when the enduser clicks the href, #t=1,0,0 (play) will be applied
       to the src value
   5.  to play a single animation, add href: #animationname=1,0,0
       somewhere else

   |  NOTE: hardcoded framestart/framestop uses sampleRate/fps of
   |  embedded audio/video, otherwise the global fps applies.  For more
   |  info see [[#t|t]].

14.  XR Fragment filters

   Include, exclude, hide/shows objects using space-separated strings:

    +====================+===========================================+
    | example            | outcome                                   |
    +====================+===========================================+
    | #-sky              | show everything except object named sky   |
    +--------------------+-------------------------------------------+
    | #-language&english | hide everything with tag language, but    |
    |                    | show all tag english objects              |
    +--------------------+-------------------------------------------+
    | #-price&price=>10  | hide all objects with property price,     |
    |                    | then only show object with price above 10 |
    +--------------------+-------------------------------------------+

                                 Table 10

   It's simple but powerful syntax which allows filtering the scene
   using searchengine prompt-style feeling:

   1.  filters are a way to traverse a scene, and filter objects based
       on their name, tag- or property-values.

   *  see an (outdated) example video here
      (https://coderofsalvation.github.io/xrfragment.media/queries.mp4)
      which used a dedicated q= variable (now deprecated and usable
      directly)





van Kammen                 Expires 8 June 2024                 [Page 17]

Internet-Draft                XR Fragments                 December 2023


14.1.  including/excluding

   By default, selectors work like photoshop-layers: they scan for
   matching layer(name/properties) within the scene-graph.  Each matched
   object (not their children) will be toggled (in)visible when
   selecting.

        +==========+==============================================+
        | operator | info                                         |
        +==========+==============================================+
        | -        | hides object(s) (#-myobject&-objects e.g.    |
        +----------+----------------------------------------------+
        | =        | indicates an object-embedded custom property |
        |          | key/value (#price=4&category=foo e.g.)       |
        +----------+----------------------------------------------+
        | => =<    | compare float or int number (#price=>4 e.g.) |
        +----------+----------------------------------------------+
        | *        | deepselect: automatically select children of |
        |          | selected object, including local (nonremote) |
        |          | embedded objects (starting with #)           |
        +----------+----------------------------------------------+

                                  Table 11

   |  NOTE 1: after an external embedded object has been instanced (src:
   |  https://y.com/bar.fbx#room e.g.), filters do not affect them
   |  anymore (reason: local tag/name collisions can be mitigated
   |  easily, but not in case of remote content).
   |  
   |  NOTE 2: depending on the used 3D framework, toggling objects
   |  (in)visible should happen by enabling/disableing writing to the
   |  colorbuffer (to allow children being still visible while their
   |  parents are invisible).

   » example implementation
   (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
   three/xrf/q.js) » example 3D asset
   (https://github.com/coderofsalvation/xrfragment/blob/main/example/
   assets/filter.gltf#L192) » discussion
   (https://github.com/coderofsalvation/xrfragment/issues/3)

14.2.  Filter Parser

   Here's how to write a filter parser:

   1.   create an associative array/object to store filter-arguments as
        objects




van Kammen                 Expires 8 June 2024                 [Page 18]

Internet-Draft                XR Fragments                 December 2023


   2.   detect object id's & properties foo=1 and foo (reference regex=
        ~/^.*=[><=]?/ )
   3.   detect excluders like -foo,-foo=1,-.foo,-/foo (reference regex=
        /^-/ )
   4.   detect root selectors like /foo (reference regex= /^[-]?\// )
   5.   detect number values like foo=1 (reference regex= /^[0-9\.]+$/ )
   6.   detect operators so you can easily strip keys (reference regex=
        /(^-|\*$)/ )
   7.   detect exclude keys like -foo (reference regex= /^-/ )
   8.   for every filter token split string on =
   9.   and we set root to true or false (true=/ root selector is
        present)
   10.  therefore we we set show to true or false (false=excluder -)

   |  An example filter-parser (which compiles to many languages) can be
   |  found here
   |  (https://github.com/coderofsalvation/xrfragment/blob/main/src/
   |  xrfragment/Filter.hx)

15.  Visible links

   When predefined views, XRWG fragments and ID fragments (#cube or
   #mytag e.g.) are triggered by the enduser (via toplevel URL or
   clicking href):

   1.  draw a wire from the enduser (preferabbly a bit below the camera,
       heartposition) to object(s) matching that ID (objectname)
   2.  draw a wire from the enduser (preferabbly a bit below the camera,
       heartposition) to object(s) matching that tag value
   3.  draw a wire from the enduser (preferabbly a bit below the camera,
       heartposition) to object(s) containing that in their src or href
       value

   The obvious approach for this, is to consult the XRWG (example
   (https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
   src/3rd/js/XRWG.js)), which basically has all these things already
   collected/organized for you during scene-load.

   *UX*

   4.  do not update the wires when the enduser moves, leave them as is
   5.  offer a control near the back/forward button which allows the
       user to (turn off) control the correlation-intensity of the XRWG

16.  Text in XR (tagging,linking to spatial objects)

   How does XR Fragments interlink text with objects?




van Kammen                 Expires 8 June 2024                 [Page 19]

Internet-Draft                XR Fragments                 December 2023


   |  The XR Fragments does this by collapsing space into a *Word Graph*
   |  (the *XRWG* example
   |  (https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
   |  src/3rd/js/XRWG.js)), augmented by Bib(s)Tex.

   Instead of just throwing together all kinds media types into one
   experience (games), what about their tagged/semantical relationships?
   Perhaps the following question is related: why is HTML adopted less
   in games outside the browser?  Through the lens of constructive lazy
   game-developers, ideally metadata must come *with* text, but not
   *obfuscate* the text, or *spawning another request* to fetch it.
   XR Fragments does this by detecting Bib(s)Tex, without introducing a
   new language or fileformat

   |  Why Bib(s)Tex?  Because its seems to be the lowest common
   |  denominator for an human-curated XRWG (extendable by
   |  speech/scanner/writing/typing e.g, see further motivation here
   |  (https://github.com/coderofsalvation/hashtagbibs#bibs--bibtex-
   |  combo-lowest-common-denominator-for-linking-data))

   Hence:

   1.   XR Fragments promotes (de)serializing a scene to the XRWG
        (example (https://github.com/coderofsalvation/xrfragment/blob/fe
        at/macros/src/3rd/js/XRWG.js))
   2.   XR Fragments primes the XRWG, by collecting words from the tag
        and name-property of 3D objects.
   3.   XR Fragments primes the XRWG, by collecting words from
        *optional* metadata *at the end of content* of text (see default
        mimetype & Data URI)
   4.   Bib's (https://github.com/coderofsalvation/hashtagbibs) and
        BibTex are first tag citizens for priming the XRWG with words
        (from XR text)
   5.   Like Bibs, XR Fragments generalizes the BibTex author/title-
        semantics (author{title}) into *this* points to *that*
        (this{that})
   6.   The XRWG should be recalculated when textvalues (in src) change
   7.   HTML/RDF/JSON is still great, but is beyond the XRWG-scope (they
        fit better in the application-layer)
   8.   Applications don't have to be able to access the XRWG
        programmatically, as they can easily generate one themselves by
        traversing the scene-nodes.
   9.   The XR Fragment focuses on fast and easy-to-generate end-user
        controllable word graphs (instead of complex implementations
        that try to defeat word ambiguity)
   10.  Tags are the scope for now (supporting https://github.com/WICG/
        scroll-to-text-fragment (https://github.com/WICG/scroll-to-text-
        fragment) will be considered)



van Kammen                 Expires 8 June 2024                 [Page 20]

Internet-Draft                XR Fragments                 December 2023


   Example:

  http://y.io/z.fbx                                                           | Derived XRWG (expressed as BibTex)
  ----------------------------------------------------------------------------+--------------------------------------
                                                                              | @house{castle,
  +-[src: data:.....]----------------------+   +-[3D mesh]-+                  |   url = {https://y.io/z.fbx#castle}
  | Chapter one                            |   |    / \    |                  | }
  |                                        |   |   /   \   |                  | @baroque{castle,
  | John built houses in baroque style.    |   |  /     \  |                  |   url = {https://y.io/z.fbx#castle}
  |                                        |   |  |_____|  |                  | }
  | #john@baroque                          |   +-----│-----+                  | @baroque{john}
  |                                        |         │                        |
  |                                        |         ├─ name: castle          |
  |                                        |         └─ tag: house baroque    |
  +----------------------------------------+                                  |
                                               [3D mesh ]                     |
                                               |    O   ├─ name: john         |
                                               |   /|\  |                     |
                                               |   / \  |                     |
                                               +--------+                     |

   |  the #john@baroque-bib associates both text John and objectname
   |  john, with tag baroque

   Another example:

  http://y.io/z.fbx                                                           | Derived XRWG (expressed as BibTex)
  ----------------------------------------------------------------------------+--------------------------------------
                                                                              |
  +-[src: data:.....]----------------------+   +-[3D mesh]-+                  | @house{castle,
  | Chapter one                            |   |    / \    |                  |   url = {https://y.io/z.fbx#castle}
  |                                        |   |   /   \   |                  | }
  | John built houses in baroque style.    |   |  /     \  |                  | @baroque{castle,
  |                                        |   |  |_____|  |                  |   url = {https://y.io/z.fbx#castle}
  | #john@baroque                          |   +-----│-----+                  | }
  | @baroque{john}                         |         │                        | @baroque{john}
  |                                        |         ├─ name: castle          |
  |                                        |         └─ tag: house baroque    |
  +----------------------------------------+                                  | @house{baroque}
                                               [3D mesh ]                     | @todo{baroque}
  +-[remotestorage.io / localstorage]------+   |    O   + name: john          |
  | #baroque@todo@house                    |   |   /|\  |                     |
  | ...                                    |   |   / \  |                     |
  +----------------------------------------+   +--------+                     |

   |  both #john@baroque-bib and BibTex @baroque{john} result in the
   |  same XRWG, however on top of that 2 tages (house and todo) are now
   |  associated with text/objectname/tag 'baroque'.



van Kammen                 Expires 8 June 2024                 [Page 21]

Internet-Draft                XR Fragments                 December 2023


   As seen above, the XRWG can expand bibs
   (https://github.com/coderofsalvation/hashtagbibs) (and the whole
   scene) to BibTeX.
   This allows hasslefree authoring and copy-paste of associations *for
   and by humans*, but also makes these URLs possible:

        +==================+======================================+
        | URL example      | Result                               |
        +==================+======================================+
        | https://my.com/  | draws lines between mesh john, 3D    |
        | foo.gltf#baroque | mesh castle, text John built(..)     |
        +------------------+--------------------------------------+
        | https://my.com/  | draws lines between mesh john, and   |
        | foo.gltf#john    | the text John built (..)             |
        +------------------+--------------------------------------+
        | https://my.com/  | draws lines between mesh castle, and |
        | foo.gltf#house   | other objects with tag house or todo |
        +------------------+--------------------------------------+

                                  Table 12

   |  hashtagbibs (https://github.com/coderofsalvation/hashtagbibs)
   |  potentially allow the enduser to annotate text/objects by
   |  *speaking/typing/scanning associations*, which the XR Browser
   |  saves to remotestorage (or localStorage per toplevel URL).  As
   |  well as, referencing BibTags per URI later on: https://y.io/
   |  z.fbx#@baroque@todo e.g.

   The XRWG allows XR Browsers to show/hide relationships in realtime at
   various levels:

   *  wordmatch *inside* src text
   *  wordmatch *inside* href text
   *  wordmatch object-names
   *  wordmatch object-tagnames

   Spatial wires can be rendered between words/objects etc.
   Some pointers for good UX (but not necessary to be XR Fragment
   compatible):

   9.  The XR Browser needs to adjust tag-scope based on the endusers
       needs/focus (infinite tagging only makes sense when environment
       is scaled down significantly)
   10. The XR Browser should always allow the human to view/edit the
       metadata, by clicking 'toggle metadata' on the 'back'
       (contextmenu e.g.) of any XR text, anywhere anytime.
   11. respect multi-line BiBTeX metadata in text because of the core
       principle (#core-principle)



van Kammen                 Expires 8 June 2024                 [Page 22]

Internet-Draft                XR Fragments                 December 2023


   12. Default font (unless specified otherwise) is a modern monospace
       font, for maximized tabular expressiveness (see the core
       principle (#core-principle)).
   13. anti-pattern: hardcoupling an XR Browser with a mandatory
       *markup/scripting-language* which departs from onubtrusive plain
       text (HTML/VRML/Javascript) (see the core principle (#core-
       principle))
   14. anti-pattern: limiting human introspection, by abandoning plain
       text as first tag citizen.

   |  The simplicity of appending metadata (and leveling the metadata-
   |  playfield between humans and machines) is also demonstrated by
   |  visual-meta (https://visual-meta.info) in greater detail.

   Fictional chat:

<John> Hey what about this: https://my.com/station.gltf#pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> I'm checking it right now
<Sarah> I don't see everything..where's our text from yesterday?
<John> Ah wait, that's tagged with tag 'draft' (and hidden)..hold on, try this:
<John> https://my.com/station.gltf#.draft&pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> how about we link the draft to the upcoming YELLO-event?
<John> ok I'm adding #draft@YELLO
<Sarah> Yesterday I also came up with other usefull assocations between other texts in the scene:
#event#YELLO
#2025@YELLO
<John> thanks, added.
<Sarah> Btw. I stumbled upon this spatial book which references station.gltf in some chapters:
<Sarah> https://thecommunity.org/forum/foo/mytrainstory.txt
<John> interesting, I'm importing mytrainstory.txt into station.gltf
<John> ah yes, chapter three points to trainterminal_2A in the scene, cool

16.1.  Default Data URI mimetype

   The src-values work as expected (respecting mime-types), however:

   The XR Fragment specification bumps the traditional default browser-
   mimetype

   text/plain;charset=US-ASCII

   to a hashtagbib(tex)-friendly one:

   text/plain;charset=utf-8;bib=^@

   This indicates that:

   *  utf-8 is supported by default



van Kammen                 Expires 8 June 2024                 [Page 23]

Internet-Draft                XR Fragments                 December 2023


   *  lines beginning with @ will not be rendered verbatim by default
      (read more (https://github.com/coderofsalvation/
      hashtagbibs#hashtagbib-mimetypes))
   *  the XRWG should expand bibs to BibTex occurring in text
      (#contactjohn@todo@important e.g.)

   By doing so, the XR Browser (applications-layer) can interpret
   microformats (visual-meta (https://visual-meta.info) to connect text
   further with its environment ( setup links between textual/spatial
   objects automatically e.g.).

   |  for more info on this mimetype see bibs
   |  (https://github.com/coderofsalvation/hashtagbibs)

   Advantages:

   *  auto-expanding of hashtagbibs
      (https://github.com/coderofsalvation/hashtagbibs) associations
   *  out-of-the-box (de)multiplex human text and metadata in one go
      (see the core principle (#core-principle))
   *  no network-overhead for metadata (see the core principle (#core-
      principle))
   *  ensuring high FPS: HTML/RDF historically is too 'requesty'/'parsy'
      for game studios
   *  rich send/receive/copy-paste everywhere by default, metadata being
      retained (see the core principle (#core-principle))
   *  netto result: less webservices, therefore less servers, and
      overall better FPS in XR

   |  This significantly expands expressiveness and portability of human
   |  tagged text, by *postponing machine-concerns to the end of the
   |  human text* in contrast to literal interweaving of content and
   |  markupsymbols (or extra network requests, webservices e.g.).

   For all other purposes, regular mimetypes can be used (but are not
   required by the spec).

16.2.  URL and Data URI













van Kammen                 Expires 8 June 2024                 [Page 24]

Internet-Draft                XR Fragments                 December 2023


  +--------------------------------------------------------------+  +------------------------+
  |                                                              |  | author.com/article.txt |
  |  index.gltf                                                  |  +------------------------+
  |    │                                                         |  |                        |
  |    ├── ◻ article_canvas                                      |  | Hello friends.         |
  |    │    └ src: ://author.com/article.txt                     |  |                        |
  |    │                                                         |  | @book{greatgatsby      |
  |    └── ◻ note_canvas                                         |  |   ...                  |
  |           └ src:`data:welcome human\n@book{sunday...}`       |  | }                      |
  |                                                              |  +------------------------+
  |                                                              |
  +--------------------------------------------------------------+

   The enduser will only see welcome human and Hello friends rendered
   verbatim (see mimetype).  The beauty is that text in Data URI
   automatically promotes rich copy-paste (retaining metadata).  In both
   cases, the text gets rendered immediately (onto a plane geometry,
   hence the name '_canvas').  The XR Fragment-compatible browser can
   let the enduser access visual-meta(data)-fields after interacting
   with the object (contextmenu e.g.).

   |  additional tagging using bibs
   |  (https://github.com/coderofsalvation/hashtagbibs): to tag spatial
   |  object note_canvas with 'todo', the enduser can type or speak
   |  #note_canvas@todo

16.3.  XR Text example parser

   To prime the XRWG with text from plain text src-values, here's an
   example XR Text (de)multiplexer in javascript (which supports inline
   bibs & bibtex):

xrtext = {

  expandBibs: (text) => {
    let bibs   = { regex: /(#[a-zA-Z0-9_+@\-]+(#)?)/g, tags: {}}
    text.replace( bibs.regex , (m,k,v) => {
       tok   = m.substr(1).split("@")
       match = tok.shift()
       if( tok.length ) tok.map( (t) => bibs.tags[t] = `@${t}{${match},\n}` )
       else if( match.substr(-1) == '#' )
          bibs.tags[match] = `@{${match.replace(/#/,'')}}`
       else bibs.tags[match] = `@${match}{${match},\n}`
    })
    return text.replace( bibs.regex, '') + Object.values(bibs.tags).join('\n')
  },

  decode: (str) => {



van Kammen                 Expires 8 June 2024                 [Page 25]

Internet-Draft                XR Fragments                 December 2023


    // bibtex:     ↓@   ↓<tag|tag{phrase,|{ruler}>  ↓property  ↓end
    let pat    = [ /@/, /^\S+[,{}]/,                /},/,      /}/ ]
    let tags   = [], text='', i=0, prop=''
    let lines  = xrtext.expandBibs(str).replace(/\r?\n/g,'\n').split(/\n/)
    for( let i = 0; i < lines.length && !String(lines[i]).match( /^@/ ); i++ )
        text += lines[i]+'\n'

    bibtex = lines.join('\n').substr( text.length )
    bibtex.split( pat[0] ).map( (t) => {
        try{
           let v = {}
           if( !(t = t.trim())         ) return
           if( tag = t.match( pat[1] ) ) tag = tag[0]
           if( tag.match( /^{.*}$/ )   ) return tags.push({ruler:tag})
           if( tag.match( /}$/ )       ) return tags.push({k: tag.replace(/}$/,''), v: {}})
           t = t.substr( tag.length )
           t.split( pat[2] )
           .map( kv => {
             if( !(kv = kv.trim()) || kv == "}" ) return
             v[ kv.match(/\s?(\S+)\s?=/)[1] ] = kv.substr( kv.indexOf("{")+1 )
           })
           tags.push( { k:tag, v } )
        }catch(e){ console.error(e) }
    })
    return {text, tags}
  },

  encode: (text,tags) => {
    let str = text+"\n"
    for( let i in tags ){
      let item = tags[i]
      if( item.ruler ){
          str += `@${item.ruler}\n`
          continue;
      }
      str += `@${item.k}\n`
      for( let j in item.v ) str += `  ${j} = {${item.v[j]}}\n`
      str += `}\n`
    }
    return str
  }
}

   The above functions (de)multiplexe text/metadata, expands bibs,
   (de)serialize bibtex and vice versa

   |  above can be used as a startingpoint for LLVM's to translate/
   |  steelman to a more formal form/language.



van Kammen                 Expires 8 June 2024                 [Page 26]

Internet-Draft                XR Fragments                 December 2023


str = `
hello world
here are some hashtagbibs followed by bibtex:

#world
#hello@greeting
#another-section#

@{some-section}
@flap{
  asdf = {23423}
}`

var {tags,text} = xrtext.decode(str)          // demultiplex text & bibtex
tags.find( (t) => t.k == 'flap{' ).v.asdf = 1 // edit tag
tags.push({ k:'bar{', v:{abc:123} })          // add tag
console.log( xrtext.encode(text,tags) )       // multiplex text & bibtex back together

   This expands to the following (hidden by default) BibTex appendix:

   hello world
   here are some hashtagbibs followed by bibtex:

   @{some-section}
   @flap{
     asdf = {1}
   }
   @world{world,
   }
   @greeting{hello,
   }
   @{another-section}
   @bar{
     abc = {123}
   }

   |  when an XR browser updates the human text, a quick scan for
   |  nonmatching tags (@book{nonmatchingbook e.g.) should be performed
   |  and prompt the enduser for deleting them.

17.  Transclusion (broken link) resolution

   In spirit of Ted Nelson's 'transclusion resolution', there's a soft-
   mechanism to harden links & minimize broken links in various ways:

   1.  defining a different transport protocol (https vs ipfs or DAT) in
       src or href values can make a difference




van Kammen                 Expires 8 June 2024                 [Page 27]

Internet-Draft                XR Fragments                 December 2023


   2.  mirroring files on another protocol using (HTTP) errorcode tags
       in src or href properties
   3.  in case of src: nesting a copy of the embedded object in the
       placeholder object (embeddedObject) will not be replaced when the
       request fails

   |  due to the popularity, maturity and extensiveness of HTTP codes
   |  for client/server communication, non-HTTP protocols easily map to
   |  HTTP codes (ipfs ERR_NOT_FOUND maps to 404 e.g.)

   For example:

  +────────────────────────────────────────────────────────+
  │                                                        │
  │  index.gltf                                            │
  │    │                                                   │
  │    │ #: #-offlinetext                                │
  │    │                                                   │
  │    ├── ◻ buttonA                                       │
  │    │      └ href:     http://foo.io/campagne.fbx       │
  │    │      └ href@404: ipfs://foo.io/campagne.fbx       │
  │    │      └ href@400: #clienterrortext               │
  │    │      └ ◻ offlinetext                              │
  │    │                                                   │
  │    └── ◻ embeddedObject                          <--------- the meshdata inside embeddedObject will (not)
  │           └ src: https://foo.io/bar.gltf               │    be flushed when the request (does not) succeed.
  │           └ src@404: http://foo.io/bar.gltf            │    So worstcase the 3D data (of the time of publishing index.gltf)
  │           └ src@400: https://archive.org/l2kj43.gltf   │    will be displayed.
  │                                                        │
  +────────────────────────────────────────────────────────+

18.  Topic-based index-less Webrings

   As hashtags in URLs map to the XWRG, href-values can be used to
   promote topic-based index-less webrings.
   Consider 3D scenes linking to eachother using these href values:

   *  href: schoolA.edu/projects.gltf#math
   *  href: schoolB.edu/projects.gltf#math
   *  href: university.edu/projects.gltf#math

   These links would all show visible links to math-tagged objects in
   the scene.
   To filter out non-related objects one could take it a step further
   using filters:

   *  href: schoolA.edu/projects.gltf#math&-topics math
   *  href: schoolB.edu/projects.gltf#math&-courses math



van Kammen                 Expires 8 June 2024                 [Page 28]

Internet-Draft                XR Fragments                 December 2023


   *  href: university.edu/projects.gltf#math&-theme math

   |  This would hide all object tagged with topic, courses or theme
   |  (including math) so that later only objects tagged with math will
   |  be visible

   This makes spatial content multi-purpose, without the need to
   separate content into separate files, or show/hide things using a
   complex logiclayer like javascript.

19.  Security Considerations

   Since XR Text contains metadata too, the user should be able to set
   up tagging-rules, so the copy-paste feature can :

   *  filter out sensitive data when copy/pasting (XR text with
      tag:secret e.g.)

20.  FAQ

   *Q:* Why is everything HTTP GET-based, what about POST/PUT/DELETE
   HATEOS
   *A:* Because it's out of scope: XR Fragment specifies a read-only way
   to surf XR documents.  These things belong in the application layer
   (for example, an XR Hypermedia browser can decide to support
   POST/PUT/DELETE requests for embedded HTML thru src values)

   *Q:* Why isn't there support for scripting, while we have things like
   WASM *A:* This is out of scope as it unhyperifies hypermedia, and
   this is up to XR hypermedia browser-extensions.
   Historically scripting/Javascript seems to been able to turn webpages
   from hypermedia documents into its opposite (hyperscripted
   nonhypermedia documents).
   In order to prevent this backward-movement (hypermedia tends to
   liberate people from finnicky scripting) XR Fragments should never
   unhyperify itself by hardcoupling to a particular markup or scripting
   language.  XR Macro's (https://xrfragment.org/doc/RFC_XR_Macros.html)
   are an example of something which is probably smarter and safer for
   hypermedia browsers to implement, instead of going full-in with a
   turing-complete scripting language (and suffer the security
   consequences later).
   XR Fragments supports filtering objects in a scene only, because in
   the history of the javascript-powered web, showing/hiding document-
   entities seems to be one of the most popular basic usecases.
   Doing advanced scripting & networkrequests under the hood are
   obviously interesting endavours, but this is something which should
   not be hardcoupled with hypermedia.
   This belongs to browser extensions.



van Kammen                 Expires 8 June 2024                 [Page 29]

Internet-Draft                XR Fragments                 December 2023


   Non-HTML Hypermedia browsers should make browser extensions the right
   place, to 'extend' experiences, in contrast to code/javascript inside
   hypermedia documents (this turned out as a hypermedia antipattern).

21.  IANA Considerations

   This document has no IANA actions.

22.  Acknowledgments

   *  NLNET (https://nlnet.nl)
   *  Future of Text (https://futureoftext.org)
   *  visual-meta.info (https://visual-meta.info)
   *  Michiel Leenaars
   *  Gerben van der Broeke
   *  Mauve
   *  Jens Finkhäuser
   *  Marc Belmont
   *  Tim Gerritsen
   *  Frode Hegland
   *  Brandel Zackernuk
   *  Mark Anderson

23.  Appendix: Definitions

    +=================+===============================================+
    | definition      | explanation                                   |
    +=================+===============================================+
    | human           | a sentient being who thinks fuzzy, absorbs,   |
    |                 | and shares thought (by plain text, not        |
    |                 | markuplanguage)                               |
    +-----------------+-----------------------------------------------+
    | scene           | a (local/remote) 3D scene or 3D file          |
    |                 | (index.gltf e.g.)                             |
    +-----------------+-----------------------------------------------+
    | 3D object       | an object inside a scene characterized by     |
    |                 | vertex-, face- and customproperty data.       |
    +-----------------+-----------------------------------------------+
    | metadata        | custom properties of text, 3D Scene or        |
    |                 | Object(nodes), relevant to machines and a     |
    |                 | human minority (academics/developers)         |
    +-----------------+-----------------------------------------------+
    | XR fragment     | URI Fragment with spatial hints like          |
    |                 | #pos=0,0,0&t=1,100 e.g.                       |
    +-----------------+-----------------------------------------------+
    | the XRWG        | wordgraph (collapses 3D scene to tags)        |
    +-----------------+-----------------------------------------------+
    | the hashbus     | hashtags map to camera/scene-projections      |



van Kammen                 Expires 8 June 2024                 [Page 30]

Internet-Draft                XR Fragments                 December 2023


    +-----------------+-----------------------------------------------+
    | spacetime       | positions camera, triggers scene-preset/time  |
    | hashtags        |                                               |
    +-----------------+-----------------------------------------------+
    | teleportation   | repositioning the enduser to a different      |
    |                 | position (or 3D scene/file)                   |
    +-----------------+-----------------------------------------------+
    | sourceportation | teleporting the enduser to the original XR    |
    |                 | Document of an src embedded object.           |
    +-----------------+-----------------------------------------------+
    | placeholder     | a 3D object which with src-metadata (which    |
    | object          | will be replaced by the src-data.)            |
    +-----------------+-----------------------------------------------+
    | src             | (HTML-piggybacked) metadata of a 3D object    |
    |                 | which instances content                       |
    +-----------------+-----------------------------------------------+
    | href            | (HTML-piggybacked) metadata of a 3D object    |
    |                 | which links to content                        |
    +-----------------+-----------------------------------------------+
    | filter          | URI Fragment(s) which show/hide object(s) in  |
    |                 | a scene based on name/tag/property (#cube&-   |
    |                 | price=>3)                                     |
    +-----------------+-----------------------------------------------+
    | visual-meta     | visual-meta (https://visual.meta.info) data   |
    |                 | appended to text/books/papers which is        |
    |                 | indirectly visible/editable in XR.            |
    +-----------------+-----------------------------------------------+
    | requestless     | metadata which never spawns new requests      |
    | metadata        | (unlike RDF/HTML, which can cause framerate-  |
    |                 | dropping, hence not used a lot in games)      |
    +-----------------+-----------------------------------------------+
    | FPS             | frames per second in spatial experiences      |
    |                 | (games,VR,AR e.g.), should be as high as      |
    |                 | possible                                      |
    +-----------------+-----------------------------------------------+
    | introspective   | inward sensemaking ("I feel this belongs to   |
    |                 | that")                                        |
    +-----------------+-----------------------------------------------+
    | extrospective   | outward sensemaking ("I'm fairly sure John is |
    |                 | a person who lives in oklahoma")              |
    +-----------------+-----------------------------------------------+
    | ◻               | ascii representation of an 3D object/mesh     |
    +-----------------+-----------------------------------------------+
    | (un)obtrusive   | obtrusive: wrapping human text/thought in     |
    |                 | XML/HTML/JSON obfuscates human text into a    |
    |                 | salad of machine-symbols and words            |
    +-----------------+-----------------------------------------------+
    | flat 3D object  | a 3D object of which all verticies share a    |



van Kammen                 Expires 8 June 2024                 [Page 31]

Internet-Draft                XR Fragments                 December 2023


    |                 | plane                                         |
    +-----------------+-----------------------------------------------+
    | BibTeX          | simple tagging/citing/referencing standard    |
    |                 | for plaintext                                 |
    +-----------------+-----------------------------------------------+
    | BibTag          | a BibTeX tag                                  |
    +-----------------+-----------------------------------------------+
    | (hashtag)bibs   | an easy to speak/type/scan tagging SDL (see   |
    |                 | here (https://github.com/coderofsalvation/    |
    |                 | hashtagbibs) which expands to BibTex/JSON/XML |
    +-----------------+-----------------------------------------------+

                                  Table 13






































van Kammen                 Expires 8 June 2024                 [Page 32]