From 6ffd8a5d816bbe2b8230633f6131201f94bf38da Mon Sep 17 00:00:00 2001 From: Leon van Kammen Date: Wed, 6 Dec 2023 16:40:55 +0100 Subject: [PATCH] update documentation --- doc/RFC_XR_Fragments.html | 408 ++++++++++---- doc/RFC_XR_Fragments.txt | 1102 ++++++++++++++++++++++--------------- doc/RFC_XR_Fragments.xml | 514 +++++++++++------ doc/RFC_XR_Macros.txt | 36 +- doc/RFC_XR_Macros.xml | 6 +- 5 files changed, 1347 insertions(+), 719 deletions(-) diff --git a/doc/RFC_XR_Fragments.html b/doc/RFC_XR_Fragments.html index c29b4f1..6a216f5 100644 --- a/doc/RFC_XR_Fragments.html +++ b/doc/RFC_XR_Fragments.html @@ -81,8 +81,8 @@ value: draft-XRFRAGMENTS-leonvankammen-00

Abstract

This draft is a specification for 4D URLs & hypermediatic navigation, which links together space, time & text together, for hypermedia browsers with- or without a network-connection.
-The specification promotes spatial addressibility, sharing, navigation, query-ing and annotating interactive (text)objects across for (XR) Browsers.
-XR Fragments allows us to enrich existing dataformats, by recursive use of existing proven technologies like URI Fragments and BibTags notation.

+The specification promotes spatial addressibility, sharing, navigation, filtering and databinding objects for (XR) Browsers.
+XR Fragments allows us to better use existing metadata inside 3D scene(files), by connecting it to proven technologies like URI Fragments.

Almost every idea in this document is demonstrated at https://xrfragment.org

@@ -90,14 +90,18 @@ XR Fragments allows us to enrich existing dataformats, by recursive use of exist

Introduction

-

How can we add more features to existing text & 3D scenes, without introducing new dataformats?
+

How can we add more control to existing text & 3D scenes, without introducing new dataformats?
Historically, there’s many attempts to create the ultimate markuplanguage or 3D fileformat.
-The lowest common denominator is: describing/tagging/naming nodes using plain text.
-XR Fragments allows us to enrich/connect existing dataformats, by introducing existing technologies/ideas:

+The lowest common denominator is: designers describing/tagging/naming things using plain text.
+XR Fragments exploits the fact that all 3D models already contain such metadata:

+ +

XR Fragments allows controlling of metadata in 3D scene(files) using URLs

+ +

Or more detailed:

  1. addressibility and hypermediatic navigation of 3D scenes/objects: URI Fragments + src/href spatial metadata
    2. -
  2. Interlinking text/& 3D by collapsing space into a Word Graph (XRWG) to show visible links (and augmenting text with bibs / BibTags appendices (see visual-meta e.g.)
    3. +
  3. Interlinking (text)objects by collapsing space into a Word Graph (XRWG) to show visible links
  4. unlocking spatial potential of the (originally 2D) hashtag (which jumps to a chapter) for navigating XR documents
@@ -107,10 +111,10 @@ XR Fragments allows us to enrich/connect existing dataformats, by introducing ex

Core principle

-

XR Fragments strives to serve (nontechnical/fuzzy) humans first, and machine(implementations) later, by ensuring hasslefree text-vs-thought feedback loops.
-This also means that the repair-ability of machine-matters should be human friendly too (not too complex).
-XR Fragments tries to seek to connect the world of text (semantical web / RDF), and the world of pixels.
-Instead of combining them (in a game-editor e.g.), XR Fragments is opting for a more integrated path towards them, by describing how to make browsers 4D URL-ready:

+

XR Fragments allows controlling of metadata in 3D scene(files) using URLs

+ +

XR Fragments tries to seek to connect the world of text (semantical web / RDF), and the world of pixels.
+Instead of combining them (in a game-editor e.g.), XR Fragments integrates all, by collecting metadata into an XRWG and control it via URL:

@@ -130,14 +134,8 @@ Instead of combining them (in a game-editor e.g.), XR Fragments is opting for a - - - - - - - - + + @@ -154,20 +152,26 @@ Instead of combining them (in a game-editor e.g.), XR Fragments is opting for a - - + + + + + + + + - + - - + +
the hashbushashtags map to camera/scene-projectionshashtags map to document positions
spacetime hashtagspositions camera, triggers scene-preset/timejumps/scrolls to chapterhashtags alter camera/scene/object-projectionshashtags alter document positions
href metadatarepositions camera or animation-rangejumps to cameratriggers predefined viewMedia fragments
href metadatatriggers camera/scene/object/projectionsn/a
href metadata draws visible connection(s) for XRWG ‘tag’n/a
href metadatatriggers predefined viewMedia fragmentsfilters certain (in)visible objectsn/a
@@ -183,13 +187,16 @@ Instead of combining them (in a game-editor e.g.), XR Fragments is opting for a │ 2D URL: ://library.com /document ?search #chapter │ │ │ │ 4D URL: ://park.com /4Dscene.fbx ──> ?misc ──> #view ───> hashbus │ - │ │ #query │ │ + │ │ #filter │ │ │ │ #tag │ │ + │ │ #material │ │ + │ │ #animation │ │ + │ │ #texture │ │ + │ │ #variable │ │ │ │ │ │ │ XRWG <─────────────────────<────────────+ │ │ │ │ │ - │ ├─ objects ───────────────>────────────│ │ - │ └─ text ───────────────>────────────+ │ + │ └─ objects ──────────────>────────────+ │ │ │ │ │ +──────────────────────────────────────────────────────────────────────────────────────────────+ @@ -236,7 +243,7 @@ sub-delims = "," / "=" -pos=1,2,3&rot=0,90,0&q=.foo +pos=1,2,3&rot=0,90,0&foo combinators @@ -275,25 +282,14 @@ sub-delims = "," / "=" #t -vector3 -#t=1,500,1000 -play animation-loop range between frame 500 and 1000, at normal speed - - - -#...... -string -#.cubes #cube -predefined views, XRWG fragments and ID fragments +timevector +#t=2,2000,1 +play animation-loop range between frame 2 and 2000 at (normal) speed 1 -
-

xyz coordinates are similar to ones found in SVG Media Fragments

-
- -

List of metadata for 3D nodes

+

List of metadata for 3D nodes

@@ -327,17 +323,149 @@ sub-delims = "," / "=" - +
tag string "tag": "cubes geo"tag object (for query-use / XRWG highlighting)tag object (for filter-use / XRWG highlighting) custom property in 3D fileformats
+ +

Supported popular compatible 3D fileformats: .gltf, .obj, .fbx, .usdz, .json (THREE.js), .dae and so on.

+
+ +

vector datatypes

+ +

| type | syntax | example | info |

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
vector2
vector3
timevector

NOTE: XR Fragments are optional but also file- and protocol-agnostic, which means that programmatic 3D scene(nodes) can also use the mechanism/metadata.

+

Dynamic XR Fragments (+databindings)

+ +

These are automatic fragment-to-metadata mappings, which only trigger if the 3D scene metadata matches a specific identifier (aliasname e.g.)

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
fragmenttypeexampleinfo
#<aliasname>string#cubesevaluate predefined views (#cubes: #foo&bar e.g.)
#<tag_or_objectname>string#personfocus object(s) with tag: person or name person by looking up XRWG
#<cameraname>string#cam01set camera as active camera
#<objectname>=<material>string=string#car=metallicset material of car to material with name metallic
string=string#product=metallicset material of objects tagged with product to material with name metallic
#<objectname>=<mediafrag>string=media frag#foo=0,1play media src using media fragment URI
#<objectname>=<timevector>string=timevector#sky=0,0.5,0.1,0sets 1D/2D/3D time(line) vectors (uv-position e.g.) to 0,0.5 (and autoscroll x with max 0.1 every second)
#music=1,2play media of object (src: podcast.mp3 e.g.) from beginning (1) at double speed (2)
+

Spatial Referencing 3D

XR Fragments assume the following objectname-to-URIFragment mapping:

@@ -390,8 +518,8 @@ For example, to render a portal with a preview-version of the scene, create an 3 #pos=0,0,0 -vector3 -(re)position camera +vector3 or string +(re)position camera based on coordinates directly, or indirectly using objectname (its worldposition) @@ -415,7 +543,7 @@ For example, to render a portal with a preview-version of the scene, create an 3
  • set the position of the camera accordingly to the vector3 values of #pos
  • rot sets the rotation of the camera (only for non-VR/AR headsets)
  • t sets the playbackspeed and animation-range of the current scene animation(s) or src-mediacontent (video/audioframes e.g., use t=0,7,7 to ‘STOP’ at frame 7 e.g.)
    • -
  • in case an href does not mention any pos-coordinate, pos=0,0,0 will be assumed
    • +
  • after scene load: in case an href does not mention any pos-coordinate, pos=0,0,0 will be assumed

    • Here’s an ascii representation of a 3D scene-graph which contains 3D objects ◻ and their metadata:

    @@ -453,7 +581,7 @@ In case of buttonA the end-user will be teleported to another locat
  • IF a #cube matches anything else in the XR Word Graph (XRWG) draw wires to them (text or related objects).
    • -

    Embedding XR content (src-instancing)

    +

    Embedding XR content using src

    src is the 3D version of the iframe.
    It instances content (in objects) in the current scene/asset.

    @@ -470,22 +598,22 @@ It instances content (in objects) in the current scene/asset.

    src -string (uri, hashtag/query) -#cube
    #sometag
    #q=-ball_inside_cube<br>#q=-/sky -rain<br>#q=-.language .english<br>#q=price:>2 price:<5`
    https://linux.org/penguin.png
    https://linux.world/distrowatch.gltf#t=1,100
    linuxapp://conference/nixworkshop/apply.gltf#q=flyer
    androidapp://page1?tutorial#pos=0,0,1&t1,100 +string (uri, hashtag/filter) +#cube
    #sometag
    #cube&-ball_inside_cube<br>#-sky&-rain<br>#-language&english<br>#price=>5<br>https://linux.org/penguin.png`
    https://linux.world/distrowatch.gltf#t=1,100
    linuxapp://conference/nixworkshop/apply.gltf#-cta&cta_apply
    androidapp://page1?tutorial#pos=0,0,1&t1,100
    foo.mp3#0,0,0 -

    Here’s an ascii representation of a 3D scene-graph with 3D objects ◻ which embeds remote & local 3D objects ◻ with/out using queries:

    +

    Here’s an ascii representation of a 3D scene-graph with 3D objects ◻ which embeds remote & local 3D objects ◻ with/out using filters:

      +────────────────────────────────────────────────────────+  +─────────────────────────+ 
   │                                                        │  │                         │
   │  index.gltf                                            │  │ ocean.com/aquarium.fbx  │
-  │    │                                                   │  │   │                     │
+  │    │                                                   │  │   ├ room                │
   │    ├── ◻ canvas                                        │  │   └── ◻ fishbowl        │
   │    │      └ src: painting.png                          │  │         ├─ ◻ bass       │
   │    │                                                   │  │         └─ ◻ tuna       │
   │    ├── ◻ aquariumcube                                  │  │                         │       
-  │    │      └ src: ://rescue.com/fish.gltf#bass%20tuna   │  +─────────────────────────+
+  │    │      └ src: ://rescue.com/fish.gltf#fishbowl      │  +─────────────────────────+
   │    │                                                   │    
   │    ├── ◻ bedroom                                       │   
   │    │      └ src: #canvas                               │
@@ -497,41 +625,45 @@ It instances content (in objects) in the current scene/asset.

    An XR Fragment-compatible browser viewing this scene, lazy-loads and projects painting.png onto the (plane) object called canvas (which is copy-instanced in the bed and livingroom).
    -Also, after lazy-loading ocean.com/aquarium.gltf, only the queried objects bass and tuna will be instanced inside aquariumcube.
    +Also, after lazy-loading ocean.com/aquarium.gltf, only the queried objects fishbowl (and bass and tuna) will be instanced inside aquariumcube.
    Resizing will be happen accordingly to its placeholder object aquariumcube, see chapter Scaling.

    -

    Instead of cherrypicking objects with #bass&tuna thru src, queries can be used to import the whole scene (and filter out certain objects). See next chapter below.

    +

    Instead of cherrypicking a rootobject #fishbowl with src, additional filters can be used to include/exclude certain objects. See next chapter on filtering below.

    Specification:

      -
    1. local/remote content is instanced by the src (query) value (and attaches it to the placeholder mesh containing the src property)
      2. -
    2. local src values (URL starting with #, like #cube&foo) means only the mentioned objectnames will be copied to the instanced scene (from the current scene) while preserving their names (to support recursive selectors). (example code)
      3. -
    3. local src values indicating a query (#q=), means that all included objects (from the current scene) will be copied to the instanced scene (before applying the query) while preserving their names (to support recursive selectors). (example code)
      4. +
    4. local/remote content is instanced by the src (filter) value (and attaches it to the placeholder mesh containing the src property)
      5. +
    5. by default all objects are loaded into the instanced src (scene) object (but not shown yet)
      6. +
    6. local src values (#... e.g.) starting with a non-negating filter (#cube e.g.) will (deep)reparent that object (with name cube) as the new root of the scene at position 0,0,0
      7. +
    7. local src values should respect (negative) filters (#-foo&price=>3)
    8. the instanced scene (from a src value) should be scaled accordingly to its placeholder object or scaled relatively based on the scale-property (of a geometry-less placeholder, an ‘empty’-object in blender e.g.). For more info see Chapter Scaling.
    9. external src values should be served with appropriate mimetype (so the XR Fragment-compatible browser will now how to render it). The bare minimum supported mimetypes are:
    10. src values should make its placeholder object invisible, and only flush its children when the resolved content can succesfully be retrieved (see broken links)
    11. external src values should respect the fallback link mechanism (see broken links
    12. when the placeholder object is a 2D plane, but the mimetype is 3D, then render the spatial content on that plane via a stencil buffer.
    13. src-values are non-recursive: when linking to an external object (src: foo.fbx#bar), then src-metadata on object bar should be ignored.
      14. -
    14. clicking on external src-values always allow sourceportation: teleporting to the origin URI to which the object belongs.
      15. +
    15. an external src-value should always allow a sourceportation icon within 3 meter: teleporting to the origin URI to which the object belongs.
    16. when only one object was cherrypicked (#cube e.g.), set its position to 0,0,0
      17. +
    17. when the enduser clicks an href with #t=1,0,0 (play) will be applied to all src mediacontent with a timeline (mp4/mp3 e.g.)
      18. +
    18. a non-euclidian portal can be rendered for flat 3D objects (using stencil buffer e.g.) in case ofspatial src-values (an object #world3 or URL world3.fbx e.g.).
      +
    • model/gltf-binary
    • model/gltf+json
    • image/png
    • image/jpg
      • -
    • text/plain;charset=utf-8;bib=^@
      • +
    • text/plain;charset=utf-8

    » example implementation
    » example 3D asset
    » discussion

    -

    Navigating content (internal/outbound href portals)

    +

    Navigating content href portals

    navigation, portals & mutations

    @@ -566,13 +698,25 @@ Resizing will be happen accordingly to its placeholder object aquariumcube

  • in case of navigating to a new [[pos)ition, “first” navigate to the “current position” so that the “back-button” of the “browser-history” always refers to the previous position (see [here)

    • -

  • portal-rendering: a 2:1 ratio texture-material indicates an equirectangular projection

    • +

  • ignore previous rule in special cases, like clicking an href using camera-portal collision (the back-button would cause a teleport-loop)

    • » example implementation
    » example 3D asset
    » discussion

    +

    Walking surfaces

    + +

    XR Fragment-compatible viewers can infer this data based scanning the scene for:

    + +
      +
    1. materialless (nameless & textureless) mesh-objects (without src and href)
      2. +
    + +
    +

    optionally the viewer can offer thumbstick, mouse or joystick teleport-tools for non-roomscale VR/AR setups.

    +
    +

    UX spec

    End-users should always have read/write access to:

    @@ -616,7 +760,53 @@ How does the scale of the object (with the embedded properties) impact the scale

    TODO: needs intermediate visuals to make things more obvious

    -

    XR Fragment queries

    +

    XR Fragment: pos

    + +

    XR Fragment: rot

    + +

    XR Fragment: t

    + +

    controls the animation(s) of the scene (or src resource which contains a timeline)

    + +

    | fragment | type | functionality | +| #t=1,1,100 | [vector3|vector] | speed,framestart,framestop |

    + + + +

    | Example Value | Explanation | +|-|-| +| 1,1,100 | play loop between frame 1 and 100 | +| 1,1,0 | play once from frame 1 (oneshot) | +| 1,0,0 | play (previously set looprange if any) | +| 0,0,0 | pause | +| 1,1,1 | play and auto-loop between begin and end of duration | +| -1,0,0 | reverse playback speed | +| 2.3,0,0 | set (forward) playback speed to 2.3 (no restart) | +| -2.3,0,0 | set (reverse) playback speed to -2.3 ( no restart)| +| -2.3,100,0 | set (reverse) playback speed to -2.3 restarting from frame 100 |

    + +

    [[» example implementation|https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/xrf/t.js]]
    +[[» discussion|https://github.com/coderofsalvation/xrfragment/issues/10]]

    + +

    XR audio/video integration

    + +

    To play global audio/video items:

    + +
      +
    1. add a src: foo.mp3 or src: bar.mp4 metadata to a 3D object (cube e.g.)
      2. +
    2. to disable auto-play and global timeline ([[#t=|t]]) control: hardcode a [[#t=|t]] XR Fragment: (src: bar.mp3#t=0,0,0 e.g.)
      3. +
    3. to play it, add href: #cube somewhere else
      4. +
    4. when the enduser clicks the href, #t=1,0,0 (play) will be applied to the src value
      5. +
    5. to play a single animation, add href: #animationname=1,0,0 somewhere else
      6. +
    + +
    +

    NOTE: hardcoded framestart/framestop uses sampleRate/fps of embedded audio/video, otherwise the global fps applies. For more info see [[#t|t]].

    +
    + +

    XR Fragment filters

    Include, exclude, hide/shows objects using space-separated strings:

    @@ -630,35 +820,36 @@ How does the scale of the object (with the embedded properties) impact the scale -#q=-sky +#-sky show everything except object named sky -#q=-tag:language tag:english +#-language&english hide everything with tag language, but show all tag english objects -#q=price:>2 price:<5 -of all objects with property price, show only objects with value between 2 and 5 +#-price&price=>10 +hide all objects with property price, then only show object with price above 10

    It’s simple but powerful syntax which allows filtering the scene using searchengine prompt-style feeling:

      -
    1. queries are a way to traverse a scene, and filter objects based on their tag- or property-values.
      2. -
    2. words like german match tag-metadata of 3D objects like "tag":"german"
      3. -
    3. words like german match (XR Text) objects with (Bib(s)TeX) tags like #KarlHeinz@german or @german{KarlHeinz, ... e.g.
      4. +
    4. filters are a way to traverse a scene, and filter objects based on their name, tag- or property-values.

    including/excluding

    +

    By default, selectors work like photoshop-layers: they scan for matching layer(name/properties) within the scene-graph. +Each matched object (not their children) will be toggled (in)visible when selecting.

    + @@ -670,56 +861,55 @@ How does the scale of the object (with the embedded properties) impact the scale - + - - + + - - + + - - + +
    -removes/hides object(s)hides object(s) (#-myobject&-objects e.g.
    :indicates an object-embedded custom property key/value=indicates an object-embedded custom property key/value (#price=4&category=foo e.g.)
    > <compare float or int number=> =<compare float or int number (#price=>4 e.g.)
    /reference to root-scene.
    Useful in case of (preventing) showing/hiding objects in nested scenes (instanced by src) (*)    		*deepselect: automatically select children of selected object, including local (nonremote) embedded objects (starting with #)
    -

    * = #q=-/cube hides object cube only in the root-scene (not nested cube objects)
    #q=-cube hides both object cube in the root-scene AND nested skybox objects |

    +

    NOTE 1: after an external embedded object has been instanced (src: https://y.com/bar.fbx#room e.g.), filters do not affect them anymore (reason: local tag/name collisions can be mitigated easily, but not in case of remote content).

    + +

    NOTE 2: depending on the used 3D framework, toggling objects (in)visible should happen by enabling/disableing writing to the colorbuffer (to allow children being still visible while their parents are invisible).

    » example implementation -» example 3D asset +» example 3D asset » discussion

    -

    Query Parser

    +

    Filter Parser

    -

    Here’s how to write a query parser:

    +

    Here’s how to write a filter parser:

      -
    1. create an associative array/object to store query-arguments as objects
      2. -
    2. detect object id’s & properties foo:1 and foo (reference regex: /^.*:[><=!]?/ )
      3. -
    3. detect excluders like -foo,-foo:1,-.foo,-/foo (reference regex: /^-/ )
      4. -
    4. detect root selectors like /foo (reference regex: /^[-]?\// )
      5. -
    5. detect number values like foo:1 (reference regex: /^[0-9\.]+$/ )
      6. -
    6. for every query token split string on :
      7. -
    7. create an empty array rules
      8. -
    8. then strip key-operator: convert “-foo” into “foo”
      9. -
    9. add operator and value to rule-array
      10. -
    10. therefore we we set id to true or false (false=excluder -)
      11. +
    11. create an associative array/object to store filter-arguments as objects
      12. +
    12. detect object id’s & properties foo=1 and foo (reference regex= ~/^.*=[><=]?/ )
      13. +
    13. detect excluders like -foo,-foo=1,-.foo,-/foo (reference regex= /^-/ )
      14. +
    14. detect root selectors like /foo (reference regex= /^[-]?\// )
      15. +
    15. detect number values like foo=1 (reference regex= /^[0-9\.]+$/ )
      16. +
    16. detect operators so you can easily strip keys (reference regex= /(^-|\*$)/ )
      17. +
    17. detect exclude keys like -foo (reference regex= /^-/ )
      18. +
    18. for every filter token split string on =
    19. and we set root to true or false (true=/ root selector is present)
      20. -
    20. we convert key ‘/foo’ into ‘foo’
      21. -
    21. finally we add the key/value to the store like store.foo = {id:false,root:true} e.g.
      22. +
    22. therefore we we set show to true or false (false=excluder -)
    -

    An example query-parser (which compiles to many languages) can be found here

    +

    An example filter-parser (which compiles to many languages) can be found here

    Visible links

    @@ -1103,12 +1293,12 @@ here are some hashtagbibs followed by bibtex: │ │ │ index.gltf │ │ │ │ - │ │ #: #q=-offlinetext │ + │ │ #: #-offlinetext │ │ │ │ │ ├── ◻ buttonA │ │ │ └ href: http://foo.io/campagne.fbx │ │ │ └ href@404: ipfs://foo.io/campagne.fbx │ - │ │ └ href@400: #q=clienterrortext │ + │ │ └ href@400: #clienterrortext │ │ │ └ ◻ offlinetext │ │ │ │ │ └── ◻ embeddedObject <--------- the meshdata inside embeddedObject will (not) @@ -1132,12 +1322,12 @@ Consider 3D scenes linking to eachother using these href values:

    These links would all show visible links to math-tagged objects in the scene.
    -To filter out non-related objects one could take it a step further using queries:

    +To filter out non-related objects one could take it a step further using filters:

    @@ -1177,6 +1367,15 @@ Non-HTML Hypermedia browsers should make browser extensions the right place, to
  • NLNET
  • Future of Text
  • visual-meta.info
    • +
  • Michiel Leenaars
    • +
  • Gerben van der Broeke
    • +
  • Mauve
    • +
  • Jens Finkhäuser
    • +
  • Marc Belmont
    • +
  • Tim Gerritsen
    • +
  • Frode Hegland
    • +
  • Brandel Zackernuk
    • +
  • Mark Anderson

    • Appendix: Definitions

    @@ -1256,8 +1455,8 @@ Non-HTML Hypermedia browsers should make browser extensions the right place, to -query -an URI Fragment-operator which queries object(s) from a scene like #q=cube +filter +URI Fragment(s) which show/hide object(s) in a scene based on name/tag/property (#cube&-price=>3) @@ -1295,6 +1494,11 @@ Non-HTML Hypermedia browsers should make browser extensions the right place, to obtrusive: wrapping human text/thought in XML/HTML/JSON obfuscates human text into a salad of machine-symbols and words + +flat 3D object +a 3D object of which all verticies share a plane + + BibTeX simple tagging/citing/referencing standard for plaintext diff --git a/doc/RFC_XR_Fragments.txt b/doc/RFC_XR_Fragments.txt index e2d6609..b96a15e 100644 --- a/doc/RFC_XR_Fragments.txt +++ b/doc/RFC_XR_Fragments.txt @@ -3,7 +3,7 @@ Internet Engineering Task Force L.R. van Kammen -Internet-Draft 12 October 2023 +Internet-Draft 6 December 2023 Intended status: Informational @@ -18,11 +18,10 @@ Abstract links together space, time & text together, for hypermedia browsers with- or without a network-connection. The specification promotes spatial addressibility, sharing, - navigation, query-ing and annotating interactive (text)objects across - for (XR) Browsers. - XR Fragments allows us to enrich existing dataformats, by recursive - use of existing proven technologies like URI Fragments - (https://en.wikipedia.org/wiki/URI_fragment) and BibTags notation. + navigation, filtering and databinding objects for (XR) Browsers. + XR Fragments allows us to better use existing metadata inside 3D + scene(files), by connecting it to proven technologies like URI + Fragments (https://en.wikipedia.org/wiki/URI_fragment). Almost every idea in this document is demonstrated at https://xrfragment.org (https://xrfragment.org) @@ -42,7 +41,7 @@ Status of This Memo time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on 14 April 2024. + This Internet-Draft will expire on 8 June 2024. Copyright Notice @@ -53,9 +52,10 @@ Copyright Notice -van Kammen Expires 14 April 2024 [Page 1] + +van Kammen Expires 8 June 2024 [Page 1] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 This document is subject to BCP 78 and the IETF Trust's Legal @@ -69,63 +69,73 @@ Internet-Draft XR Fragments October 2023 Table of Contents - 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 + 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Core principle . . . . . . . . . . . . . . . . . . . . . . . 3 3. Conventions and Definitions . . . . . . . . . . . . . . . . . 5 3.1. XR Fragment URI Grammar . . . . . . . . . . . . . . . . . 5 4. List of URI Fragments . . . . . . . . . . . . . . . . . . . . 6 - 5. List of metadata for 3D nodes . . . . . . . . . . . . . . . . 6 - 6. Spatial Referencing 3D . . . . . . . . . . . . . . . . . . . 7 - 7. Navigating 3D . . . . . . . . . . . . . . . . . . . . . . . . 8 - 8. Top-level URL processing . . . . . . . . . . . . . . . . . . 9 - 9. Embedding XR content (src-instancing) . . . . . . . . . . . . 9 - 10. Navigating content (internal/outbound href portals) . . . . . 12 - 10.1. UX spec . . . . . . . . . . . . . . . . . . . . . . . . 13 - 10.2. Scaling instanced content . . . . . . . . . . . . . . . 13 - 11. XR Fragment queries . . . . . . . . . . . . . . . . . . . . . 14 - 11.1. including/excluding . . . . . . . . . . . . . . . . . . 15 - 11.2. Query Parser . . . . . . . . . . . . . . . . . . . . . . 15 - 12. Visible links . . . . . . . . . . . . . . . . . . . . . . . . 16 - 13. Text in XR (tagging,linking to spatial objects) . . . . . . . 16 - 13.1. Default Data URI mimetype . . . . . . . . . . . . . . . 20 - 13.2. URL and Data URI . . . . . . . . . . . . . . . . . . . . 21 - 13.3. XR Text example parser . . . . . . . . . . . . . . . . . 22 - 14. Transclusion (broken link) resolution . . . . . . . . . . . . 24 - 15. Topic-based index-less Webrings . . . . . . . . . . . . . . . 25 - 16. Security Considerations . . . . . . . . . . . . . . . . . . . 26 - 17. FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 - 18. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 27 - 19. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 27 - 20. Appendix: Definitions . . . . . . . . . . . . . . . . . . . . 27 + 4.1. List of metadata for 3D nodes . . . . . . . . . . . . . . 6 + 4.2. vector datatypes . . . . . . . . . . . . . . . . . . . . 7 + 4.3. Dynamic XR Fragments (+databindings) . . . . . . . . . . 7 + 5. Spatial Referencing 3D . . . . . . . . . . . . . . . . . . . 8 + 6. Navigating 3D . . . . . . . . . . . . . . . . . . . . . . . . 9 + 7. Top-level URL processing . . . . . . . . . . . . . . . . . . 10 + 8. Embedding XR content using src . . . . . . . . . . . . . . . 11 + 9. Navigating content href portals . . . . . . . . . . . . . . . 13 + 9.1. Walking surfaces . . . . . . . . . . . . . . . . . . . . 15 + 9.2. UX spec . . . . . . . . . . . . . . . . . . . . . . . . . 15 + 9.3. Scaling instanced content . . . . . . . . . . . . . . . . 15 + 10. XR Fragment: pos . . . . . . . . . . . . . . . . . . . . . . 16 + 11. XR Fragment: rot . . . . . . . . . . . . . . . . . . . . . . 16 + 12. XR Fragment: t . . . . . . . . . . . . . . . . . . . . . . . 16 + 13. XR audio/video integration . . . . . . . . . . . . . . . . . 17 + 14. XR Fragment filters . . . . . . . . . . . . . . . . . . . . . 17 + 14.1. including/excluding . . . . . . . . . . . . . . . . . . 18 + 14.2. Filter Parser . . . . . . . . . . . . . . . . . . . . . 18 + 15. Visible links . . . . . . . . . . . . . . . . . . . . . . . . 19 + 16. Text in XR (tagging,linking to spatial objects) . . . . . . . 19 + 16.1. Default Data URI mimetype . . . . . . . . . . . . . . . 23 + 16.2. URL and Data URI . . . . . . . . . . . . . . . . . . . . 24 + 16.3. XR Text example parser . . . . . . . . . . . . . . . . . 25 + 17. Transclusion (broken link) resolution . . . . . . . . . . . . 27 + 18. Topic-based index-less Webrings . . . . . . . . . . . . . . . 28 + 19. Security Considerations . . . . . . . . . . . . . . . . . . . 29 + 20. FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 + 21. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 30 + 22. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 30 + 23. Appendix: Definitions . . . . . . . . . . . . . . . . . . . . 30 + + + + + +van Kammen Expires 8 June 2024 [Page 2] + +Internet-Draft XR Fragments December 2023 + 1. Introduction - How can we add more features to existing text & 3D scenes, without + How can we add more control to existing text & 3D scenes, without introducing new dataformats? Historically, there's many attempts to create the ultimate markuplanguage or 3D fileformat. - The lowest common denominator is: describing/tagging/naming nodes - using *plain text*. + The lowest common denominator is: designers describing/tagging/naming + things using *plain text*. + XR Fragments exploits the fact that all 3D models already contain + such metadata: + *XR Fragments allows controlling of metadata in 3D scene(files) using + URLs* - -van Kammen Expires 14 April 2024 [Page 2] - -Internet-Draft XR Fragments October 2023 - - - XR Fragments allows us to enrich/connect existing dataformats, by - introducing existing technologies/ideas: + Or more detailed: 1. addressibility and hypermediatic (https://github.com/coderofsalvation/hypermediatic) navigation of 3D scenes/objects: URI Fragments (https://en.wikipedia.org/wiki/ URI_fragment) + src/href spatial metadata - 2. Interlinking text/& 3D by collapsing space into a Word Graph - (XRWG) to show visible links (#visible-links) (and augmenting - text with bibs (https://github.com/coderofsalvation/tagbibs) / - BibTags (https://en.wikipedia.org/wiki/BibTeX) appendices (see - visual-meta (https://visual-meta.info) e.g.) + 2. Interlinking (text)objects by collapsing space into a Word Graph + (XRWG) to show visible links (#visible-links) 3. unlocking spatial potential of the (originally 2D) hashtag (which jumps to a chapter) for navigating XR documents @@ -134,16 +144,14 @@ Internet-Draft XR Fragments October 2023 2. Core principle - XR Fragments strives to serve (nontechnical/fuzzy) humans first, and - machine(implementations) later, by ensuring hasslefree text-vs- - thought feedback loops. - This also means that the repair-ability of machine-matters should be - human friendly too (not too complex). + *XR Fragments allows controlling of metadata in 3D scene(files) using + URLs* + XR Fragments tries to seek to connect the world of text (semantical web / RDF), and the world of pixels. - Instead of combining them (in a game-editor e.g.), XR Fragments is - opting for a more integrated path *towards* them, by describing how - to make browsers *4D URL-ready*: + Instead of combining them (in a game-editor e.g.), XR Fragments + *integrates all*, by collecting metadata into an XRWG and control it + via URL: @@ -157,46 +165,38 @@ Internet-Draft XR Fragments October 2023 - - - - - - - - -van Kammen Expires 14 April 2024 [Page 3] +van Kammen Expires 8 June 2024 [Page 3] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 - +===========+=============================+====================+ - | principle | XR 4D URL | HTML 2D URL | - +===========+=============================+====================+ - | the XRWG | wordgraph (collapses 3D | Ctrl-F (find) | - | | scene to tags) | | - +-----------+-----------------------------+--------------------+ - | the | hashtags map to camera/ | hashtags map to | - | hashbus | scene-projections | document positions | - +-----------+-----------------------------+--------------------+ - | spacetime | positions camera, triggers | jumps/scrolls to | - | hashtags | scene-preset/time | chapter | - +-----------+-----------------------------+--------------------+ - | src | renders content and offers | renders content | - | metadata | sourceportation | | - +-----------+-----------------------------+--------------------+ - | href | teleports to other XR | jumps to other | - | metadata | document | HTML document | - +-----------+-----------------------------+--------------------+ - | href | repositions camera or | jumps to camera | - | metadata | animation-range | | - +-----------+-----------------------------+--------------------+ - | href | draws visible connection(s) | | - | metadata | for XRWG 'tag' | | - +-----------+-----------------------------+--------------------+ - | href | triggers predefined view | Media fragments | - | metadata | | | - +-----------+-----------------------------+--------------------+ + +===========+===============================+====================+ + | principle | XR 4D URL | HTML 2D URL | + +===========+===============================+====================+ + | the XRWG | wordgraph (collapses 3D scene | Ctrl-F (find) | + | | to tags) | | + +-----------+-------------------------------+--------------------+ + | the | hashtags alter camera/scene/ | hashtags alter | + | hashbus | object-projections | document positions | + +-----------+-------------------------------+--------------------+ + | src | renders content and offers | renders content | + | metadata | sourceportation | | + +-----------+-------------------------------+--------------------+ + | href | teleports to other XR | jumps to other | + | metadata | document | HTML document | + +-----------+-------------------------------+--------------------+ + | href | triggers predefined view | Media fragments | + | metadata | | | + +-----------+-------------------------------+--------------------+ + | href | triggers camera/scene/object/ | n/a | + | metadata | projections | | + +-----------+-------------------------------+--------------------+ + | href | draws visible connection(s) | n/a | + | metadata | for XRWG 'tag' | | + +-----------+-------------------------------+--------------------+ + | href | filters certain (in)visible | n/a | + | metadata | objects | | + +-----------+-------------------------------+--------------------+ Table 1 @@ -221,9 +221,9 @@ Internet-Draft XR Fragments October 2023 -van Kammen Expires 14 April 2024 [Page 4] +van Kammen Expires 8 June 2024 [Page 4] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 +──────────────────────────────────────────────────────────────────────────────────────────────+ @@ -233,13 +233,16 @@ Internet-Draft XR Fragments October 2023 │ 2D URL: ://library.com /document ?search #chapter │ │ │ │ 4D URL: ://park.com /4Dscene.fbx ──> ?misc ──> #view ───> hashbus │ - │ │ #query │ │ + │ │ #filter │ │ │ │ #tag │ │ + │ │ #material │ │ + │ │ #animation │ │ + │ │ #texture │ │ + │ │ #variable │ │ │ │ │ │ │ XRWG <─────────────────────<────────────+ │ │ │ │ │ - │ ├─ objects ───────────────>────────────│ │ - │ └─ text ───────────────>────────────+ │ + │ └─ objects ──────────────>────────────+ │ │ │ │ │ +──────────────────────────────────────────────────────────────────────────────────────────────+ @@ -274,89 +277,179 @@ Internet-Draft XR Fragments October 2023 - - - -van Kammen Expires 14 April 2024 [Page 5] +van Kammen Expires 8 June 2024 [Page 5] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 - +=============================+=================================+ - | Demo | Explanation | - +=============================+=================================+ - | pos=1,2,3 | vector/coordinate argument e.g. | - +-----------------------------+---------------------------------+ - | pos=1,2,3&rot=0,90,0&q=.foo | combinators | - +-----------------------------+---------------------------------+ + +==========================+=================================+ + | Demo | Explanation | + +==========================+=================================+ + | pos=1,2,3 | vector/coordinate argument e.g. | + +--------------------------+---------------------------------+ + | pos=1,2,3&rot=0,90,0&foo | combinators | + +--------------------------+---------------------------------+ - Table 2 + Table 2 | this is already implemented in all browsers 4. List of URI Fragments - +==========+=========+===============+==============================+ - | fragment | type | example | info | - +==========+=========+===============+==============================+ - | #pos | vector3 | #pos=0.5,0,0 | positions camera (or XR | - | | | | floor) to xyz-coord | - | | | | 0.5,0,0, | - +----------+---------+---------------+------------------------------+ - | #rot | vector3 | #rot=0,90,0 | rotates camera to xyz- | - | | | | coord 0.5,0,0 | - +----------+---------+---------------+------------------------------+ - | #t | vector3 | #t=1,500,1000 | play animation-loop range | - | | | | between frame 500 and | - | | | | 1000, at normal speed | - +----------+---------+---------------+------------------------------+ - | #...... | string | #.cubes #cube | predefined views, XRWG | - | | | | fragments and ID | - | | | | fragments | - +----------+---------+---------------+------------------------------+ + +==========+============+==============+===========================+ + | fragment | type | example | info | + +==========+============+==============+===========================+ + | #pos | vector3 | #pos=0.5,0,0 | positions camera (or XR | + | | | | floor) to xyz-coord | + | | | | 0.5,0,0, | + +----------+------------+--------------+---------------------------+ + | #rot | vector3 | #rot=0,90,0 | rotates camera to xyz- | + | | | | coord 0.5,0,0 | + +----------+------------+--------------+---------------------------+ + | #t | timevector | #t=2,2000,1 | play animation-loop range | + | | | | between frame 2 and 2000 | + | | | | at (normal) speed 1 | + +----------+------------+--------------+---------------------------+ - Table 3 + Table 3 - | xyz coordinates are similar to ones found in SVG Media Fragments +4.1. List of metadata for 3D nodes -5. List of metadata for 3D nodes - - +======+========+==========+==================+===================+ - | key | type | example | function | existing | - | | | (JSON) | | compatibility | - +======+========+==========+==================+===================+ - | href | string | "href": | XR teleport | custom property | - | | | "b.gltf" | | in 3D fileformats | - +------+--------+----------+------------------+-------------------+ - | src | string | "src": | XR embed / | custom property | - | | | "#cube" | teleport | in 3D fileformats | + +======+========+==========+===================+===================+ + | key | type | example | function | existing | + | | | (JSON) | | compatibility | + +======+========+==========+===================+===================+ + | href | string | "href": | XR teleport | custom property | + | | | "b.gltf" | | in 3D fileformats | + +------+--------+----------+-------------------+-------------------+ + | src | string | "src": | XR embed / | custom property | + | | | "#cube" | teleport | in 3D fileformats | + +------+--------+----------+-------------------+-------------------+ + | tag | string | "tag": | tag object (for | custom property | + | | | "cubes | filter-use / XRWG | in 3D fileformats | + | | | geo" | highlighting) | | + +------+--------+----------+-------------------+-------------------+ -van Kammen Expires 14 April 2024 [Page 6] + +van Kammen Expires 8 June 2024 [Page 6] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 - +------+--------+----------+------------------+-------------------+ - | tag | string | "tag": | tag object (for | custom property | - | | | "cubes | query-use / XRWG | in 3D fileformats | - | | | geo" | highlighting) | | - +------+--------+----------+------------------+-------------------+ + Table 4 - Table 4 + | Supported popular compatible 3D fileformats: .gltf, .obj, .fbx, + | .usdz, .json (THREE.js), .dae and so on. - Supported popular compatible 3D fileformats: .gltf, .obj, .fbx, - .usdz, .json (THREE.js), .dae and so on. +4.2. vector datatypes + + | type | syntax | example | info | + + +------------+ + | vector2 | + +------------+ + | vector3 | + +------------+ + | timevector | + +------------+ + +------------+ + +------------+ + +------------+ + +------------+ + +------------+ + +------------+ + +------------+ + +------------+ + +------------+ + + Table 5 | NOTE: XR Fragments are optional but also file- and protocol- | agnostic, which means that programmatic 3D scene(nodes) can also | use the mechanism/metadata. -6. Spatial Referencing 3D +4.3. Dynamic XR Fragments (+databindings) + + These are automatic fragment-to-metadata mappings, which only trigger + if the 3D scene metadata matches a specific identifier (aliasname + e.g.) + + + + + + + + + + + + + + +van Kammen Expires 8 June 2024 [Page 7] + +Internet-Draft XR Fragments December 2023 + + + +==========================+=======================+=================+=======================+ + |fragment |type |example |info | + +==========================+=======================+=================+=======================+ + |# |string |#cubes |evaluate predefined | + | | | |views (#cubes: #foo&bar| + | | | |e.g.) | + +--------------------------+-----------------------+-----------------+-----------------------+ + |# |string |#person |focus object(s) with | + | | | |tag: person or name | + | | | |person by looking up | + | | | |XRWG | + +--------------------------+-----------------------+-----------------+-----------------------+ + |# |string |#cam01 |set camera as active | + | | | |camera | + +--------------------------+-----------------------+-----------------+-----------------------+ + |#= |string=string |#car=metallic |set material of car to | + | | | |material with name | + | | | |metallic | + +--------------------------+-----------------------+-----------------+-----------------------+ + | |string=string |#product=metallic|set material of objects| + | | | |tagged with product to | + | | | |material with name | + | | | |metallic | + +--------------------------+-----------------------+-----------------+-----------------------+ + |#= |string=media frag |#foo=0,1 |play media src using | + | |(https://www.w3.org/TR/| |media fragment URI | + | |media-frags/#valid-uri)| |(https://www.w3.org/TR/| + | | | |media-frags/#valid-uri)| + +--------------------------+-----------------------+-----------------+-----------------------+ + |#=|string=timevector |#sky=0,0.5,0.1,0 |sets 1D/2D/3D | + | | | |time(line) vectors (uv-| + | | | |position e.g.) to 0,0.5| + | | | |(and autoscroll x with | + | | | |max 0.1 every second) | + +--------------------------+-----------------------+-----------------+-----------------------+ + | | |#music=1,2 |play media of object | + | | | |(src: podcast.mp3 e.g.)| + | | | |from beginning (1) at | + | | | |double speed (2) | + +--------------------------+-----------------------+-----------------+-----------------------+ + + Table 6 + +5. Spatial Referencing 3D XR Fragments assume the following objectname-to-URIFragment mapping: + + + + +van Kammen Expires 8 June 2024 [Page 8] + +Internet-Draft XR Fragments December 2023 + + my.io/scene.fbx +─────────────────────────────+ │ sky │ src: http://my.io/scene.fbx#sky (includes building,mainobject,floor) @@ -383,43 +476,44 @@ Internet-Draft XR Fragments October 2023 * href: https://scene.fbx * src: https://otherworld.gltf#mainobject - - - - - - -van Kammen Expires 14 April 2024 [Page 7] - -Internet-Draft XR Fragments October 2023 - - | It also allows *sourceportation*, which basically means the | enduser can teleport to the original XR Document of an src | embedded object, and see a visible connection to the particular | embedded object. Basically an embedded link becoming an outbound | link by activating it. -7. Navigating 3D +6. Navigating 3D - +====================+=========+==================================+ - | fragment | type | functionality | - +====================+=========+==================================+ - | #pos=0,0,0 | vector3 | (re)position camera | - +--------------------+---------+----------------------------------+ - | #t=0,100 | vector3 | set playback speed, and | - | | | (re)position looprange of scene- | - | | | animation or src-mediacontent | - +--------------------+---------+----------------------------------+ - | #rot=0,90,0 | vector3 | rotate camera | - +--------------------+---------+----------------------------------+ + +====================+=========+=============================+ + | fragment | type | functionality | + +====================+=========+=============================+ + | #pos=0,0,0 | vector3 | (re)position camera based | + | | or | on coordinates directly, or | + | | string | indirectly using objectname | + | | | (its worldposition) | + +--------------------+---------+-----------------------------+ + | #t=0,100 | vector3 | set playback speed, and | + | | | (re)position looprange of | + | | | scene-animation or src- | + | | | mediacontent | + +--------------------+---------+-----------------------------+ + | #rot=0,90,0 | vector3 | rotate camera | - Table 5 - » example implementation + +van Kammen Expires 8 June 2024 [Page 9] + +Internet-Draft XR Fragments December 2023 + + + +--------------------+---------+-----------------------------+ + + Table 7 + + » example implementation (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/ three/xrf/pos.js) - » discussion (https://github.com/coderofsalvation/xrfragment/ + » discussion (https://github.com/coderofsalvation/xrfragment/ issues/5) 1. the Y-coordinate of pos identifies the floorposition. This means @@ -431,24 +525,11 @@ Internet-Draft XR Fragments October 2023 4. t sets the playbackspeed and animation-range of the current scene animation(s) or src-mediacontent (video/audioframes e.g., use t=0,7,7 to 'STOP' at frame 7 e.g.) - 5. in case an href does not mention any pos-coordinate, pos=0,0,0 - will be assumed + 5. after scene load: in case an href does not mention any pos- + coordinate, pos=0,0,0 will be assumed Here's an ascii representation of a 3D scene-graph which contains 3D - objects ◻ and their metadata: - - - - - - - - - -van Kammen Expires 14 April 2024 [Page 8] - -Internet-Draft XR Fragments October 2023 - + objects ◻ and their metadata: +────────────────────────────────────────────────────────+ │ │ @@ -469,10 +550,18 @@ Internet-Draft XR Fragments October 2023 *replace the current scene* with a new one, like other.fbx, and assume pos=0,0,0. -8. Top-level URL processing +7. Top-level URL processing | Example URL: ://foo/world.gltf#cube&pos=0,0,0 + + + +van Kammen Expires 8 June 2024 [Page 10] + +Internet-Draft XR Fragments December 2023 + + The URL-processing-flow for hypermedia browsers goes like this: 1. IF a #cube matches a custom property-key (of an object) in the 3D @@ -491,51 +580,53 @@ Internet-Draft XR Fragments October 2023 5. IF a #cube matches anything else in the XR Word Graph (XRWG) draw wires to them (text or related objects). -9. Embedding XR content (src-instancing) +8. Embedding XR content using src src is the 3D version of the iframe. It instances content (in objects) in the current scene/asset. + +========+========+===================================================+ + |fragment|type |example value | + +========+========+===================================================+ + |src |string |#cube | + | |(uri, |#sometag | + | |hashtag/|#cube&-ball_inside_cube
    #-sky&-rain
    #- | + | |filter) |language&english
    #price=>5
    https://linux.org/| + | | |penguin.png` (https://linux.org/penguin.png`) | + | | |https://linux.world/distrowatch.gltf#t=1,100 | + | | |linuxapp://conference/nixworkshop/apply.gltf#- | + | | |cta&cta_apply | + | | |androidapp://page1?tutorial#pos=0,0,1&t1,100 | + | | |foo.mp3#0,0,0 | + +--------+--------+---------------------------------------------------+ + + Table 8 + + Here's an ascii representation of a 3D scene-graph with 3D objects ◻ + which embeds remote & local 3D objects ◻ with/out using filters: -van Kammen Expires 14 April 2024 [Page 9] + + + + +van Kammen Expires 8 June 2024 [Page 11] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 - +========+==========+==============================================+ - |fragment| type | example value | - +========+==========+==============================================+ - |src | string | #cube | - | | (uri, | #sometag | - | | hashtag/ | #q=-ball_inside_cube
    #q=-/sky | - | | query) | -rain
    #q=-.language | - | | | .english
    #q=price:>2 | - | | | price:<5`
    https://linux.org/penguin.png | - | | | https://linux.world/distrowatch.gltf#t=1,100 | - | | | linuxapp://conference/nixworkshop/ | - | | | apply.gltf#q=flyer | - | | | androidapp://page1?tutorial#pos=0,0,1&t1,100 | - +--------+----------+----------------------------------------------+ - - Table 6 - - Here's an ascii representation of a 3D scene-graph with 3D objects - ◻ which embeds remote & local 3D objects ◻ with/out using - queries: - +────────────────────────────────────────────────────────+ +─────────────────────────+ │ │ │ │ │ index.gltf │ │ ocean.com/aquarium.fbx │ - │ │ │ │ │ │ + │ │ │ │ ├ room │ │ ├── ◻ canvas │ │ └── ◻ fishbowl │ │ │ └ src: painting.png │ │ ├─ ◻ bass │ │ │ │ │ └─ ◻ tuna │ │ ├── ◻ aquariumcube │ │ │ - │ │ └ src: ://rescue.com/fish.gltf#bass%20tuna │ +─────────────────────────+ + │ │ └ src: ://rescue.com/fish.gltf#fishbowl │ +─────────────────────────+ │ │ │ │ ├── ◻ bedroom │ │ │ └ src: #canvas │ @@ -549,88 +640,96 @@ Internet-Draft XR Fragments October 2023 projects painting.png onto the (plane) object called canvas (which is copy-instanced in the bed and livingroom). Also, after lazy-loading ocean.com/aquarium.gltf, only the queried - objects bass and tuna will be instanced inside aquariumcube. + objects fishbowl (and bass and tuna) will be instanced inside + aquariumcube. Resizing will be happen accordingly to its placeholder object aquariumcube, see chapter Scaling. - - - - -van Kammen Expires 14 April 2024 [Page 10] - -Internet-Draft XR Fragments October 2023 - - - | Instead of cherrypicking objects with #bass&tuna thru src, queries - | can be used to import the whole scene (and filter out certain - | objects). See next chapter below. + | Instead of cherrypicking a rootobject #fishbowl with src, + | additional filters can be used to include/exclude certain objects. + | See next chapter on filtering below. *Specification*: - 1. local/remote content is instanced by the src (query) value (and + 1. local/remote content is instanced by the src (filter) value (and attaches it to the placeholder mesh containing the src property) - 2. local src values (URL *starting* with #, like #cube&foo) - means *only* the mentioned objectnames will be copied to the - instanced scene (from the current scene) while preserving their - names (to support recursive selectors). (example code) (https:// - github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/ - three/xrf/src.js) - 3. local src values indicating a query (#q=), means that all - included objects (from the current scene) will be copied to the - instanced scene (before applying the query) while preserving - their names (to support recursive selectors). (example code) (ht - tps://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/j - s/three/xrf/src.js) - 4. the instanced scene (from a src value) should be scaled + 2. by default all objects are loaded into the instanced src (scene) + object (but not shown yet) + 3. local src values (#... e.g.) starting with a non-negating + filter (#cube e.g.) will (deep)reparent that object (with name + cube) as the new root of the scene at position 0,0,0 + 4. local src values should respect (negative) filters + (#-foo&price=>3) + 5. the instanced scene (from a src value) should be scaled accordingly to its placeholder object or scaled relatively based on the scale-property (of a geometry-less placeholder, an 'empty'-object in blender e.g.). For more info see Chapter Scaling. - 5. external src values should be served with appropriate + + + + +van Kammen Expires 8 June 2024 [Page 12] + +Internet-Draft XR Fragments December 2023 + + + 6. external src values should be served with appropriate mimetype (so the XR Fragment-compatible browser will now how to render it). The bare minimum supported mimetypes are: - 6. src values should make its placeholder object invisible, and + 7. src values should make its placeholder object invisible, and only flush its children when the resolved content can succesfully be retrieved (see broken links (#links)) - 7. external src values should respect the fallback link + 8. external src values should respect the fallback link mechanism (see broken links (#broken-links) - 8. when the placeholder object is a 2D plane, but the mimetype is + 9. when the placeholder object is a 2D plane, but the mimetype is 3D, then render the spatial content on that plane via a stencil buffer. - 9. src-values are non-recursive: when linking to an external object + 10. src-values are non-recursive: when linking to an external object (src: foo.fbx#bar), then src-metadata on object bar should be ignored. - 10. clicking on external src-values always allow sourceportation: - teleporting to the origin URI to which the object belongs. - 11. when only one object was cherrypicked (#cube e.g.), set its + 11. an external src-value should always allow a sourceportation icon + within 3 meter: teleporting to the origin URI to which the + object belongs. + 12. when only one object was cherrypicked (#cube e.g.), set its position to 0,0,0 + 13. when the enduser clicks an href with #t=1,0,0 (play) will be + applied to all src mediacontent with a timeline (mp4/mp3 e.g.) + 14. a non-euclidian portal can be rendered for flat 3D objects + (using stencil buffer e.g.) in case ofspatial src-values (an + object #world3 or URL world3.fbx e.g.). + * model/gltf-binary * model/gltf+json * image/png * image/jpg - * text/plain;charset=utf-8;bib=^@ + * text/plain;charset=utf-8 - - -van Kammen Expires 14 April 2024 [Page 11] - -Internet-Draft XR Fragments October 2023 - - - » example implementation + » example implementation (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/ three/xrf/src.js) - » example 3D asset + » example 3D asset (https://github.com/coderofsalvation/xrfragment/blob/main/example/ assets/src.gltf#L192) - » discussion (https://github.com/coderofsalvation/xrfragment/ + » discussion (https://github.com/coderofsalvation/xrfragment/ issues/4) -10. Navigating content (internal/outbound href portals) +9. Navigating content href portals navigation, portals & mutations + + + + + + + +van Kammen Expires 8 June 2024 [Page 13] + +Internet-Draft XR Fragments December 2023 + + +==========+==================+============================+ | fragment | type | example value | +==========+==================+============================+ @@ -639,7 +738,7 @@ Internet-Draft XR Fragments October 2023 | | | ://somefile.gltf#pos=1,1,0 | +----------+------------------+----------------------------+ - Table 7 + Table 9 1. clicking an outbound ''external''- or ''file URI'' fully replaces the current scene and assumes pos=0,0,0&rot=0,0,0 by default @@ -662,37 +761,52 @@ Internet-Draft XR Fragments October 2023 (https://github.com/coderofsalvation/xrfragment/blob/dev/example/ aframe/sandbox/index.html#L26-L29) for an example wearable) - - - - - - - -van Kammen Expires 14 April 2024 [Page 12] - -Internet-Draft XR Fragments October 2023 - - 6. in case of navigating to a new [[pos)ition, ''first'' navigate to the ''current position'' so that the ''back-button'' of the ''browser-history'' always refers to the previous position (see [here (https://github.com/coderofsalvation/xrfragment/blob/main/s rc/3rd/js/three/xrf/href.js#L97)) - 7. portal-rendering: a 2:1 ratio texture-material indicates an - equirectangular projection + 7. ignore previous rule in special cases, like clicking an href + using camera-portal collision (the back-button would cause a + teleport-loop) - » example implementation + + + + + + + + + + +van Kammen Expires 8 June 2024 [Page 14] + +Internet-Draft XR Fragments December 2023 + + + » example implementation (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/ three/xrf/href.js) - » example 3D asset + » example 3D asset (https://github.com/coderofsalvation/xrfragment/blob/main/example/ assets/href.gltf#L192) - » discussion (https://github.com/coderofsalvation/xrfragment/ + » discussion (https://github.com/coderofsalvation/xrfragment/ issues/1) -10.1. UX spec +9.1. Walking surfaces + + XR Fragment-compatible viewers can infer this data based scanning the + scene for: + + 1. materialless (nameless & textureless) mesh-objects (without src + and href) + + | optionally the viewer can offer thumbstick, mouse or joystick + | teleport-tools for non-roomscale VR/AR setups. + +9.2. UX spec End-users should always have read/write access to: @@ -702,7 +816,7 @@ Internet-Draft XR Fragments October 2023 another scene/file (and coordinate e.g. in case the URL contains XR Fragments). -10.2. Scaling instanced content +9.3. Scaling instanced content Sometimes embedded properties (like src) instance new objects. But what about their scale? @@ -720,16 +834,17 @@ Internet-Draft XR Fragments October 2023 (maxsize=1.4 e.g.) * hide the ''placeholder'' object (material e.g.) * instance the src scene as a child of the existing object + + + +van Kammen Expires 8 June 2024 [Page 15] + +Internet-Draft XR Fragments December 2023 + + * calculate the bounding box of the instanced scene, and scale it accordingly (to 1.4 e.g.) - - -van Kammen Expires 14 April 2024 [Page 13] - -Internet-Draft XR Fragments October 2023 - - | REASON: non-empty placeholder object can act as a protective | bounding-box (for remote content of which might grow over time | e.g.) @@ -740,120 +855,181 @@ Internet-Draft XR Fragments October 2023 | TODO: needs intermediate visuals to make things more obvious -11. XR Fragment queries +10. XR Fragment: pos + +11. XR Fragment: rot + +12. XR Fragment: t + + controls the animation(s) of the scene (or src resource which + contains a timeline) + + | fragment | type | functionality | | #t=1,1,100 | + [vector3|vector] (default:`#t=1,0,0`) | speed,framestart,framestop | + + * playposition is reset to framestart, when framestart or framestop + is greater than 0 | + + | Example Value | Explanation | |-|-| | 1,1,100 | play loop between + frame 1 and 100 | | 1,1,0 | play once from frame 1 (oneshot) | | + 1,0,0 | play (previously set looprange if any) | | 0,0,0 | pause | | + 1,1,1 | play and auto-loop between begin and end of duration | | + -1,0,0 | reverse playback speed | | 2.3,0,0 | set (forward) playback + speed to 2.3 (no restart) | | -2.3,0,0 | set (reverse) playback speed + to -2.3 ( no restart)| | -2.3,100,0 | set (reverse) playback speed to + -2.3 restarting from frame 100 | + + [[» example implementation|https://github.com/coderofsalvation/xrfrag + ment/blob/main/src/3rd/js/three/xrf/t.js] + (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/ + three/xrf/t.js])] + [[» discussion|https://github.com/coderofsalvation/xrfragment/ + issues/10] (https://github.com/coderofsalvation/xrfragment/ + issues/10])] + + + + + + + +van Kammen Expires 8 June 2024 [Page 16] + +Internet-Draft XR Fragments December 2023 + + +13. XR audio/video integration + + To play global audio/video items: + + 1. add a src: foo.mp3 or src: bar.mp4 metadata to a 3D object (cube + e.g.) + 2. to disable auto-play and global timeline ([[#t=|t]]) control: + hardcode a [[#t=|t]] XR Fragment: (src: bar.mp3#t=0,0,0 e.g.) + 3. to play it, add href: #cube somewhere else + 4. when the enduser clicks the href, #t=1,0,0 (play) will be applied + to the src value + 5. to play a single animation, add href: #animationname=1,0,0 + somewhere else + + | NOTE: hardcoded framestart/framestop uses sampleRate/fps of + | embedded audio/video, otherwise the global fps applies. For more + | info see [[#t|t]]. + +14. XR Fragment filters Include, exclude, hide/shows objects using space-separated strings: - +==================+==========================================+ - | example | outcome | - +==================+==========================================+ - | #q=-sky | show everything except object named sky | - +------------------+------------------------------------------+ - | #q=-tag:language | hide everything with tag language, but | - | tag:english | show all tag english objects | - +------------------+------------------------------------------+ - | #q=price:>2 | of all objects with property price, show | - | price:<5 | only objects with value between 2 and 5 | - +------------------+------------------------------------------+ + +====================+===========================================+ + | example | outcome | + +====================+===========================================+ + | #-sky | show everything except object named sky | + +--------------------+-------------------------------------------+ + | #-language&english | hide everything with tag language, but | + | | show all tag english objects | + +--------------------+-------------------------------------------+ + | #-price&price=>10 | hide all objects with property price, | + | | then only show object with price above 10 | + +--------------------+-------------------------------------------+ - Table 8 + Table 10 It's simple but powerful syntax which allows filtering the scene using searchengine prompt-style feeling: - 1. queries are a way to traverse a scene, and filter objects based - on their tag- or property-values. - 2. words like german match tag-metadata of 3D objects like - "tag":"german" - 3. words like german match (XR Text) objects with (Bib(s)TeX) tags - like #KarlHeinz@german or @german{KarlHeinz, ... e.g. + 1. filters are a way to traverse a scene, and filter objects based + on their name, tag- or property-values. * see an (outdated) example video here (https://coderofsalvation.github.io/xrfragment.media/queries.mp4) + which used a dedicated q= variable (now deprecated and usable + directly) - - - - - - -van Kammen Expires 14 April 2024 [Page 14] +van Kammen Expires 8 June 2024 [Page 17] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 -11.1. including/excluding +14.1. including/excluding - +==========+=================================================+ - | operator | info | - +==========+=================================================+ - | - | removes/hides object(s) | - +----------+-------------------------------------------------+ - | : | indicates an object-embedded custom property | - | | key/value | - +----------+-------------------------------------------------+ - | > < | compare float or int number | - +----------+-------------------------------------------------+ - | / | reference to root-scene. | - | | Useful in case of (preventing) showing/hiding | - | | objects in nested scenes (instanced by src) (*) | - +----------+-------------------------------------------------+ + By default, selectors work like photoshop-layers: they scan for + matching layer(name/properties) within the scene-graph. Each matched + object (not their children) will be toggled (in)visible when + selecting. - Table 9 + +==========+==============================================+ + | operator | info | + +==========+==============================================+ + | - | hides object(s) (#-myobject&-objects e.g. | + +----------+----------------------------------------------+ + | = | indicates an object-embedded custom property | + | | key/value (#price=4&category=foo e.g.) | + +----------+----------------------------------------------+ + | => =< | compare float or int number (#price=>4 e.g.) | + +----------+----------------------------------------------+ + | * | deepselect: automatically select children of | + | | selected object, including local (nonremote) | + | | embedded objects (starting with #) | + +----------+----------------------------------------------+ - | * = #q=-/cube hides object cube only in the root-scene (not nested - | cube objects) - | #q=-cube hides both object cube in the root-scene AND - | nested skybox objects | + Table 11 - » example implementation + | NOTE 1: after an external embedded object has been instanced (src: + | https://y.com/bar.fbx#room e.g.), filters do not affect them + | anymore (reason: local tag/name collisions can be mitigated + | easily, but not in case of remote content). + | + | NOTE 2: depending on the used 3D framework, toggling objects + | (in)visible should happen by enabling/disableing writing to the + | colorbuffer (to allow children being still visible while their + | parents are invisible). + + » example implementation (https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/ - three/xrf/q.js) » example 3D asset + three/xrf/q.js) » example 3D asset (https://github.com/coderofsalvation/xrfragment/blob/main/example/ - assets/query.gltf#L192) » discussion + assets/filter.gltf#L192) » discussion (https://github.com/coderofsalvation/xrfragment/issues/3) -11.2. Query Parser +14.2. Filter Parser - Here's how to write a query parser: + Here's how to write a filter parser: - 1. create an associative array/object to store query-arguments as + 1. create an associative array/object to store filter-arguments as objects - 2. detect object id's & properties foo:1 and foo (reference regex: - /^.*:[><=!]?/ ) - 3. detect excluders like -foo,-foo:1,-.foo,-/foo (reference regex: - /^-/ ) - 4. detect root selectors like /foo (reference regex: /^[-]?\// ) - 5. detect number values like foo:1 (reference regex: /^[0-9\.]+$/ ) - 6. for every query token split string on : - 7. create an empty array rules - 8. then strip key-operator: convert "-foo" into "foo" - 9. add operator and value to rule-array - 10. therefore we we set id to true or false (false=excluder -) -van Kammen Expires 14 April 2024 [Page 15] + +van Kammen Expires 8 June 2024 [Page 18] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 - 11. and we set root to true or false (true=/ root selector is + 2. detect object id's & properties foo=1 and foo (reference regex= + ~/^.*=[><=]?/ ) + 3. detect excluders like -foo,-foo=1,-.foo,-/foo (reference regex= + /^-/ ) + 4. detect root selectors like /foo (reference regex= /^[-]?\// ) + 5. detect number values like foo=1 (reference regex= /^[0-9\.]+$/ ) + 6. detect operators so you can easily strip keys (reference regex= + /(^-|\*$)/ ) + 7. detect exclude keys like -foo (reference regex= /^-/ ) + 8. for every filter token split string on = + 9. and we set root to true or false (true=/ root selector is present) - 12. we convert key '/foo' into 'foo' - 13. finally we add the key/value to the store like store.foo = - {id:false,root:true} e.g. + 10. therefore we we set show to true or false (false=excluder -) - | An example query-parser (which compiles to many languages) can be + | An example filter-parser (which compiles to many languages) can be | found here | (https://github.com/coderofsalvation/xrfragment/blob/main/src/ - | xrfragment/Query.hx) + | xrfragment/Filter.hx) -12. Visible links +15. Visible links When predefined views, XRWG fragments and ID fragments (#cube or #mytag e.g.) are triggered by the enduser (via toplevel URL or @@ -878,10 +1054,18 @@ Internet-Draft XR Fragments October 2023 5. offer a control near the back/forward button which allows the user to (turn off) control the correlation-intensity of the XRWG -13. Text in XR (tagging,linking to spatial objects) +16. Text in XR (tagging,linking to spatial objects) How does XR Fragments interlink text with objects? + + + +van Kammen Expires 8 June 2024 [Page 19] + +Internet-Draft XR Fragments December 2023 + + | The XR Fragments does this by collapsing space into a *Word Graph* | (the *XRWG* example | (https://github.com/coderofsalvation/xrfragment/blob/feat/macros/ @@ -890,14 +1074,6 @@ Internet-Draft XR Fragments October 2023 Instead of just throwing together all kinds media types into one experience (games), what about their tagged/semantical relationships? Perhaps the following question is related: why is HTML adopted less - - - -van Kammen Expires 14 April 2024 [Page 16] - -Internet-Draft XR Fragments October 2023 - - in games outside the browser? Through the lens of constructive lazy game-developers, ideally metadata must come *with* text, but not *obfuscate* the text, or *spawning another request* to fetch it. @@ -939,21 +1115,15 @@ Internet-Draft XR Fragments October 2023 scroll-to-text-fragment (https://github.com/WICG/scroll-to-text- fragment) will be considered) - Example: - - - - - - - -van Kammen Expires 14 April 2024 [Page 17] +van Kammen Expires 8 June 2024 [Page 20] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 + Example: + http://y.io/z.fbx | Derived XRWG (expressed as BibTex) ----------------------------------------------------------------------------+-------------------------------------- | @house{castle, @@ -1003,11 +1173,9 @@ Internet-Draft XR Fragments October 2023 - - -van Kammen Expires 14 April 2024 [Page 18] +van Kammen Expires 8 June 2024 [Page 21] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 As seen above, the XRWG can expand bibs @@ -1029,7 +1197,7 @@ Internet-Draft XR Fragments October 2023 | foo.gltf#house | other objects with tag house or todo | +------------------+--------------------------------------+ - Table 10 + Table 12 | hashtagbibs (https://github.com/coderofsalvation/hashtagbibs) | potentially allow the enduser to annotate text/objects by @@ -1061,9 +1229,9 @@ Internet-Draft XR Fragments October 2023 -van Kammen Expires 14 April 2024 [Page 19] +van Kammen Expires 8 June 2024 [Page 22] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 12. Default font (unless specified otherwise) is a modern monospace @@ -1098,7 +1266,7 @@ Internet-Draft XR Fragments October 2023 interesting, I'm importing mytrainstory.txt into station.gltf ah yes, chapter three points to trainterminal_2A in the scene, cool -13.1. Default Data URI mimetype +16.1. Default Data URI mimetype The src-values work as expected (respecting mime-types), however: @@ -1117,9 +1285,9 @@ Internet-Draft XR Fragments October 2023 -van Kammen Expires 14 April 2024 [Page 20] +van Kammen Expires 8 June 2024 [Page 23] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 * lines beginning with @ will not be rendered verbatim by default @@ -1159,7 +1327,7 @@ Internet-Draft XR Fragments October 2023 For all other purposes, regular mimetypes can be used (but are not required by the spec). -13.2. URL and Data URI +16.2. URL and Data URI @@ -1173,9 +1341,9 @@ Internet-Draft XR Fragments October 2023 -van Kammen Expires 14 April 2024 [Page 21] +van Kammen Expires 8 June 2024 [Page 24] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 +--------------------------------------------------------------+ +------------------------+ @@ -1204,7 +1372,7 @@ Internet-Draft XR Fragments October 2023 | object note_canvas with 'todo', the enduser can type or speak | #note_canvas@todo -13.3. XR Text example parser +16.3. XR Text example parser To prime the XRWG with text from plain text src-values, here's an example XR Text (de)multiplexer in javascript (which supports inline @@ -1229,9 +1397,9 @@ xrtext = { -van Kammen Expires 14 April 2024 [Page 22] +van Kammen Expires 8 June 2024 [Page 25] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 // bibtex: ↓@ ↓ ↓property ↓end @@ -1285,9 +1453,9 @@ Internet-Draft XR Fragments October 2023 -van Kammen Expires 14 April 2024 [Page 23] +van Kammen Expires 8 June 2024 [Page 26] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 str = ` @@ -1330,7 +1498,7 @@ console.log( xrtext.encode(text,tags) ) // multiplex text & bibtex back to | nonmatching tags (@book{nonmatchingbook e.g.) should be performed | and prompt the enduser for deleting them. -14. Transclusion (broken link) resolution +17. Transclusion (broken link) resolution In spirit of Ted Nelson's 'transclusion resolution', there's a soft- mechanism to harden links & minimize broken links in various ways: @@ -1341,9 +1509,9 @@ console.log( xrtext.encode(text,tags) ) // multiplex text & bibtex back to -van Kammen Expires 14 April 2024 [Page 24] +van Kammen Expires 8 June 2024 [Page 27] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 2. mirroring files on another protocol using (HTTP) errorcode tags @@ -1362,12 +1530,12 @@ Internet-Draft XR Fragments October 2023 │ │ │ index.gltf │ │ │ │ - │ │ #: #q=-offlinetext │ + │ │ #: #-offlinetext │ │ │ │ │ ├── ◻ buttonA │ │ │ └ href: http://foo.io/campagne.fbx │ │ │ └ href@404: ipfs://foo.io/campagne.fbx │ - │ │ └ href@400: #q=clienterrortext │ + │ │ └ href@400: #clienterrortext │ │ │ └ ◻ offlinetext │ │ │ │ │ └── ◻ embeddedObject <--------- the meshdata inside embeddedObject will (not) @@ -1377,7 +1545,7 @@ Internet-Draft XR Fragments October 2023 │ │ +────────────────────────────────────────────────────────+ -15. Topic-based index-less Webrings +18. Topic-based index-less Webrings As hashtags in URLs map to the XWRG, href-values can be used to promote topic-based index-less webrings. @@ -1390,19 +1558,19 @@ Internet-Draft XR Fragments October 2023 These links would all show visible links to math-tagged objects in the scene. To filter out non-related objects one could take it a step further - using queries: + using filters: - * href: schoolA.edu/projects.gltf#math&q=-topics math - * href: schoolB.edu/projects.gltf#math&q=-courses math + * href: schoolA.edu/projects.gltf#math&-topics math + * href: schoolB.edu/projects.gltf#math&-courses math -van Kammen Expires 14 April 2024 [Page 25] +van Kammen Expires 8 June 2024 [Page 28] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 - * href: university.edu/projects.gltf#math&q=-theme math + * href: university.edu/projects.gltf#math&-theme math | This would hide all object tagged with topic, courses or theme | (including math) so that later only objects tagged with math will @@ -1412,7 +1580,7 @@ Internet-Draft XR Fragments October 2023 separate content into separate files, or show/hide things using a complex logiclayer like javascript. -16. Security Considerations +19. Security Considerations Since XR Text contains metadata too, the user should be able to set up tagging-rules, so the copy-paste feature can : @@ -1420,7 +1588,7 @@ Internet-Draft XR Fragments October 2023 * filter out sensitive data when copy/pasting (XR text with tag:secret e.g.) -17. FAQ +20. FAQ *Q:* Why is everything HTTP GET-based, what about POST/PUT/DELETE HATEOS @@ -1453,26 +1621,35 @@ Internet-Draft XR Fragments October 2023 -van Kammen Expires 14 April 2024 [Page 26] +van Kammen Expires 8 June 2024 [Page 29] -Internet-Draft XR Fragments October 2023 +Internet-Draft XR Fragments December 2023 Non-HTML Hypermedia browsers should make browser extensions the right place, to 'extend' experiences, in contrast to code/javascript inside hypermedia documents (this turned out as a hypermedia antipattern). -18. IANA Considerations +21. IANA Considerations This document has no IANA actions. -19. Acknowledgments +22. Acknowledgments * NLNET (https://nlnet.nl) * Future of Text (https://futureoftext.org) * visual-meta.info (https://visual-meta.info) + * Michiel Leenaars + * Gerben van der Broeke + * Mauve + * Jens Finkhäuser + * Marc Belmont + * Tim Gerritsen + * Frode Hegland + * Brandel Zackernuk + * Mark Anderson -20. Appendix: Definitions +23. Appendix: Definitions +=================+===============================================+ | definition | explanation | @@ -1497,6 +1674,14 @@ Internet-Draft XR Fragments October 2023 | the XRWG | wordgraph (collapses 3D scene to tags) | +-----------------+-----------------------------------------------+ | the hashbus | hashtags map to camera/scene-projections | + + + +van Kammen Expires 8 June 2024 [Page 30] + +Internet-Draft XR Fragments December 2023 + + +-----------------+-----------------------------------------------+ | spacetime | positions camera, triggers scene-preset/time | | hashtags | | @@ -1506,14 +1691,6 @@ Internet-Draft XR Fragments October 2023 +-----------------+-----------------------------------------------+ | sourceportation | teleporting the enduser to the original XR | | | Document of an src embedded object. | - - - -van Kammen Expires 14 April 2024 [Page 27] - -Internet-Draft XR Fragments October 2023 - - +-----------------+-----------------------------------------------+ | placeholder | a 3D object which with src-metadata (which | | object | will be replaced by the src-data.) | @@ -1524,8 +1701,9 @@ Internet-Draft XR Fragments October 2023 | href | (HTML-piggybacked) metadata of a 3D object | | | which links to content | +-----------------+-----------------------------------------------+ - | query | an URI Fragment-operator which queries | - | | object(s) from a scene like #q=cube | + | filter | URI Fragment(s) which show/hide object(s) in | + | | a scene based on name/tag/property (#cube&- | + | | price=>3) | +-----------------+-----------------------------------------------+ | visual-meta | visual-meta (https://visual.meta.info) data | | | appended to text/books/papers which is | @@ -1545,12 +1723,23 @@ Internet-Draft XR Fragments October 2023 | extrospective | outward sensemaking ("I'm fairly sure John is | | | a person who lives in oklahoma") | +-----------------+-----------------------------------------------+ - | ◻ | ascii representation of an 3D object/mesh | + | ◻ | ascii representation of an 3D object/mesh | +-----------------+-----------------------------------------------+ | (un)obtrusive | obtrusive: wrapping human text/thought in | | | XML/HTML/JSON obfuscates human text into a | | | salad of machine-symbols and words | +-----------------+-----------------------------------------------+ + | flat 3D object | a 3D object of which all verticies share a | + + + +van Kammen Expires 8 June 2024 [Page 31] + +Internet-Draft XR Fragments December 2023 + + + | | plane | + +-----------------+-----------------------------------------------+ | BibTeX | simple tagging/citing/referencing standard | | | for plaintext | +-----------------+-----------------------------------------------+ @@ -1561,8 +1750,43 @@ Internet-Draft XR Fragments October 2023 | | hashtagbibs) which expands to BibTex/JSON/XML | +-----------------+-----------------------------------------------+ - Table 11 + Table 13 -van Kammen Expires 14 April 2024 [Page 28] + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +van Kammen Expires 8 June 2024 [Page 32] diff --git a/doc/RFC_XR_Fragments.xml b/doc/RFC_XR_Fragments.xml index b654827..4f2e152 100644 --- a/doc/RFC_XR_Fragments.xml +++ b/doc/RFC_XR_Fragments.xml @@ -12,10 +12,9 @@ This draft is a specification for 4D URLs & hypermediatic navigation, which links together space, time & text together, for hypermedia browsers with- or without a network-connection.
    -The specification promotes spatial addressibility, sharing, navigation, query-ing and annotating interactive (text)objects across for (XR) Browsers.
    +The specification promotes spatial addressibility, sharing, navigation, filtering and databinding objects for (XR) Browsers.
    -XR Fragments allows us to enrich existing dataformats, by recursive use of existing proven technologies like URI Fragments and BibTags notation.
    -     +XR Fragments allows us to better use existing metadata inside 3D scene(files), by connecting it to proven technologies like URI Fragments. Almost every idea in this document is demonstrated at https://xrfragment.org     @@ -24,31 +23,29 @@ XR Fragments allows us to enrich existing dataformats, by recursive use of exist
    Introduction -How can we add more features to existing text & 3D scenes, without introducing new dataformats?
    +How can we add more control to existing text & 3D scenes, without introducing new dataformats?
    Historically, there's many attempts to create the ultimate markuplanguage or 3D fileformat.
    -The lowest common denominator is: describing/tagging/naming nodes using plain text.
    +The lowest common denominator is: designers describing/tagging/naming things using plain text.
    -XR Fragments allows us to enrich/connect existing dataformats, by introducing existing technologies/ideas:
    -     +XR Fragments exploits the fact that all 3D models already contain such metadata:     +XR Fragments allows controlling of metadata in 3D scene(files) using URLs +Or more detailed:
    1. addressibility and hypermediatic navigation of 3D scenes/objects: URI Fragments + src/href spatial metadata
      2. -
    2. Interlinking text/& 3D by collapsing space into a Word Graph (XRWG) to show visible links (and augmenting text with bibs / BibTags appendices (see visual-meta e.g.)
      3. +
    3. Interlinking (text)objects by collapsing space into a Word Graph (XRWG) to show visible links
    4. unlocking spatial potential of the (originally 2D) hashtag (which jumps to a chapter) for navigating XR documents
    NOTE: The chapters in this document are ordered from highlevel to lowlevel (technical) as much as possible
    Core principle -XR Fragments strives to serve (nontechnical/fuzzy) humans first, and machine(implementations) later, by ensuring hasslefree text-vs-thought feedback loops.
    +XR Fragments allows controlling of metadata in 3D scene(files) using URLs +XR Fragments tries to seek to connect the world of text (semantical web / RDF), and the world of pixels.
    -This also means that the repair-ability of machine-matters should be human friendly too (not too complex).
    - -XR Fragments tries to seek to connect the world of text (semantical web / RDF), and the world of pixels.
    - -Instead of combining them (in a game-editor e.g.), XR Fragments is opting for a more integrated path towards them, by describing how to make browsers 4D URL-ready:     +Instead of combining them (in a game-editor e.g.), XR Fragments integrates all, by collecting metadata into an XRWG and control it via URL:     @@ -67,14 +64,8 @@ Instead of combining them (in a game-editor e.g.), XR Fragments is opting for a - - - - - - - - + + @@ -91,43 +82,53 @@ Instead of combining them (in a game-editor e.g.), XR Fragments is opting for a - - + + + + + + + + - + - - + +
    the hashbushashtags map to camera/scene-projectionshashtags map to document positions
    spacetime hashtagspositions camera, triggers scene-preset/timejumps/scrolls to chapterhashtags alter camera/scene/object-projectionshashtags alter document positions
    href metadatarepositions camera or animation-rangejumps to cameratriggers predefined viewMedia fragments
    href metadatatriggers camera/scene/object/projectionsn/a
    href metadata draws visible connection(s) for XRWG 'tag'n/a
    href metadatatriggers predefined viewMedia fragmentsfilters certain (in)visible objectsn/a
    XR Fragments does not look at XR (or the web) thru the lens of HTML.
    But approaches things from a higherlevel feedbackloop/hypermedia browser-perspective:
    - +──────────────────────────────────────────────────────────────────────────────────────────────+ + ?misc ──> #view ───> hashbus │ + │ │ #filter │ │ │ │ #tag │ │ + │ │ #material │ │ + │ │ #animation │ │ + │ │ #texture │ │ + │ │ #variable │ │ │ │ │ │ - │ XRWG <─────────────────────<────────────+ │ + │ XRWG <─────────────────────<────────────+ │ │ │ │ │ - │ ├─ objects ───────────────>────────────│ │ - │ └─ text ───────────────>────────────+ │ + │ └─ objects ──────────────>────────────+ │ │ │ │ │ +──────────────────────────────────────────────────────────────────────────────────────────────+ +]]> Traditional webbrowsers can become 4D document-ready by: @@ -144,9 +145,10 @@ But approaches things from a higherlevel feedbackloop/hypermedia browser-perspec
    XR Fragment URI Grammar -reserved = gen-delims / sub-delims -gen-delims = "#" / "&" -sub-delims = "," / "=" +
    Example: ://foo.com/my3d.gltf#pos=1,0,0&prio=-5&t=0,100
    @@ -164,7 +166,7 @@ sub-delims = "," / "=" - + @@ -200,21 +202,12 @@ sub-delims = "," / "=" - - - - - - - - - - + + + -
    pos=1,2,3&rot=0,90,0&q=.foopos=1,2,3&rot=0,90,0&foo combinators
    #tvector3#t=1,500,1000play animation-loop range between frame 500 and 1000, at normal speed
    #......string#.cubes #cubepredefined views, XRWG fragments and ID fragmentstimevector#t=2,2000,1play animation-loop range between frame 2 and 2000 at (normal) speed 1
    xyz coordinates are similar to ones found in SVG Media Fragments -
    - +
    List of metadata for 3D nodes @@ -248,18 +241,144 @@ sub-delims = "," / "=" - + -
    tag string "tag": "cubes geo"tag object (for query-use / XRWG highlighting)tag object (for filter-use / XRWG highlighting) custom property in 3D fileformats
    Supported popular compatible 3D fileformats: .gltf, .obj, .fbx, .usdz, .json (THREE.js), .dae and so on. -
    NOTE: XR Fragments are optional but also file- and protocol-agnostic, which means that programmatic 3D scene(nodes) can also use the mechanism/metadata. +
    Supported popular compatible 3D fileformats: .gltf, .obj, .fbx, .usdz, .json (THREE.js), .dae and so on.
    +
    vector datatypes +| type | syntax | example | info | + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    vector2
    vector3
    timevector
    NOTE: XR Fragments are optional but also file- and protocol-agnostic, which means that programmatic 3D scene(nodes) can also use the mechanism/metadata. +
    + +
    Dynamic XR Fragments (+databindings) +These are automatic fragment-to-metadata mappings, which only trigger if the 3D scene metadata matches a specific identifier (aliasname e.g.) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    fragmenttypeexampleinfo
    #<aliasname>string#cubesevaluate predefined views (#cubes: #foo&bar e.g.)
    #<tag_or_objectname>string#personfocus object(s) with tag: person or name person by looking up XRWG
    #<cameraname>string#cam01set camera as active camera
    #<objectname>=<material>string=string#car=metallicset material of car to material with name metallic
    string=string#product=metallicset material of objects tagged with product to material with name metallic
    #<objectname>=<mediafrag>string=media frag#foo=0,1play media src using media fragment URI
    #<objectname>=<timevector>string=timevector#sky=0,0.5,0.1,0sets 1D/2D/3D time(line) vectors (uv-position e.g.) to 0,0.5 (and autoscroll x with max 0.1 every second)
    #music=1,2play media of object (src: podcast.mp3 e.g.) from beginning (1) at double speed (2)
    +
    +
    Spatial Referencing 3D XR Fragments assume the following objectname-to-URIFragment mapping: - +
    Every 3D fileformat supports named 3D object, and this name allows URLs (fragments) to reference them (and their children objects).
    Clever nested design of 3D scenes allow great ways for re-using content, and/or previewing scenes.
    @@ -301,8 +421,8 @@ For example, to render a portal with a preview-version of the scene, create an 3 <b>#pos</b>=0,0,0 -vector3 -(re)position camera +vector3 or string +(re)position camera based on coordinates directly, or indirectly using objectname (its worldposition) @@ -327,22 +447,23 @@ For example, to render a portal with a preview-version of the scene, create an 3
  • set the position of the camera accordingly to the vector3 values of #pos
  • rot sets the rotation of the camera (only for non-VR/AR headsets)
  • t sets the playbackspeed and animation-range of the current scene animation(s) or src-mediacontent (video/audioframes e.g., use t=0,7,7 to 'STOP' at frame 7 e.g.)
    • -
  • in case an href does not mention any pos-coordinate, pos=0,0,0 will be assumed
    • +
  • after scene load: in case an href does not mention any pos-coordinate, pos=0,0,0 will be assumed
    • Here's an ascii representation of a 3D scene-graph which contains 3D objects ◻ and their metadata: - +────────────────────────────────────────────────────────+ + An XR Fragment-compatible browser viewing this scene, allows the end-user to interact with the buttonA and buttonB.
    @@ -362,7 +483,7 @@ In case of buttonA the end-user will be teleported to another location
    -
    Embedding XR content (src-instancing) +
    Embedding XR content using src src is the 3D version of the <a target="_blank" href="https://www.w3.org/html/wiki/Elements/iframe">iframe</a>.
    It instances content (in objects) in the current scene/asset.     @@ -378,26 +499,27 @@ It instances content (in objects) in the current scene/asset. src -string (uri, hashtag/query) +string (uri, hashtag/filter) #cube
    #sometag
    -#q=-ball_inside_cube<br>#q=-/sky -rain<br>#q=-.language .english<br>#q=price:>2 price:<5`<br>https://linux.org/penguin.png
    +#cube&-ball_inside_cube<br>#-sky&-rain<br>#-language&english<br>#price=>5<br>https://linux.org/penguin.png`
    https://linux.world/distrowatch.gltf#t=1,100
    -linuxapp://conference/nixworkshop/apply.gltf#q=flyer
    -androidapp://page1?tutorial#pos=0,0,1&t1,100 +linuxapp://conference/nixworkshop/apply.gltf#-cta&cta_apply
    +androidapp://page1?tutorial#pos=0,0,1&t1,100
    +foo.mp3#0,0,0 -Here's an ascii representation of a 3D scene-graph with 3D objects ◻ which embeds remote & local 3D objects ◻ with/out using queries: +Here's an ascii representation of a 3D scene-graph with 3D objects ◻ which embeds remote & local 3D objects ◻ with/out using filters: - +────────────────────────────────────────────────────────+ +─────────────────────────+ + │ └ src: #canvas │ │ │ +────────────────────────────────────────────────────────+ +]]> An XR Fragment-compatible browser viewing this scene, lazy-loads and projects painting.png onto the (plane) object called canvas (which is copy-instanced in the bed and livingroom).
    -Also, after lazy-loading ocean.com/aquarium.gltf, only the queried objects bass and tuna will be instanced inside aquariumcube.
    +Also, after lazy-loading ocean.com/aquarium.gltf, only the queried objects fishbowl (and bass and tuna) will be instanced inside aquariumcube.
    Resizing will be happen accordingly to its placeholder object aquariumcube, see chapter Scaling.
     -
    Instead of cherrypicking objects with #bass&tuna thru src, queries can be used to import the whole scene (and filter out certain objects). See next chapter below. +
    Instead of cherrypicking a rootobject #fishbowl with src, additional filters can be used to include/exclude certain objects. See next chapter on filtering below.
    Specification:
      -
    1. local/remote content is instanced by the src (query) value (and attaches it to the placeholder mesh containing the src property)
      2. -
    2. <b>local</b> src values (URL starting with #, like #cube&foo) means only the mentioned objectnames will be copied to the instanced scene (from the current scene) while preserving their names (to support recursive selectors). (example code)
      3. -
    3. <b>local</b> src values indicating a query (#q=), means that all included objects (from the current scene) will be copied to the instanced scene (before applying the query) while preserving their names (to support recursive selectors). (example code)
      4. +
    4. local/remote content is instanced by the src (filter) value (and attaches it to the placeholder mesh containing the src property)
      5. +
    5. by default all objects are loaded into the instanced src (scene) object (but not shown yet)
      6. +
    6. <b>local</b> src values (#... e.g.) starting with a non-negating filter (#cube e.g.) will (deep)reparent that object (with name cube) as the new root of the scene at position 0,0,0
      7. +
    7. <b>local</b> src values should respect (negative) filters (#-foo&price=>3)
    8. the instanced scene (from a src value) should be <b>scaled accordingly</b> to its placeholder object or <b>scaled relatively</b> based on the scale-property (of a geometry-less placeholder, an 'empty'-object in blender e.g.). For more info see Chapter Scaling.
    9. <b>external</b> src values should be served with appropriate mimetype (so the XR Fragment-compatible browser will now how to render it). The bare minimum supported mimetypes are:
    10. src values should make its placeholder object invisible, and only flush its children when the resolved content can succesfully be retrieved (see broken links)
    11. <b>external</b> src values should respect the fallback link mechanism (see broken links
    12. when the placeholder object is a 2D plane, but the mimetype is 3D, then render the spatial content on that plane via a stencil buffer.
    13. src-values are non-recursive: when linking to an external object (src: foo.fbx#bar), then src-metadata on object bar should be ignored.
      14. -
    14. clicking on external src-values always allow sourceportation: teleporting to the origin URI to which the object belongs.
      15. +
    15. an external src-value should always allow a sourceportation icon within 3 meter: teleporting to the origin URI to which the object belongs.
    16. when only one object was cherrypicked (#cube e.g.), set its position to 0,0,0
      17. +
    17. when the enduser clicks an href with #t=1,0,0 (play) will be applied to all src mediacontent with a timeline (mp4/mp3 e.g.)
      18. +
    18. a non-euclidian portal can be rendered for flat 3D objects (using stencil buffer e.g.) in case ofspatial src-values (an object #world3 or URL world3.fbx e.g.).
      +
    • model/gltf-binary
    • model/gltf+json
    • image/png
    • image/jpg
      • -
    • text/plain;charset=utf-8;bib=^@
      • +
    • text/plain;charset=utf-8
    » example implementation
    @@ -444,7 +571,7 @@ Resizing will be happen accordingly to its placeholder object aquariumcube
    -
    Navigating content (internal/outbound href portals) +
    Navigating content href portals navigation, portals & mutations @@ -479,7 +606,7 @@ Resizing will be happen accordingly to its placeholder object aquariumcube
  • in case of navigating to a new [[pos)ition, ''first'' navigate to the ''current position'' so that the ''back-button'' of the ''browser-history'' always refers to the previous position (see [here)
    • -
  • portal-rendering: a 2:1 ratio texture-material indicates an equirectangular projection +
  • ignore previous rule in special cases, like clicking an href using camera-portal collision (the back-button would cause a teleport-loop)
    • » example implementation
    @@ -489,6 +616,15 @@ Resizing will be happen accordingly to its placeholder object aquariumcube» discussion
     +
    Walking surfaces +XR Fragment-compatible viewers can infer this data based scanning the scene for: + +
      +
    1. materialless (nameless & textureless) mesh-objects (without src and href)
      2. +
    +
    optionally the viewer can offer thumbstick, mouse or joystick teleport-tools for non-roomscale VR/AR setups. +
    +
    UX spec End-users should always have read/write access to: @@ -527,7 +663,51 @@ How does the scale of the object (with the embedded properties) impact the scale
    -
    XR Fragment queries +
    XR Fragment: pos +
    + +
    XR Fragment: rot +
    + +
    XR Fragment: t +controls the animation(s) of the scene (or src resource which contains a timeline) +| fragment | type | functionality | +| <b>#t</b>=1,1,100 | [vector3|vector] | speed,framestart,framestop | + +
      +
    • playposition is reset to framestart, when framestart or framestop is greater than 0 |
      • +
    +| Example Value | Explanation | +|-|-| +| 1,1,100 | play loop between frame 1 and 100 | +| 1,1,0 | play once from frame 1 (oneshot) | +| 1,0,0 | play (previously set looprange if any) | +| 0,0,0 | pause | +| 1,1,1 | play and auto-loop between begin and end of duration | +| -1,0,0 | reverse playback speed | +| 2.3,0,0 | set (forward) playback speed to 2.3 (no restart) | +| -2.3,0,0 | set (reverse) playback speed to -2.3 ( no restart)| +| -2.3,100,0 | set (reverse) playback speed to -2.3 restarting from frame 100 | +[[» example implementation|https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/xrf/t.js]]
    + +[[» discussion|https://github.com/coderofsalvation/xrfragment/issues/10]]
    +     +
    + +
    XR audio/video integration +To play global audio/video items: + +
      +
    1. add a src: foo.mp3 or src: bar.mp4 metadata to a 3D object (cube e.g.)
      2. +
    2. to disable auto-play and global timeline ([[#t=|t]]) control: hardcode a [[#t=|t]] XR Fragment: (src: bar.mp3#t=0,0,0 e.g.)
      3. +
    3. to play it, add href: #cube somewhere else
      4. +
    4. when the enduser clicks the href, #t=1,0,0 (play) will be applied to the src value
      5. +
    5. to play a single animation, add href: #animationname=1,0,0 somewhere else
      6. +
    +
    NOTE: hardcoded framestart/framestop uses sampleRate/fps of embedded audio/video, otherwise the global fps applies. For more info see [[#t|t]]. +
    + +
    XR Fragment filters Include, exclude, hide/shows objects using space-separated strings:
    @@ -539,33 +719,33 @@ How does the scale of the object (with the embedded properties) impact the scale - + - + - - + +
    #q=-sky#-sky show everything except object named sky
    #q=-tag:language tag:english#-language&english hide everything with tag language, but show all tag english objects
    #q=price:>2 price:<5of all objects with property price, show only objects with value between 2 and 5#-price&price=>10hide all objects with property price, then only show object with price above 10
    It's simple but powerful syntax which allows filtering the scene using searchengine prompt-style feeling:
      -
    1. queries are a way to traverse a scene, and filter objects based on their tag- or property-values.
      2. -
    2. words like german match tag-metadata of 3D objects like "tag":"german"
      3. -
    3. words like german match (XR Text) objects with (Bib(s)TeX) tags like #KarlHeinz@german or @german{KarlHeinz, ... e.g.
      4. +
    4. filters are a way to traverse a scene, and filter objects based on their name, tag- or property-values.
      -
    • see an (outdated) example video here
      • +
    • see an (outdated) example video here which used a dedicated q= variable (now deprecated and usable directly)
    including/excluding +By default, selectors work like photoshop-layers: they scan for matching layer(name/properties) within the scene-graph. +Each matched object (not their children) will be toggled (in)visible when selecting. @@ -577,51 +757,47 @@ How does the scale of the object (with the embedded properties) impact the scale - + - - + + - - + + - - + + -
    -removes/hides object(s)hides object(s) (#-myobject&-objects e.g.
    :indicates an object-embedded custom property key/value=indicates an object-embedded custom property key/value (#price=4&category=foo e.g.)
    > <compare float or int number=> =<compare float or int number (#price=>4 e.g.)
    /reference to root-scene.
    -Useful in case of (preventing) showing/hiding objects in nested scenes (instanced by src) (*)    		*deepselect: automatically select children of selected object, including local (nonremote) embedded objects (starting with #)
    * = #q=-/cube hides object cube only in the root-scene (not nested cube objects)
    - #q=-cube hides both object cube in the root-scene <b>AND</b> nested skybox objects |     +
    NOTE 1: after an external embedded object has been instanced (src: https://y.com/bar.fbx#room e.g.), filters do not affect them anymore (reason: local tag/name collisions can be mitigated easily, but not in case of remote content). +NOTE 2: depending on the used 3D framework, toggling objects (in)visible should happen by enabling/disableing writing to the colorbuffer (to allow children being still visible while their parents are invisible).
    » example implementation -» example 3D asset +» example 3D asset » discussion
    -
    Query Parser -Here's how to write a query parser: +
    Filter Parser +Here's how to write a filter parser:
      -
    1. create an associative array/object to store query-arguments as objects
      2. -
    2. detect object id's & properties foo:1 and foo (reference regex: /^.*:[><=!]?/ )
      3. -
    3. detect excluders like -foo,-foo:1,-.foo,-/foo (reference regex: /^-/ )
      4. -
    4. detect root selectors like /foo (reference regex: /^[-]?\// )
      5. -
    5. detect number values like foo:1 (reference regex: /^[0-9\.]+$/ )
      6. -
    6. for every query token split string on :
      7. -
    7. create an empty array rules
      8. -
    8. then strip key-operator: convert "-foo" into "foo"
      9. -
    9. add operator and value to rule-array
      10. -
    10. therefore we we set id to true or false (false=excluder -)
      11. +
    11. create an associative array/object to store filter-arguments as objects
      12. +
    12. detect object id's & properties foo=1 and foo (reference regex= ~/^.*=[><=]?/ )
      13. +
    13. detect excluders like -foo,-foo=1,-.foo,-/foo (reference regex= /^-/ )
      14. +
    14. detect root selectors like /foo (reference regex= /^[-]?\// )
      15. +
    15. detect number values like foo=1 (reference regex= /^[0-9\.]+$/ )
      16. +
    16. detect operators so you can easily strip keys (reference regex= /(^-|\*$)/ )
      17. +
    17. detect exclude keys like -foo (reference regex= /^-/ )
      18. +
    18. for every filter token split string on =
    19. and we set root to true or false (true=/ root selector is present)
      20. -
    20. we convert key '/foo' into 'foo'
      21. -
    21. finally we add the key/value to the store like store.foo = {id:false,root:true} e.g.
      22. +
    22. therefore we we set show to true or false (false=excluder -)
    -
    An example query-parser (which compiles to many languages) can be found here +
    An example filter-parser (which compiles to many languages) can be found here
    @@ -669,7 +845,7 @@ XR Fragments does this by detecting Bib(s)Tex, without introducing a new languag Example: - http://y.io/z.fbx | Derived XRWG (expressed as BibTex) +
    the #john@baroque-bib associates both text John and objectname john, with tag baroque
    Another example: - http://y.io/z.fbx | Derived XRWG (expressed as BibTex) +
    both #john@baroque-bib and BibTex @baroque{john} result in the same XRWG, however on top of that 2 tages (house and todo) are now associated with text/objectname/tag 'baroque'.
    As seen above, the XRWG can expand bibs (and the whole scene) to BibTeX.
    @@ -762,21 +940,22 @@ Some pointers for good UX (but not necessary to be XR Fragment compatible):
    The simplicity of appending metadata (and leveling the metadata-playfield between humans and machines) is also demonstrated by visual-meta in greater detail.
    Fictional chat: -<John> Hey what about this: https://my.com/station.gltf#pos=0,0,1&rot=90,2,0&t=500,1000 -<Sarah> I'm checking it right now -<Sarah> I don't see everything..where's our text from yesterday? -<John> Ah wait, that's tagged with tag 'draft' (and hidden)..hold on, try this: -<John> https://my.com/station.gltf#.draft&pos=0,0,1&rot=90,2,0&t=500,1000 -<Sarah> how about we link the draft to the upcoming YELLO-event? -<John> ok I'm adding #draft@YELLO -<Sarah> Yesterday I also came up with other usefull assocations between other texts in the scene: + Hey what about this: https://my.com/station.gltf#pos=0,0,1&rot=90,2,0&t=500,1000 + I'm checking it right now + I don't see everything..where's our text from yesterday? + Ah wait, that's tagged with tag 'draft' (and hidden)..hold on, try this: + https://my.com/station.gltf#.draft&pos=0,0,1&rot=90,2,0&t=500,1000 + how about we link the draft to the upcoming YELLO-event? + ok I'm adding #draft@YELLO + Yesterday I also came up with other usefull assocations between other texts in the scene: #event#YELLO #2025@YELLO -<John> thanks, added. -<Sarah> Btw. I stumbled upon this spatial book which references station.gltf in some chapters: -<Sarah> https://thecommunity.org/forum/foo/mytrainstory.txt -<John> interesting, I'm importing mytrainstory.txt into station.gltf -<John> ah yes, chapter three points to trainterminal_2A in the scene, cool + thanks, added. + Btw. I stumbled upon this spatial book which references station.gltf in some chapters: + https://thecommunity.org/forum/foo/mytrainstory.txt + interesting, I'm importing mytrainstory.txt into station.gltf + ah yes, chapter three points to trainterminal_2A in the scene, cool +]]>
    Default Data URI mimetype @@ -812,7 +991,7 @@ to connect text further with its environment ( setup links between textual/spati
    URL and Data URI - +--------------------------------------------------------------+ +------------------------+ + The enduser will only see welcome human and Hello friends rendered verbatim (see mimetype). The beauty is that text in Data URI automatically promotes rich copy-paste (retaining metadata). @@ -835,14 +1015,14 @@ The XR Fragment-compatible browser can let the enduser access visual-meta(data)-
    XR Text example parser To prime the XRWG with text from plain text src-values, here's an example XR Text (de)multiplexer in javascript (which supports inline bibs & bibtex): -xrtext = { + { let bibs = { regex: /(#[a-zA-Z0-9_+@\-]+(#)?)/g, tags: {}} - text.replace( bibs.regex , (m,k,v) => { - tok = m.substr(1).split("@") + text.replace( bibs.regex , (m,k,v) => { + tok = m.substr(1).split("@") match = tok.shift() - if( tok.length ) tok.map( (t) => bibs.tags[t] = `@${t}{${match},\n}` ) + if( tok.length ) tok.map( (t) => bibs.tags[t] = `@${t}{${match},\n}` ) else if( match.substr(-1) == '#' ) bibs.tags[match] = `@{${match.replace(/#/,'')}}` else bibs.tags[match] = `@${match}{${match},\n}` @@ -850,16 +1030,16 @@ The XR Fragment-compatible browser can let the enduser access visual-meta(data)- return text.replace( bibs.regex, '') + Object.values(bibs.tags).join('\n') }, - decode: (str) => { - // bibtex: ↓@ ↓<tag|tag{phrase,|{ruler}> ↓property ↓end + decode: (str) => { + // bibtex: ↓@ ↓ ↓property ↓end let pat = [ /@/, /^\S+[,{}]/, /},/, /}/ ] let tags = [], text='', i=0, prop='' let lines = xrtext.expandBibs(str).replace(/\r?\n/g,'\n').split(/\n/) - for( let i = 0; i < lines.length && !String(lines[i]).match( /^@/ ); i++ ) + for( let i = 0; i < lines.length && !String(lines[i]).match( /^@/ ); i++ ) text += lines[i]+'\n' bibtex = lines.join('\n').substr( text.length ) - bibtex.split( pat[0] ).map( (t) => { + bibtex.split( pat[0] ).map( (t) => { try{ let v = {} if( !(t = t.trim()) ) return @@ -868,9 +1048,9 @@ The XR Fragment-compatible browser can let the enduser access visual-meta(data)- if( tag.match( /}$/ ) ) return tags.push({k: tag.replace(/}$/,''), v: {}}) t = t.substr( tag.length ) t.split( pat[2] ) - .map( kv => { - if( !(kv = kv.trim()) || kv == "}" ) return - v[ kv.match(/\s?(\S+)\s?=/)[1] ] = kv.substr( kv.indexOf("{")+1 ) + .map( kv => { + if( !(kv = kv.trim()) || kv == "}" ) return + v[ kv.match(/\s?(\S+)\s?=/)[1] ] = kv.substr( kv.indexOf("{")+1 ) }) tags.push( { k:tag, v } ) }catch(e){ console.error(e) } @@ -878,8 +1058,8 @@ The XR Fragment-compatible browser can let the enduser access visual-meta(data)- return {text, tags} }, - encode: (text,tags) => { - let str = text+"\n" + encode: (text,tags) => { + let str = text+"\n" for( let i in tags ){ let item = tags[i] if( item.ruler ){ @@ -893,11 +1073,12 @@ The XR Fragment-compatible browser can let the enduser access visual-meta(data)- return str } } +]]> The above functions (de)multiplexe text/metadata, expands bibs, (de)serialize bibtex and vice versa
    above can be used as a startingpoint for LLVM's to translate/steelman to a more formal form/language.
    -str = ` + t.k == 'flap{' ).v.asdf = 1 // edit tag tags.push({ k:'bar{', v:{abc:123} }) // add tag -console.log( xrtext.encode(text,tags) ) // multiplex text & bibtex back together +console.log( xrtext.encode(text,tags) ) // multiplex text & bibtex back together +]]> This expands to the following (hidden by default) BibTex appendix: -hello world +
    when an XR browser updates the human text, a quick scan for nonmatching tags (@book{nonmatchingbook e.g.) should be performed and prompt the enduser for deleting them.
    @@ -948,25 +1131,26 @@ here are some hashtagbibs followed by bibtex:
    due to the popularity, maturity and extensiveness of HTTP codes for client/server communication, non-HTTP protocols easily map to HTTP codes (ipfs ERR_NOT_FOUND maps to 404 e.g.)
    For example: - +────────────────────────────────────────────────────────+ +
    @@ -982,12 +1166,12 @@ Consider 3D scenes linking to eachother using these href values: These links would all show visible links to math-tagged objects in the scene.
    -To filter out non-related objects one could take it a step further using queries:     +To filter out non-related objects one could take it a step further using filters:
      -
    • href: schoolA.edu/projects.gltf#math&q=-topics math
      • -
    • href: schoolB.edu/projects.gltf#math&q=-courses math
      • -
    • href: university.edu/projects.gltf#math&q=-theme math
      • +
    • href: schoolA.edu/projects.gltf#math&-topics math
      • +
    • href: schoolB.edu/projects.gltf#math&-courses math
      • +
    • href: university.edu/projects.gltf#math&-theme math
    This would hide all object tagged with topic, courses or theme (including math) so that later only objects tagged with math will be visible
    This makes spatial content multi-purpose, without the need to separate content into separate files, or show/hide things using a complex logiclayer like javascript. @@ -1028,6 +1212,15 @@ Non-HTML Hypermedia browsers should make browser extensions the right place, to
  • NLNET
  • Future of Text
  • visual-meta.info
    • +
  • Michiel Leenaars
    • +
  • Gerben van der Broeke
    • +
  • Mauve
    • +
  • Jens Finkhäuser
    • +
  • Marc Belmont
    • +
  • Tim Gerritsen
    • +
  • Frode Hegland
    • +
  • Brandel Zackernuk
    • +
  • Mark Anderson
    • @@ -1107,8 +1300,8 @@ Non-HTML Hypermedia browsers should make browser extensions the right place, to -query -an URI Fragment-operator which queries object(s) from a scene like #q=cube +filter +URI Fragment(s) which show/hide object(s) in a scene based on name/tag/property (#cube&-price=>3) @@ -1146,6 +1339,11 @@ Non-HTML Hypermedia browsers should make browser extensions the right place, to obtrusive: wrapping human text/thought in XML/HTML/JSON obfuscates human text into a salad of machine-symbols and words + +flat 3D object +a 3D object of which all verticies share a plane + + BibTeX simple tagging/citing/referencing standard for plaintext diff --git a/doc/RFC_XR_Macros.txt b/doc/RFC_XR_Macros.txt index 3523bc4..c5a2ebd 100644 --- a/doc/RFC_XR_Macros.txt +++ b/doc/RFC_XR_Macros.txt @@ -3,7 +3,7 @@ Internet Engineering Task Force L.R. van Kammen -Internet-Draft 12 October 2023 +Internet-Draft 6 December 2023 Intended status: Informational @@ -38,7 +38,7 @@ Status of This Memo time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on 14 April 2024. + This Internet-Draft will expire on 8 June 2024. Copyright Notice @@ -53,9 +53,9 @@ Copyright Notice -van Kammen Expires 14 April 2024 [Page 1] +van Kammen Expires 8 June 2024 [Page 1] -Internet-Draft XR Macros October 2023 +Internet-Draft XR Macros December 2023 extracted from this document must include Revised BSD License text as @@ -109,13 +109,13 @@ Table of Contents -van Kammen Expires 14 April 2024 [Page 2] +van Kammen Expires 8 June 2024 [Page 2] -Internet-Draft XR Macros October 2023 +Internet-Draft XR Macros December 2023 - 3. Metadata-values can contain the | symbol to 🎲 roundrobin - variable values (!toggleme: fog=0,10|fog=0,1000 e.g.) + 3. Metadata-values can contain the | symbol to 🎲 roundrobin variable + values (!toggleme: fog=0,10|fog=0,1000 e.g.) 4. XR Macros acts as simple eventhandlers for URI Fragments: they are automatically published on the (XR Fragments (https://xrfragment.org)) hashbus, to act as events (so more @@ -165,9 +165,9 @@ Internet-Draft XR Macros October 2023 -van Kammen Expires 14 April 2024 [Page 3] +van Kammen Expires 8 June 2024 [Page 3] -Internet-Draft XR Macros October 2023 +Internet-Draft XR Macros December 2023 +=========+======+===================+=================+=============+ @@ -221,9 +221,9 @@ Internet-Draft XR Macros October 2023 -van Kammen Expires 14 April 2024 [Page 4] +van Kammen Expires 8 June 2024 [Page 4] -Internet-Draft XR Macros October 2023 +Internet-Draft XR Macros December 2023 Table 3 @@ -277,9 +277,9 @@ Internet-Draft XR Macros October 2023 -van Kammen Expires 14 April 2024 [Page 5] +van Kammen Expires 8 June 2024 [Page 5] -Internet-Draft XR Macros October 2023 +Internet-Draft XR Macros December 2023 4.5. Usecase: present context menu with options @@ -333,9 +333,9 @@ click object with (`!clickme`:`!foo|!bar|!flop` e.g.) -van Kammen Expires 14 April 2024 [Page 6] +van Kammen Expires 8 June 2024 [Page 6] -Internet-Draft XR Macros October 2023 +Internet-Draft XR Macros December 2023 | Note that only macro's can trigger roundrobin values or @@ -375,7 +375,7 @@ Internet-Draft XR Macros October 2023 | | (games,VR,AR e.g.), should be as high as | | | possible | +---------------+---------------------------------------------------+ - | ◻ | ascii representation of an 3D object/mesh | + | ◻ | ascii representation of an 3D object/mesh | +---------------+---------------------------------------------------+ | (un)obtrusive | obtrusive: wrapping human text/thought in | | | XML/HTML/JSON obfuscates human text into | @@ -389,4 +389,4 @@ Internet-Draft XR Macros October 2023 -van Kammen Expires 14 April 2024 [Page 7] +van Kammen Expires 8 June 2024 [Page 7] diff --git a/doc/RFC_XR_Macros.xml b/doc/RFC_XR_Macros.xml index b08a4f2..0150e79 100644 --- a/doc/RFC_XR_Macros.xml +++ b/doc/RFC_XR_Macros.xml @@ -282,16 +282,17 @@ Macros also act as events, so more serious scripting languages can react to them
    Event Bubble-flow click object with (!clickme:AR or !clickme: !reset e.g.) - ◻ + click object with (!clickme:#AR|#VR e.g.) - ◻ + ◻ │