2353 lines
107 KiB
Plaintext
2353 lines
107 KiB
Plaintext
|
||
|
||
|
||
|
||
Jens & Leon Internet Engineering Task Force L.R. van Kammen
|
||
Internet-Draft 19 June 2024
|
||
Intended status: Informational
|
||
|
||
|
||
|
||
XR Fragments
|
||
draft-XRFRAGMENTS-leonvankammen-00
|
||
|
||
Abstract
|
||
|
||
This draft is a specification for 4D URI's & hypermediatic
|
||
(https://github.com/coderofsalvation/hypermediatic) navigation, to
|
||
enable a spatial web for hypermedia browsers with- or without a
|
||
network-connection.
|
||
The specification uses W3C Media Fragments (https://www.w3.org/TR/
|
||
media-frags/) and URI Templates (RFC6570) (https://www.rfc-
|
||
editor.org/rfc/rfc6570) to promote spatial addressibility, sharing,
|
||
navigation, filtering and databinding objects for (XR) Browsers.
|
||
XR Fragments allows us to better use existing metadata inside 3D
|
||
scene(files), by connecting it to proven technologies like URI
|
||
Fragments (https://en.wikipedia.org/wiki/URI_fragment).
|
||
XR Fragments views spatial webs thru the lens of 3D scene URI's,
|
||
rather than thru code(frameworks) or protocol-specific browsers
|
||
(webbrowser e.g.).
|
||
|
||
Almost every idea in this document is demonstrated at
|
||
https://xrfragment.org (https://xrfragment.org)
|
||
|
||
Status of This Memo
|
||
|
||
This Internet-Draft is submitted in full conformance with the
|
||
provisions of BCP 78 and BCP 79.
|
||
|
||
Internet-Drafts are working documents of the Internet Engineering
|
||
Task Force (IETF). Note that other groups may also distribute
|
||
working documents as Internet-Drafts. The list of current Internet-
|
||
Drafts is at https://datatracker.ietf.org/drafts/current/.
|
||
|
||
Internet-Drafts are draft documents valid for a maximum of six months
|
||
and may be updated, replaced, or obsoleted by other documents at any
|
||
time. It is inappropriate to use Internet-Drafts as reference
|
||
material or to cite them other than as "work in progress."
|
||
|
||
This Internet-Draft will expire on 21 December 2024.
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 1]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
Copyright Notice
|
||
|
||
Copyright (c) 2024 IETF Trust and the persons identified as the
|
||
document authors. All rights reserved.
|
||
|
||
This document is subject to BCP 78 and the IETF Trust's Legal
|
||
Provisions Relating to IETF Documents (https://trustee.ietf.org/
|
||
license-info) in effect on the date of publication of this document.
|
||
Please review these documents carefully, as they describe your rights
|
||
and restrictions with respect to this document. Code Components
|
||
extracted from this document must include Revised BSD License text as
|
||
described in Section 4.e of the Trust Legal Provisions and are
|
||
provided without warranty as described in the Revised BSD License.
|
||
|
||
Table of Contents
|
||
|
||
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
|
||
2. Core principle . . . . . . . . . . . . . . . . . . . . . . . 3
|
||
3. Hypermediatic FeedbackLoop for XR browsers . . . . . . . . . 5
|
||
4. Conventions and Definitions . . . . . . . . . . . . . . . . . 8
|
||
4.1. XR Fragment URL Grammar . . . . . . . . . . . . . . . . . 8
|
||
5. Spatial Referencing 3D . . . . . . . . . . . . . . . . . . . 8
|
||
6. List of URI Fragments . . . . . . . . . . . . . . . . . . . . 9
|
||
6.1. List of metadata for 3D nodes . . . . . . . . . . . . . . 10
|
||
6.2. Fragment-to-metadata mapping . . . . . . . . . . . . . . 11
|
||
6.3. media fragments and datatypes . . . . . . . . . . . . . . 12
|
||
7. Navigating 3D . . . . . . . . . . . . . . . . . . . . . . . . 15
|
||
8. Top-level URL processing . . . . . . . . . . . . . . . . . . 16
|
||
9. Embedding XR content using src . . . . . . . . . . . . . . . 17
|
||
10. Navigating content href portals . . . . . . . . . . . . . . . 19
|
||
10.1. Walking surfaces . . . . . . . . . . . . . . . . . . . . 21
|
||
10.2. UX spec . . . . . . . . . . . . . . . . . . . . . . . . 21
|
||
10.3. Scaling instanced content . . . . . . . . . . . . . . . 21
|
||
11. XR Fragment: pos . . . . . . . . . . . . . . . . . . . . . . 22
|
||
12. XR Fragment: rot . . . . . . . . . . . . . . . . . . . . . . 22
|
||
13. XR Fragment: t . . . . . . . . . . . . . . . . . . . . . . . 22
|
||
14. XR audio/video integration . . . . . . . . . . . . . . . . . 22
|
||
15. XR Fragment filters . . . . . . . . . . . . . . . . . . . . . 23
|
||
15.1. including/excluding . . . . . . . . . . . . . . . . . . 23
|
||
15.2. Filter Parser . . . . . . . . . . . . . . . . . . . . . 24
|
||
16. Visible links . . . . . . . . . . . . . . . . . . . . . . . . 25
|
||
17. Text in XR (tagging,linking to spatial objects) . . . . . . . 25
|
||
17.1. Default Data URI mimetype . . . . . . . . . . . . . . . 29
|
||
17.2. URL and Data URI . . . . . . . . . . . . . . . . . . . . 30
|
||
17.3. XR Text example parser . . . . . . . . . . . . . . . . . 31
|
||
18. Transclusion (broken link) resolution . . . . . . . . . . . . 33
|
||
19. Topic-based index-less Webrings . . . . . . . . . . . . . . . 34
|
||
20. URI Templates (RFC6570) . . . . . . . . . . . . . . . . . . . 35
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 2]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
21. Additional scene metadata . . . . . . . . . . . . . . . . . . 35
|
||
22. Accessibility interface . . . . . . . . . . . . . . . . . . . 37
|
||
22.1. Two-button navigation . . . . . . . . . . . . . . . . . 37
|
||
23. Security Considerations . . . . . . . . . . . . . . . . . . . 38
|
||
24. FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
|
||
25. authors . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
|
||
26. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 39
|
||
27. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 39
|
||
28. Appendix: Definitions . . . . . . . . . . . . . . . . . . . . 40
|
||
|
||
1. Introduction
|
||
|
||
How can we add more control to existing text and 3D scenes, without
|
||
introducing new dataformats?
|
||
Historically, there's many attempts to create the ultimate 3D
|
||
fileformat.
|
||
The lowest common denominator is: designers describing/tagging/naming
|
||
things using *plain text*.
|
||
XR Fragments exploits the fact that all 3D models already contain
|
||
such metadata:
|
||
|
||
*XR Fragments allows controlling of metadata in 3D scene(files) using
|
||
URI's*
|
||
|
||
It solves:
|
||
|
||
1. addressibility and hypermediatic
|
||
(https://github.com/coderofsalvation/hypermediatic) navigation of
|
||
3D scenes/objects: URI Fragments (https://en.wikipedia.org/wiki/
|
||
URI_fragment) using src/href spatial metadata
|
||
2. Interlinking text & spatial objects by collapsing space into a
|
||
Word Graph (XRWG) to show visible links (#visible-links)
|
||
3. unlocking spatial potential of the (originally 2D) hashtag (which
|
||
jumps to a chapter) for navigating XR documents
|
||
4. refraining from introducing scripting-engines for mundane tasks
|
||
(and preventing its inevitable security-headaches)
|
||
5. the gap between text an 3d objects: object-names directly map to
|
||
hashtags (=fragments), which allows 3D to text transcription.
|
||
|
||
| NOTE: The chapters in this document are ordered from highlevel to
|
||
| lowlevel (technical) as much as possible
|
||
|
||
2. Core principle
|
||
|
||
*XR Fragments allows controlling 3D models using URLs, based on
|
||
(non)existing metadata via URI's*
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 3]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
XR Fragments tries to seek to connect the world of text (semantical
|
||
web / RDF), and the world of pixels.
|
||
Instead of forcing authors to combine 3D/2D objects programmatically
|
||
(publishing thru a game-editor e.g.), XR Fragments *integrates all*
|
||
which allows a universal viewing experience.
|
||
|
||
+───────────────────────────────────────────────────────────────────────────────────────────────+
|
||
│ │
|
||
│ U R N │
|
||
│ U R L | │
|
||
│ | |-----------------+--------| │
|
||
│ +--------------------------------------------------| │
|
||
│ | │
|
||
│ + https://foo.com/some/foo/scene.glb#someview <-- http URI (=URL and has URN) │
|
||
│ | │
|
||
│ + ipfs://cfe0987ec9r9098ecr/cats.fbx#someview <-- an IPFS URI (=URL and has URN) │
|
||
│ │
|
||
│ ec09f7e9cf8e7f09c8e7f98e79c09ef89e000efece8f7ecfe9fe <-- an interpeer URI │
|
||
│ │
|
||
│ │
|
||
│ |------------------------+-------------------------| │
|
||
│ | │
|
||
│ U R I │
|
||
│ │
|
||
+───────────────────────────────────────────────────────────────────────────────────────────────+
|
||
|
||
Fact: our typical browser URL's are just *a possible implementation*
|
||
of URI's (for untapped humancentric potential of URI's see
|
||
interpeer.io (https://interpeer.io))
|
||
|
||
| XR Fragments does not look at XR (or the web) thru the lens of
|
||
| HTML or URLs.
|
||
| But approaches things from a higherlevel feedbackloop/hypermedia
|
||
| browser-perspective.
|
||
|
||
Below you can see how this translates back into good-old URLs:
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 4]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
+───────────────────────────────────────────────────────────────────────────────────────────────+
|
||
│ │
|
||
│ the soul of any URL: ://macro /meso ?micro #nano │
|
||
│ │
|
||
│ 2D URL: ://library.com /document ?search #chapter │
|
||
│ xrf:// │
|
||
│ 4D URL: ://park.com /4Dscene.fbx ─> ?other.glb ─> #view ───> hashbus │
|
||
│ │ #filter │ │
|
||
│ │ #tag │ │
|
||
│ │ (hypermediatic) #material │ │
|
||
│ │ ( feedback ) #animation │ │
|
||
│ │ ( loop ) #texture │ │
|
||
│ │ #variable │ │
|
||
│ │ │ │
|
||
│ XRWG <─────────────────────<─────────────+ │
|
||
│ │ │ │
|
||
│ └─ objects ──────────────>─────────────+ │
|
||
│ │
|
||
│ │
|
||
+───────────────────────────────────────────────────────────────────────────────────────────────+
|
||
|
||
| ?-linked and #-linked navigation are JUST one possible way to
|
||
| implement XR Fragments: the essential goal is to allow a
|
||
| Hypermediatic FeedbackLoop (HFL) between external and internal 4D
|
||
| navigation.
|
||
|
||
Traditional webbrowsers can become 4D document-ready by:
|
||
|
||
3. Hypermediatic FeedbackLoop for XR browsers
|
||
|
||
href metadata traditionally implies *click* AND *navigate*, however
|
||
XR Fragments adds stateless *click* (xrf://#....) or *navigate*
|
||
(xrf://#pos=...) as well (which allows many extra interactions which
|
||
otherwise need a scripting language). This is known as *hashbus*-
|
||
only events (see image above).
|
||
|
||
| Being able to use the same URI Fragment DSL for navigation (href:
|
||
| #foo) as well as interactions (href: xrf://#bar) greatly
|
||
| simplifies implementation, increases HFL, and reduces need for
|
||
| scripting languages.
|
||
|
||
This opens up the following benefits for traditional & future
|
||
webbrowsers:
|
||
|
||
* hypermediatic (https://github.com/coderofsalvation/hypermediatic)
|
||
loading/clicking 3D assets (gltf/fbx e.g.) natively (with or
|
||
without using HTML).
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 5]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
* allowing 3D assets/nodes to publish XR Fragments to themselves/
|
||
eachother using the xrf:// hashbus
|
||
* collapsing the 3D scene to an wordgraph (for essential navigation
|
||
purposes) controllable thru a hash(tag)bus
|
||
* completely bypassing the security-trap of loading external scripts
|
||
(by loading 3D model-files, not HTML-javascriptable resources)
|
||
|
||
XR Fragments itself are hypermediatic
|
||
(https://github.com/coderofsalvation/hypermediatic) and HTML-
|
||
agnostic, though pseudo-XR Fragment browsers *can* be implemented on
|
||
top of HTML/Javascript.
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 6]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
+=========+======================+=====================================+
|
||
|principle|XR 4D URL |HTML 2D URL |
|
||
+=========+======================+=====================================+
|
||
|the XRWG |wordgraph (collapses |Ctrl-F (find) |
|
||
| |3D scene to tags) | |
|
||
+---------+----------------------+-------------------------------------+
|
||
|the |hashtags alter |hashtags alter document positions |
|
||
|hashbus |camera/scene/object- | |
|
||
| |projections | |
|
||
+---------+----------------------+-------------------------------------+
|
||
|src |renders content and |renders content |
|
||
|metadata |offers sourceportation| |
|
||
+---------+----------------------+-------------------------------------+
|
||
|href |teleports to other XR |jumps to other HTML document |
|
||
|metadata |document | |
|
||
+---------+----------------------+-------------------------------------+
|
||
|href |triggers predefined |Media fragments |
|
||
|metadata |view | |
|
||
+---------+----------------------+-------------------------------------+
|
||
|href |triggers |n/a |
|
||
|metadata |camera/scene/object/ | |
|
||
| |projections | |
|
||
+---------+----------------------+-------------------------------------+
|
||
|href |draws visible |n/a |
|
||
|metadata |connection(s) for XRWG| |
|
||
| |'tag' | |
|
||
+---------+----------------------+-------------------------------------+
|
||
|href |filters certain |n/a |
|
||
|metadata |(in)visible objects | |
|
||
+---------+----------------------+-------------------------------------+
|
||
|href |href="xrf://#-foo&bar"|href="javascript:hideFooAndShowBar()`|
|
||
|metadata | | |
|
||
+---------+----------------------+-------------------------------------+
|
||
| |(this does not update |(this is non-standard, non- |
|
||
| |topLevel URI) |hypermediatic) |
|
||
+---------+----------------------+-------------------------------------+
|
||
|
||
Table 1
|
||
|
||
| An important aspect of HFL is that URI Fragments can be triggered
|
||
| without updating the top-level URI (default href-behaviour) thru
|
||
| their own 'bus' (xrf://#.....). This decoupling between
|
||
| navigation and interaction prevents non-standard things like
|
||
| (href:javascript:dosomething()).
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 7]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
4. Conventions and Definitions
|
||
|
||
See appendix below in case certain terms are not clear.
|
||
|
||
4.1. XR Fragment URL Grammar
|
||
|
||
For typical HTTP-like browsers/applications:
|
||
|
||
reserved = gen-delims / sub-delims
|
||
gen-delims = "#" / "&"
|
||
sub-delims = "," / "="
|
||
|
||
| Example: ://foo.com/my3d.gltf#pos=1,0,0&prio=-5&t=0,100
|
||
|
||
+==========================+=================================+
|
||
| Demo | Explanation |
|
||
+==========================+=================================+
|
||
| pos=1,2,3 | vector/coordinate argument e.g. |
|
||
+--------------------------+---------------------------------+
|
||
| pos=1,2,3&rot=0,90,0&foo | combinators |
|
||
+--------------------------+---------------------------------+
|
||
|
||
Table 2
|
||
|
||
| this is already implemented in all browsers
|
||
|
||
Pseudo (non-native) browser-implementations (supporting XR Fragments
|
||
using HTML+JS e.g.) can use the ? search-operator to address outbound
|
||
content.
|
||
In other words, the URL updates to: https://me.com?https://me.com/
|
||
other.glb when navigating to https://me.com/other.glb from inside a
|
||
https://me.com WebXR experience e.g.
|
||
That way, if the link gets shared, the XR Fragments implementation at
|
||
https://me.com can load the latter (and still indicates which XR
|
||
Fragments entrypoint-experience/client was used).
|
||
|
||
5. Spatial Referencing 3D
|
||
|
||
XR Fragments assume the following objectname-to-URIFragment mapping:
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 8]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
my.io/scene.fbx
|
||
+─────────────────────────────+
|
||
│ sky │ src: http://my.io/scene.fbx#sky (includes building,mainobject,floor)
|
||
│ +─────────────────────────+ │
|
||
│ │ building │ │ src: http://my.io/scene.fbx#building (includes mainobject,floor)
|
||
│ │ +─────────────────────+ │ │
|
||
│ │ │ mainobject │ │ │ src: http://my.io/scene.fbx#mainobject (includes floor)
|
||
│ │ │ +─────────────────+ │ │ │
|
||
│ │ │ │ floor │ │ │ │ src: http://my.io/scene.fbx#floor (just floor object)
|
||
│ │ │ │ │ │ │ │
|
||
│ │ │ +─────────────────+ │ │ │
|
||
│ │ +─────────────────────+ │ │
|
||
│ +─────────────────────────+ │
|
||
+─────────────────────────────+
|
||
|
||
| Every 3D fileformat supports named 3D object, and this name allows
|
||
| URLs (fragments) to reference them (and their children objects).
|
||
|
||
Clever nested design of 3D scenes allow great ways for re-using
|
||
content, and/or previewing scenes.
|
||
For example, to render a portal with a preview-version of the scene,
|
||
create an 3D object with:
|
||
|
||
* href: https://scene.fbx
|
||
* src: https://otherworld.gltf#mainobject
|
||
|
||
| It also allows *sourceportation*, which basically means the
|
||
| enduser can teleport to the original XR Document of an src
|
||
| embedded object, and see a visible connection to the particular
|
||
| embedded object. Basically an embedded link becoming an outbound
|
||
| link by activating it.
|
||
|
||
6. List of URI Fragments
|
||
|
||
+=======================+======================================+============+============+
|
||
|fragment |type |example |info |
|
||
+=======================+======================================+============+============+
|
||
|#pos |vector3 |#pos=0.5,0,0|positions |
|
||
| | | |camera (or |
|
||
| | | |XR floor) to|
|
||
| | | |xyz-coord |
|
||
| | | |0.5,0,0, |
|
||
+-----------------------+--------------------------------------+------------+------------+
|
||
|#rot |vector3 |#rot=0,90,0 |rotates |
|
||
| | | |camera to |
|
||
| | | |xyz-coord |
|
||
| | | |0.5,0,0 |
|
||
+-----------------------+--------------------------------------+------------+------------+
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 9]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
|Media Fragments |media fragment |#t=0,2&loop |play (and |
|
||
|(https://www.w3.org/TR/|(#media%20fragments%20and%20datatypes)| |loop) 3D |
|
||
|media-frags/) | | |animation |
|
||
| | | |from 0 |
|
||
| | | |seconds till|
|
||
| | | |2 seconds |
|
||
+-----------------------+--------------------------------------+------------+------------+
|
||
| | | |but can also|
|
||
| | | |crop, |
|
||
| | | |animate & |
|
||
| | | |configure |
|
||
| | | |uv- |
|
||
| | | |coordinates/|
|
||
| | | |shader |
|
||
| | | |uniforms |
|
||
+-----------------------+--------------------------------------+------------+------------+
|
||
|
||
Table 3
|
||
|
||
6.1. List of metadata for 3D nodes
|
||
|
||
+======+========+============+===================+================+
|
||
| key | type | example | function | existing |
|
||
| | | (JSON) | | compatibility |
|
||
+======+========+============+===================+================+
|
||
| href | string | "href": | XR teleport | custom |
|
||
| | | "b.gltf" | | property in 3D |
|
||
| | | | | fileformats |
|
||
+------+--------+------------+-------------------+----------------+
|
||
| src | string | "src": | XR embed / | custom |
|
||
| | | "#cube" | teleport | property in 3D |
|
||
| | | | | fileformats |
|
||
+------+--------+------------+-------------------+----------------+
|
||
| tag | string | "tag": | tag object (for | custom |
|
||
| | | "cubes | filter-use / XRWG | property in 3D |
|
||
| | | geo" | highlighting) | fileformats |
|
||
+------+--------+------------+-------------------+----------------+
|
||
| # | string | "#": | trigger default | custom |
|
||
| | | "#mypreset | fragment on load | property in 3D |
|
||
| | | | | fileformats |
|
||
+------+--------+------------+-------------------+----------------+
|
||
|
||
Table 4
|
||
|
||
| Supported popular compatible 3D fileformats: .gltf, .obj, .fbx,
|
||
| .usdz, .json (THREE.js), .dae and so on.
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 10]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
6.2. Fragment-to-metadata mapping
|
||
|
||
These are automatic fragment-to-metadata mappings, which only trigger
|
||
if the 3D scene metadata matches a specific identifier:
|
||
|
||
+================+======================================+=============+=====================+=======================+
|
||
| |fragment |type |example |info |
|
||
+================+======================================+=============+=====================+=======================+
|
||
|*PRESET* |#<preset> |string |#cubes |evaluates preset |
|
||
| | | | |(#foo&bar) defined in |
|
||
| | | | |3D Object metadata |
|
||
| | | | |(#cubes: #foo&bar e.g.)|
|
||
| | | | |while URL-browserbar |
|
||
| | | | |reflects #cubes. Only |
|
||
| | | | |works when metadata-key|
|
||
| | | | |starts with # |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
|*FOCUS* |#<tag_or_objectname> |string |#person |(and show) object(s) |
|
||
| | | | |with tag: person or |
|
||
| | | | |name person (XRWG |
|
||
| | | | |lookup) |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
|*FILTERS* |#[!][-]<tag_or_objectname>[*] |string |#person (#-person) |will reset (!), show/ |
|
||
| | | | |focus or hide (-) focus|
|
||
| | | | |object(s) with tag: |
|
||
| | | | |person or name person |
|
||
| | | | |by looking up XRWG |
|
||
| | | | |(*=including children) |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
|*CAMERASWITCH* |#<cameraname> |string |#cam01 |sets camera with name |
|
||
| | | | |cam01 as active camera |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
|*MATERIALUPDATE*|#<tag_or_objectname>[*]=<materialname>|string=string|#car=metallic |sets material of car to|
|
||
| | | | |material with name |
|
||
| | | | |metallic (*=including |
|
||
| | | | |children) |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
| | | |#soldout*=halfopacity|set material of objects|
|
||
| | | | |tagged with product to |
|
||
| | | | |material with name |
|
||
| | | | |metallic |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
|*VARIABLE |#<variable>=<metadata-key> |string=string|#foo=bar |sets URI Template |
|
||
|UPDATE* | | | |(https://www.rfc- |
|
||
| | | | |editor.org/rfc/rfc6570)|
|
||
| | | | |variable foo to the |
|
||
| | | | |value #t=0 from |
|
||
| | | | |*existing* object |
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 11]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
| | | | |metadata (bar:#t=0 |
|
||
| | | | |e.g.), This allows for |
|
||
| | | | |reactive URI Template |
|
||
| | | | |(https://www.rfc- |
|
||
| | | | |editor.org/rfc/rfc6570)|
|
||
| | | | |defined in object |
|
||
| | | | |metadata elsewhere |
|
||
| | | | |(src:://m.com/ |
|
||
| | | | |cat.mp4#{foo} e.g., to |
|
||
| | | | |play media using media |
|
||
| | | | |fragment URI |
|
||
| | | | |(https://www.w3.org/TR/|
|
||
| | | | |media-frags/#valid- |
|
||
| | | | |uri)). NOTE: metadata-|
|
||
| | | | |key should not start |
|
||
| | | | |with # |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
|*ANIMATION* |#<tag_or_objectname>=<animationname> |string=string|#people=walk |assign a different |
|
||
| | | |#people=noanim |animation to object(s) |
|
||
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|
||
|
||
Table 5
|
||
|
||
6.3. media fragments and datatypes
|
||
|
||
| NOTE: below the word 'play' applies to 3D animations embedded in
|
||
| the 3D scene(file) *but also* media defined in src-metadata like
|
||
| audio/video-files (mp3/mp4 e.g.)
|
||
|
||
+===========+======================+===============+================+
|
||
| type | syntax | example | info |
|
||
+===========+======================+===============+================+
|
||
| vector2 | x,y | 2,3.0 | 2-dimensional |
|
||
| | | | vector |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| vector3 | x,y,z | 2,3.0,4 | 3-dimensional |
|
||
| | | | vector |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| temporal | t=x | 0 | play from 0 |
|
||
| W3C media | | | seconds to |
|
||
| fragment | | | end (and |
|
||
| | | | stop) |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| temporal | t=x,y | 0,2 | play from 0 |
|
||
| W3C media | | | seconds till |
|
||
| fragment | | | 2 seconds |
|
||
| | | | (and stop) |
|
||
+-----------+----------------------+---------------+----------------+
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 12]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
| temporal | s=x | 1 | set playback |
|
||
| W3C media | | | speed of |
|
||
| fragment | | | audio/ |
|
||
| * | | | video/3D anim |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| temporal | [-]loop | loop | enable looped |
|
||
| W3C media | | | playback of |
|
||
| fragment | | | audio/ |
|
||
| * | | | video/3D anim |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| | | -loop | disable |
|
||
| | | | looped |
|
||
| | | | playback |
|
||
| | | | (does not |
|
||
| | | | affect |
|
||
| | | | playbackstate |
|
||
| | | | of media) |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| vector2 | uv=u,v,uspeed,vspeed | 0,0 | set uv offset |
|
||
| | | | instantly |
|
||
| | | | (default |
|
||
| | | | speed = 1,1) |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| | | +0.5,+0.5 | scroll |
|
||
| | | | instantly by |
|
||
| | | | adding 0.5 to |
|
||
| | | | the current |
|
||
| | | | uv |
|
||
| | | | coordinates |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| | | 0.2,1,0.1,0.1 | scroll (lerp) |
|
||
| | | | to uv |
|
||
| | | | coordinate |
|
||
| | | | 0,2,1 with |
|
||
| | | | 0.1 units per |
|
||
| | | | second |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| | | 0,0,0,+0.1 | scroll v |
|
||
| | | | coordinates |
|
||
| | | | with 0.1 |
|
||
| | | | units per |
|
||
| | | | second |
|
||
| | | | (infinitely) |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| | | +0.5,+0.5 | scroll |
|
||
| | | | instantly by |
|
||
| | | | adding 0.5 to |
|
||
| | | | the current |
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 13]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
| | | | uv |
|
||
| | | | coordinates |
|
||
+-----------+----------------------+---------------+----------------+
|
||
| media | u:<uniform>=<string | float | vec2 |
|
||
| parameter | | | |
|
||
| (shader | | | |
|
||
| uniform) | | | |
|
||
+-----------+----------------------+---------------+----------------+
|
||
|
||
Table 6
|
||
|
||
| * = this is extending the W3C media fragments
|
||
| (https://www.w3.org/TR/media-frags/#mf-advanced) with (missing)
|
||
| playback/viewport-control. Normally #t=0,2 implies setting start/
|
||
| stop-values AND starting playback, whereas #s=0&loop allows
|
||
| pausing a video, speeding up/slowing down media, as well as
|
||
| enabling/disabling looping.
|
||
|
|
||
| The rationale for uv is that the xywh Media Fragment deals with
|
||
| rectangular media, which does not translate well to 3D models
|
||
| (which use triangular polygons, not rectangular) positioned by uv-
|
||
| coordinates. This also explains the absense of a scale or rotate
|
||
| primitive, which is challenged by this, as well as multiple
|
||
| origins (mesh- or texture).
|
||
|
||
Example URI's:
|
||
|
||
* https://images.org/credits.jpg#uv=0,0,0,+0.1 (infinite vertical
|
||
texturescrolling)
|
||
* https://video.org/organogram.mp4#t=0&loop&uv=0.1,0.1,0.3,0.3
|
||
(animated tween towards region in looped video)
|
||
* https://shaders.org/plasma.glsl#t=0&u:col2=0,1,0 (red-green shader
|
||
plasma starts playing from time-offset 0)
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 14]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
+──────────────────────────────────────────────────────────+
|
||
│ │
|
||
│ index.gltf#playall │
|
||
│ │ │
|
||
│ ├ # : #t=0&shared=play │ apply default XR Fragment on load (`t` plays global 3D animation timeline)
|
||
│ ├ play : #t=0&loop │ variable for [URI Templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570)
|
||
│ │ │
|
||
│ ├── ◻ plane (with material) │
|
||
│ │ └ #: #uv=0,0,0,+0.1 │ infinite texturescroll `v` of uv·coordinates with 0.1/fps
|
||
│ │ │
|
||
│ ├── ◻ plane │
|
||
│ │ └ src: foo.jpg#uv=0,0,0,+0.1 │ infinite texturescroll `v` of uv·coordinates with 0.1/fps
|
||
│ │ │
|
||
│ ├── ◻ media │
|
||
│ │ └ src: cat.mp4#t=l:2,10&uv=0.5,0.5 │ loop cat.mp4 (or mp3/wav/jpg) between 2 and 10 seconds (uv's shifted with 0.5,0.5)
|
||
│ │ │
|
||
│ └── ◻ wall │
|
||
│ ├ href: #color=blue │ updates uniform values (IFS shader e.g.)
|
||
│ ├ blue: t=0&u:col=0,0,1 │ variable for [Level1 URI Templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570)
|
||
│ └ src: ://a.com/art.glsl#{color}&{shared} │ .fs/.vs/.glsl/.wgsl etc shader [Level1 URI Template (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570)
|
||
│ │
|
||
│ │
|
||
+──────────────────────────────────────────────────────────+
|
||
|
||
> NOTE: URI Template variables are immutable and respect scope: in other words, the end-user cannot modify `blue` by entering an URL like `#blue=.....` in the browser URL, and `blue` is not accessible by the plane/media-object (however `{play}` would work).
|
||
|
||
7. Navigating 3D
|
||
|
||
+====================+=========+=============================+
|
||
| fragment | type | functionality |
|
||
+====================+=========+=============================+
|
||
| <b>#pos</b>=0,0,0 | vector3 | (re)position camera based |
|
||
| | or | on coordinates directly, or |
|
||
| | string | indirectly using objectname |
|
||
| | | (its worldposition) |
|
||
+--------------------+---------+-----------------------------+
|
||
| <b>#rot</b>=0,90,0 | vector3 | rotate camera |
|
||
+--------------------+---------+-----------------------------+
|
||
|
||
Table 7
|
||
|
||
» example implementation
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
|
||
three/xrf/pos.js)
|
||
» discussion (https://github.com/coderofsalvation/xrfragment/
|
||
issues/5)
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 15]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
1. the Y-coordinate of pos identifies the floorposition. This means
|
||
that desktop-projections usually need to add 1.5m (average person
|
||
height) on top (which is done automatically by VR/AR headsets).
|
||
2. set the position of the camera accordingly to the vector3 values
|
||
of #pos
|
||
3. rot sets the rotation of the camera (only for non-VR/AR headsets)
|
||
4. mediafragment t in the top-URL sets the playbackspeed and
|
||
animation-range of the global scene animation
|
||
5. before scene load: the scene is cleared
|
||
6. after scene load: in case the scene (rootnode) contains an #
|
||
default view with a fragment value: execute non-positional
|
||
fragments via the hashbus (no top-level URL change)
|
||
7. after scene load: in case the scene (rootnode) contains an #
|
||
default view with a fragment value: execute positional fragment
|
||
via the hashbus + update top-level URL
|
||
8. in case of no default # view on the scene (rootnode), default
|
||
player(rig) position 0,0,0 is assumed.
|
||
9. in case a href does not mention any pos-coordinate, the current
|
||
position will be assumed
|
||
|
||
Here's an ascii representation of a 3D scene-graph which contains 3D
|
||
objects ◻ and their metadata:
|
||
|
||
+────────────────────────────────────────────────────────+
|
||
│ │
|
||
│ index.gltf │
|
||
│ │ │
|
||
│ ├── ◻ buttonA │
|
||
│ │ └ href: #pos=1,0,1&t=100,200 │
|
||
│ │ │
|
||
│ └── ◻ buttonB │
|
||
│ └ href: other.fbx │ <── file─agnostic (can be .gltf .obj etc)
|
||
│ │
|
||
+────────────────────────────────────────────────────────+
|
||
|
||
An XR Fragment-compatible browser viewing this scene, allows the end-
|
||
user to interact with the buttonA and buttonB.
|
||
In case of buttonA the end-user will be teleported to another
|
||
location and time in the *current loaded scene*, but buttonB will
|
||
*replace the current scene* with a new one, like other.fbx, and
|
||
assume pos=0,0,0.
|
||
|
||
8. Top-level URL processing
|
||
|
||
| Example URL: ://foo/world.gltf#cube&pos=0,0,0
|
||
|
||
The URL-processing-flow for hypermedia browsers goes like this:
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 16]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
1. IF a #cube matches a custom property-key (of an object) in the 3D
|
||
file/scene (#cube: #......) <b>THEN</b> execute that
|
||
predefined_view.
|
||
2. IF scene operators (pos) and/or animation operator (t) are
|
||
present in the URL then (re)position the camera and/or animation-
|
||
range accordingly.
|
||
3. IF no camera-position has been set in <b>step 1 or 2</b> update
|
||
the top-level URL with #pos=0,0,0 (example (https://github.com/co
|
||
derofsalvation/xrfragment/blob/main/src/3rd/js/three/
|
||
navigator.js#L31]]))
|
||
4. IF a #cube matches the name (of an object) in the 3D file/scene
|
||
then draw a line from the enduser('s heart) to that object (to
|
||
highlight it).
|
||
5. IF a #cube matches anything else in the XR Word Graph (XRWG) draw
|
||
wires to them (text or related objects).
|
||
|
||
9. Embedding XR content using src
|
||
|
||
src is the 3D version of the <a target="_blank"
|
||
href="https://www.w3.org/html/wiki/Elements/iframe">iframe</a>.
|
||
It instances content (in objects) in the current scene/asset, and
|
||
follows similar logic like the previous chapter, except that it does
|
||
not modify the camera.
|
||
|
||
+========+========+===================================================+
|
||
|fragment|type |example value |
|
||
+========+========+===================================================+
|
||
|src |string |#cube |
|
||
| |(uri, |#sometag |
|
||
| |hashtag/|#cube&-ball_inside_cube<br>#-sky&-rain<br>#- |
|
||
| |filter) |language&english<br>#price=>5<br>https://linux.org/|
|
||
| | |penguin.png` (https://linux.org/penguin.png`) |
|
||
| | |https://linux.world/distrowatch.gltf#t=1,100 |
|
||
| | |linuxapp://conference/nixworkshop/apply.gltf#- |
|
||
| | |cta&cta_apply |
|
||
| | |androidapp://page1?tutorial#pos=0,0,1&t1,100 |
|
||
| | |foo.mp3#0,0,0 |
|
||
+--------+--------+---------------------------------------------------+
|
||
|
||
Table 8
|
||
|
||
Here's an ascii representation of a 3D scene-graph with 3D objects
|
||
◻ which embeds remote & local 3D objects ◻ with/out using
|
||
filters:
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 17]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
+────────────────────────────────────────────────────────+ +─────────────────────────+
|
||
│ │ │ │
|
||
│ index.gltf │ │ ocean.com/aquarium.fbx │
|
||
│ │ │ │ ├ room │
|
||
│ ├── ◻ canvas │ │ └── ◻ fishbowl │
|
||
│ │ └ src: painting.png │ │ ├─ ◻ bass │
|
||
│ │ │ │ └─ ◻ tuna │
|
||
│ ├── ◻ aquariumcube │ │ │
|
||
│ │ └ src: ://rescue.com/fish.gltf#fishbowl │ +─────────────────────────+
|
||
│ │ │
|
||
│ ├── ◻ bedroom │
|
||
│ │ └ src: #canvas │
|
||
│ │ │
|
||
│ └── ◻ livingroom │
|
||
│ └ src: #canvas │
|
||
│ │
|
||
+────────────────────────────────────────────────────────+
|
||
|
||
An XR Fragment-compatible browser viewing this scene, lazy-loads and
|
||
projects painting.png onto the (plane) object called canvas (which is
|
||
copy-instanced in the bed and livingroom).
|
||
Also, after lazy-loading ocean.com/aquarium.gltf, only the queried
|
||
objects fishbowl (and bass and tuna) will be instanced inside
|
||
aquariumcube.
|
||
Resizing will be happen accordingly to its placeholder object
|
||
aquariumcube, see chapter Scaling.
|
||
|
||
| Instead of cherrypicking a rootobject #fishbowl with src,
|
||
| additional filters can be used to include/exclude certain objects.
|
||
| See next chapter on filtering below.
|
||
|
||
*Specification*:
|
||
|
||
1. local/remote content is instanced by the src (filter) value (and
|
||
attaches it to the placeholder mesh containing the src property)
|
||
2. by default all objects are loaded into the instanced src (scene)
|
||
object (but not shown yet)
|
||
3. <b>local</b> src values (#... e.g.) starting with a non-negating
|
||
filter (#cube e.g.) will (deep)reparent that object (with name
|
||
cube) as the new root of the scene at position 0,0,0
|
||
4. <b>local</b> src values should respect (negative) filters
|
||
(#-foo&price=>3)
|
||
5. the instanced scene (from a src value) should be <b>scaled
|
||
accordingly</b> to its placeholder object or <b>scaled
|
||
relatively</b> based on the scale-property (of a geometry-less
|
||
placeholder, an 'empty'-object in blender e.g.). For more info
|
||
see Chapter Scaling.
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 18]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
6. <b>external</b> src values should be served with appropriate
|
||
mimetype (so the XR Fragment-compatible browser will now how to
|
||
render it). The bare minimum supported mimetypes are:
|
||
7. src values should make its placeholder object invisible, and
|
||
only flush its children when the resolved content can
|
||
succesfully be retrieved (see broken links (#links))
|
||
8. <b>external</b> src values should respect the fallback link
|
||
mechanism (see broken links (#broken-links)
|
||
9. when the placeholder object is a 2D plane, but the mimetype is
|
||
3D, then render the spatial content on that plane via a stencil
|
||
buffer.
|
||
10. src-values are non-recursive: when linking to an external object
|
||
(src: foo.fbx#bar), then src-metadata on object bar should be
|
||
ignored.
|
||
11. an external src-value should always allow a sourceportation icon
|
||
within 3 meter: teleporting to the origin URI to which the
|
||
object belongs.
|
||
12. when only one object was cherrypicked (#cube e.g.), set its
|
||
position to 0,0,0
|
||
13. when the enduser clicks an href with #t=1,0,0 (play) will be
|
||
applied to all src mediacontent with a timeline (mp4/mp3 e.g.)
|
||
14. a non-euclidian portal can be rendered for flat 3D objects
|
||
(using stencil buffer e.g.) in case ofspatial src-values (an
|
||
object #world3 or URL world3.fbx e.g.).
|
||
|
||
* model/gltf-binary
|
||
* model/gltf+json
|
||
* image/png
|
||
* image/jpg
|
||
* text/plain;charset=utf-8
|
||
|
||
» example implementation
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
|
||
three/xrf/src.js)
|
||
» example 3D asset
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
|
||
assets/src.gltf#L192)
|
||
» discussion (https://github.com/coderofsalvation/xrfragment/
|
||
issues/4)
|
||
|
||
10. Navigating content href portals
|
||
|
||
navigation, portals & mutations
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 19]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
+==========+==================+============================+
|
||
| fragment | type | example value |
|
||
+==========+==================+============================+
|
||
| href | string (uri or | #pos=1,1,0 |
|
||
| | predefined view) | #pos=1,1,0&rot=90,0,0 |
|
||
| | | ://somefile.gltf#pos=1,1,0 |
|
||
+----------+------------------+----------------------------+
|
||
|
||
Table 9
|
||
|
||
1. clicking an outbound ''external''- or ''file URI'' fully replaces
|
||
the current scene and assumes pos=0,0,0&rot=0,0,0 by default
|
||
(unless specified)
|
||
|
||
2. relocation/reorientation should happen locally for local URI's
|
||
(#pos=....)
|
||
|
||
3. navigation should not happen ''immediately'' when user is more
|
||
than 5 meter away from the portal/object containing the href (to
|
||
prevent accidental navigation e.g.)
|
||
|
||
4. URL navigation should always be reflected in the client URL-bar
|
||
(in case of javascript: see [here
|
||
(https://github.com/coderofsalvation/xrfragment/blob/dev/src/3rd/
|
||
js/three/navigator.js) for an example navigator), and only update
|
||
the URL-bar after the scene (default fragment #) has been loaded.
|
||
|
||
5. In immersive XR mode, the navigator back/forward-buttons should
|
||
be always visible (using a wearable e.g., see [here
|
||
(https://github.com/coderofsalvation/xrfragment/blob/dev/example/
|
||
aframe/sandbox/index.html#L26-L29) for an example wearable)
|
||
|
||
6. make sure that the ''back-button'' of the ''browser-history''
|
||
always refers to the previous position (see [here (https://github
|
||
.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/xrf/
|
||
href.js#L97))
|
||
|
||
7. ignore previous rule in special cases, like clicking an href
|
||
using camera-portal collision (the back-button could cause a
|
||
teleport-loop if the previous position is too close)
|
||
|
||
8. href-events should bubble upward the node-tree (from children to
|
||
ancestors, so that ancestors can also contain an href), however
|
||
only 1 href can be executed at the same time.
|
||
|
||
9. the end-user navigator back/forward buttons should repeat a back/
|
||
forward action until a pos=... primitive is found (the stateless
|
||
xrf:// href-values should not be pushed to the url-history)
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 20]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
» example implementation
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
|
||
three/xrf/href.js)
|
||
» example 3D asset
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
|
||
assets/href.gltf#L192)
|
||
» discussion (https://github.com/coderofsalvation/xrfragment/
|
||
issues/1)
|
||
|
||
10.1. Walking surfaces
|
||
|
||
XR Fragment-compatible viewers can infer this data based scanning the
|
||
scene for:
|
||
|
||
1. materialless (nameless & textureless) mesh-objects (without src
|
||
and href)
|
||
|
||
| optionally the viewer can offer thumbstick, mouse or joystick
|
||
| teleport-tools for non-roomscale VR/AR setups.
|
||
|
||
10.2. UX spec
|
||
|
||
End-users should always have read/write access to:
|
||
|
||
1. the current (toplevel) <b>URL</b> (an URLbar etc)
|
||
2. URL-history (a <b>back/forward</b> button e.g.)
|
||
3. Clicking/Touching an href navigates (and updates the URL) to
|
||
another scene/file (and coordinate e.g. in case the URL contains
|
||
XR Fragments).
|
||
|
||
10.3. Scaling instanced content
|
||
|
||
Sometimes embedded properties (like src) instance new objects.
|
||
But what about their scale?
|
||
How does the scale of the object (with the embedded properties)
|
||
impact the scale of the referenced content?
|
||
|
||
| Rule of thumb: visible placeholder objects act as a '3D canvas'
|
||
| for the referenced scene (a plane acts like a 2D canvas for images
|
||
| e, a cube as a 3D canvas e.g.).
|
||
|
||
1. <b>IF</b> an embedded property (src e.g.) is set on an non-empty
|
||
placeholder object (geometry of >2 vertices):
|
||
|
||
* calculate the <b>bounding box</b> of the ''placeholder'' object
|
||
(maxsize=1.4 e.g.)
|
||
* hide the ''placeholder'' object (material e.g.)
|
||
* instance the src scene as a child of the existing object
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 21]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
* calculate the <b>bounding box</b> of the instanced scene, and
|
||
scale it accordingly (to 1.4 e.g.)
|
||
|
||
| REASON: non-empty placeholder object can act as a protective
|
||
| bounding-box (for remote content of which might grow over time
|
||
| e.g.)
|
||
|
||
2. ELSE multiply the scale-vector of the instanced scene with the
|
||
scale-vector (a common property of a 3D node) of the
|
||
<b>placeholder</b> object.
|
||
|
||
| TODO: needs intermediate visuals to make things more obvious
|
||
|
||
11. XR Fragment: pos
|
||
|
||
[[» example implementation|https://github.com/coderofsalvation/xrfrag
|
||
ment/blob/main/src/3rd/js/three/xrf/pos.js]
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
|
||
three/xrf/pos.js])]
|
||
|
||
12. XR Fragment: rot
|
||
|
||
[[» example implementation|https://github.com/coderofsalvation/xrfrag
|
||
ment/blob/main/src/3rd/js/three/xrf/pos.js]
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
|
||
three/xrf/pos.js])]
|
||
|
||
13. XR Fragment: t
|
||
|
||
[[» example implementation|https://github.com/coderofsalvation/xrfrag
|
||
ment/blob/main/src/3rd/js/three/xrf/t.js]
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
|
||
three/xrf/t.js])]
|
||
|
||
14. XR audio/video integration
|
||
|
||
To play global audio/video items:
|
||
|
||
1. add a src: foo.mp3 or src: bar.mp4 metadata to a 3D object (cube
|
||
e.g.)
|
||
2. to enable auto-play and global timeline ([[#t=|t]]) control:
|
||
hardcode a [[#t=|t]] XR Fragment: (src: bar.mp3#t=0&loop e.g.)
|
||
3. to play it, add href: #cube somewhere else
|
||
4. to enable enduser-triggered play, use a [[URI Template]] XR
|
||
Fragment: (src: bar.mp3#{player} and play: t=0&loop and href:
|
||
xrf://#player=play e.g.)
|
||
5. when the enduser clicks the href, #t=0&loop (play) will be
|
||
applied to the src value
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 22]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
| NOTE: hardcoded framestart/framestop uses sampleRate/fps of
|
||
| embedded audio/video, otherwise the global fps applies. For more
|
||
| info see [[#t|t]].
|
||
|
||
15. XR Fragment filters
|
||
|
||
Include, exclude, hide/shows objects using space-separated strings:
|
||
|
||
+====================+===========================================+
|
||
| example | outcome |
|
||
+====================+===========================================+
|
||
| #-sky | show everything except object named sky |
|
||
+--------------------+-------------------------------------------+
|
||
| #-language&english | hide everything with tag language, but |
|
||
| | show all tag english objects |
|
||
+--------------------+-------------------------------------------+
|
||
| #-price&price=>10 | hide all objects with property price, |
|
||
| | then only show object with price above 10 |
|
||
+--------------------+-------------------------------------------+
|
||
| #-house* | hide house object and everything inside |
|
||
| | (=*) |
|
||
+--------------------+-------------------------------------------+
|
||
|
||
Table 10
|
||
|
||
It's simple but powerful syntax which allows filtering the scene
|
||
using searchengine prompt-style feeling:
|
||
|
||
1. filters are a way to traverse a scene, and filter objects based
|
||
on their name, tag- or property-values.
|
||
|
||
* see an (outdated) example video here
|
||
(https://coderofsalvation.github.io/xrfragment.media/queries.mp4)
|
||
which used a dedicated q= variable (now deprecated and usable
|
||
directly)
|
||
|
||
15.1. including/excluding
|
||
|
||
By default, selectors work like photoshop-layers: they scan for
|
||
matching layer(name/properties) within the scene-graph. Each matched
|
||
object (not their children) will be toggled (in)visible when
|
||
selecting.
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 23]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
+==========+==============================================+
|
||
| operator | info |
|
||
+==========+==============================================+
|
||
| - | hides object(s) (#-myobject&-objects e.g. |
|
||
+----------+----------------------------------------------+
|
||
| = | indicates an object-embedded custom property |
|
||
| | key/value (#price=4&category=foo e.g.) |
|
||
+----------+----------------------------------------------+
|
||
| => =< | compare float or int number (#price=>4 e.g.) |
|
||
+----------+----------------------------------------------+
|
||
| * | deepselect: automatically select children of |
|
||
| | selected object, including local (nonremote) |
|
||
| | embedded objects (starting with #) |
|
||
+----------+----------------------------------------------+
|
||
|
||
Table 11
|
||
|
||
| NOTE 1: after an external embedded object has been instanced (src:
|
||
| https://y.com/bar.fbx#room e.g.), filters do not affect them
|
||
| anymore (reason: local tag/name collisions can be mitigated
|
||
| easily, but not in case of remote content).
|
||
|
|
||
| NOTE 2: depending on the used 3D framework, toggling objects
|
||
| (in)visible should happen by enabling/disableing writing to the
|
||
| colorbuffer (to allow children being still visible while their
|
||
| parents are invisible).
|
||
|
||
» example implementation
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
|
||
three/xrf/q.js) » example 3D asset
|
||
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
|
||
assets/filter.gltf#L192) » discussion
|
||
(https://github.com/coderofsalvation/xrfragment/issues/3)
|
||
|
||
15.2. Filter Parser
|
||
|
||
Here's how to write a filter parser:
|
||
|
||
1. create an associative array/object to store filter-arguments as
|
||
objects
|
||
2. detect object id's & properties foo=1 and foo (reference regex=
|
||
~/^.*=[><=]?/ )
|
||
3. detect excluders like -foo,-foo=1,-.foo,-/foo (reference regex=
|
||
/^-/ )
|
||
4. detect root selectors like /foo (reference regex= /^[-]?\// )
|
||
5. detect number values like foo=1 (reference regex= /^[0-9\.]+$/ )
|
||
6. detect operators so you can easily strip keys (reference regex=
|
||
/(^-|\*$)/ )
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 24]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
7. detect exclude keys like -foo (reference regex= /^-/ )
|
||
8. for every filter token split string on =
|
||
9. and we set root to true or false (true=/ root selector is
|
||
present)
|
||
10. therefore we we set show to true or false (false=excluder -)
|
||
|
||
| An example filter-parser (which compiles to many languages) can be
|
||
| found here
|
||
| (https://github.com/coderofsalvation/xrfragment/blob/main/src/
|
||
| xrfragment/Filter.hx)
|
||
|
||
16. Visible links
|
||
|
||
When predefined views, XRWG fragments and ID fragments (#cube or
|
||
#mytag e.g.) are triggered by the enduser (via toplevel URL or
|
||
clicking href):
|
||
|
||
1. draw a wire from the enduser (preferabbly a bit below the camera,
|
||
heartposition) to object(s) matching that ID (objectname)
|
||
2. draw a wire from the enduser (preferabbly a bit below the camera,
|
||
heartposition) to object(s) matching that tag value
|
||
3. draw a wire from the enduser (preferabbly a bit below the camera,
|
||
heartposition) to object(s) containing that in their src or href
|
||
value
|
||
|
||
The obvious approach for this, is to consult the XRWG (example
|
||
(https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
|
||
src/3rd/js/XRWG.js)), which basically has all these things already
|
||
collected/organized for you during scene-load.
|
||
|
||
*UX*
|
||
|
||
4. do not update the wires when the enduser moves, leave them as is
|
||
5. offer a control near the back/forward button which allows the
|
||
user to (turn off) control the correlation-intensity of the XRWG
|
||
|
||
17. Text in XR (tagging,linking to spatial objects)
|
||
|
||
How does XR Fragments interlink text with objects?
|
||
|
||
| The XR Fragments does this by collapsing space into a *Word Graph*
|
||
| (the *XRWG* example
|
||
| (https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
|
||
| src/3rd/js/XRWG.js)), augmented by Bib(s)Tex.
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 25]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
Instead of just throwing together all kinds media types into one
|
||
experience (games), what about their tagged/semantical relationships?
|
||
Perhaps the following question is related: why is HTML adopted less
|
||
in games outside the browser?
|
||
|
||
Hence:
|
||
|
||
1. XR Fragments promotes (de)serializing a scene to a (lowercase)
|
||
XRWG (example
|
||
(https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
|
||
src/3rd/js/XRWG.js))
|
||
2. XR Fragments primes the XRWG, by collecting words from the tag
|
||
and name-property of 3D objects.
|
||
3. XR Fragments primes the XRWG, by collecting words from *optional*
|
||
metadata *at the end of content* of text (see default mimetype &
|
||
Data URI)
|
||
4. The XRWG should be recalculated when textvalues (in src) change
|
||
5. HTML/RDF/JSON is still great, but is beyond the XRWG-scope (they
|
||
fit better in the application-layer, or as embedded src content)
|
||
6. Applications don't have to be able to access the XRWG
|
||
programmatically, as they can easily generate one themselves by
|
||
traversing the scene-nodes.
|
||
7. The XR Fragment focuses on fast and easy-to-generate end-user
|
||
controllable word graphs (instead of complex implementations that
|
||
try to defeat word ambiguity)
|
||
8. Tags are the scope for now (supporting https://github.com/WICG/
|
||
scroll-to-text-fragment (https://github.com/WICG/scroll-to-text-
|
||
fragment) will be considered)
|
||
|
||
Example of generating BiBTex out of the XRWG and textdata with
|
||
hashtags:
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 26]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
http://y.io/z.fbx | Derived XRWG (expressed as BibTex)
|
||
----------------------------------------------------------------------------+--------------------------------------
|
||
| @house{castle,
|
||
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | url = {https://y.io/z.fbx#castle}
|
||
| Chapter one | | / \ | | }
|
||
| | | / \ | | @baroque{castle,
|
||
| John built houses in baroque style. | | / \ | | url = {https://y.io/z.fbx#castle}
|
||
| | | |_____| | | }
|
||
| | +-----│-----+ | @baroque{john}
|
||
| | │ |
|
||
| | ├─ name: castle |
|
||
| | └─ tag: house baroque |
|
||
+----------------------------------------+ |
|
||
[3D mesh ] |
|
||
| O ├─ name: john |
|
||
| /|\ | |
|
||
| / \ | |
|
||
+--------+ |
|
||
|
||
| the #john@baroque-bib associates both text John and objectname
|
||
| john, with tag baroque
|
||
|
||
Another example of deriving a graphdata from the XRWG:
|
||
|
||
http://y.io/z.fbx | Derived XRWG (expressed as BibTex)
|
||
----------------------------------------------------------------------------+--------------------------------------
|
||
|
|
||
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | @house{castle,
|
||
| Chapter one | | / \ | | url = {https://y.io/z.fbx#castle}
|
||
| | | / \ | | }
|
||
| John built houses in baroque style. | | / \ | | @baroque{castle,
|
||
| | | |_____| | | url = {https://y.io/z.fbx#castle}
|
||
| #john@baroque | +-----│-----+ | }
|
||
| @baroque{john} | │ | @baroque{john}
|
||
| | ├─ name: castle |
|
||
| | └─ tag: house baroque |
|
||
+----------------------------------------+ | @house{baroque}
|
||
[3D mesh ] | @todo{baroque}
|
||
+-[remotestorage.io / localstorage]------+ | O + name: john |
|
||
| #baroque@todo@house | | /|\ | |
|
||
| ... | | / \ | |
|
||
+----------------------------------------+ +--------+ |
|
||
|
||
| both #john@baroque-bib and BibTex @baroque{john} result in the
|
||
| same XRWG, however on top of that 2 tages (house and todo) are now
|
||
| associated with text/objectname/tag 'baroque'.
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 27]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
As seen above, the XRWG can expand bibs
|
||
(https://github.com/coderofsalvation/hashtagbibs) (and the whole
|
||
scene) to BibTeX.
|
||
This allows hasslefree authoring and copy-paste of associations *for
|
||
and by humans*, but also makes these URLs possible:
|
||
|
||
+==================+======================================+
|
||
| URL example | Result |
|
||
+==================+======================================+
|
||
| https://my.com/ | draws lines between mesh john, 3D |
|
||
| foo.gltf#baroque | mesh castle, text John built(..) |
|
||
+------------------+--------------------------------------+
|
||
| https://my.com/ | draws lines between mesh john, and |
|
||
| foo.gltf#john | the text John built (..) |
|
||
+------------------+--------------------------------------+
|
||
| https://my.com/ | draws lines between mesh castle, and |
|
||
| foo.gltf#house | other objects with tag house or todo |
|
||
+------------------+--------------------------------------+
|
||
|
||
Table 12
|
||
|
||
| hashtagbibs (https://github.com/coderofsalvation/hashtagbibs)
|
||
| potentially allow the enduser to annotate text/objects by
|
||
| *speaking/typing/scanning associations*, which the XR Browser
|
||
| saves to remotestorage (or localStorage per toplevel URL). As
|
||
| well as, referencing BibTags per URI later on: https://y.io/
|
||
| z.fbx#@baroque@todo e.g.
|
||
|
||
The XRWG allows XR Browsers to show/hide relationships in realtime at
|
||
various levels:
|
||
|
||
* wordmatch *inside* src text
|
||
* wordmatch *inside* href text
|
||
* wordmatch object-names
|
||
* wordmatch object-tagnames
|
||
|
||
Spatial wires can be rendered between words/objects etc.
|
||
Some pointers for good UX (but not necessary to be XR Fragment
|
||
compatible):
|
||
|
||
9. The XR Browser needs to adjust tag-scope based on the endusers
|
||
needs/focus (infinite tagging only makes sense when environment
|
||
is scaled down significantly)
|
||
10. The XR Browser should always allow the human to view/edit the
|
||
metadata, by clicking 'toggle metadata' on the 'back'
|
||
(contextmenu e.g.) of any XR text, anywhere anytime.
|
||
11. respect multi-line BiBTeX metadata in text because of the core
|
||
principle (#core-principle)
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 28]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
12. Default font (unless specified otherwise) is a modern monospace
|
||
font, for maximized tabular expressiveness (see the core
|
||
principle (#core-principle)).
|
||
13. anti-pattern: hardcoupling an XR Browser with a mandatory
|
||
*markup/scripting-language* which departs from onubtrusive plain
|
||
text (HTML/VRML/Javascript) (see the core principle (#core-
|
||
principle))
|
||
14. anti-pattern: limiting human introspection, by abandoning plain
|
||
text as first tag citizen.
|
||
|
||
| The simplicity of appending metadata (and leveling the metadata-
|
||
| playfield between humans and machines) is also demonstrated by
|
||
| visual-meta (https://visual-meta.info) in greater detail.
|
||
|
||
Fictional chat:
|
||
|
||
<John> Hey what about this: https://my.com/station.gltf#pos=0,0,1&rot=90,2,0&t=500,1000
|
||
<Sarah> I'm checking it right now
|
||
<Sarah> I don't see everything..where's our text from yesterday?
|
||
<John> Ah wait, that's tagged with tag 'draft' (and hidden)..hold on, try this:
|
||
<John> https://my.com/station.gltf#.draft&pos=0,0,1&rot=90,2,0&t=500,1000
|
||
<Sarah> how about we link the draft to the upcoming YELLO-event?
|
||
<John> ok I'm adding #draft@YELLO
|
||
<Sarah> Yesterday I also came up with other usefull assocations between other texts in the scene:
|
||
#event#YELLO
|
||
#2025@YELLO
|
||
<John> thanks, added.
|
||
<Sarah> Btw. I stumbled upon this spatial book which references station.gltf in some chapters:
|
||
<Sarah> https://thecommunity.org/forum/foo/mytrainstory.txt
|
||
<John> interesting, I'm importing mytrainstory.txt into station.gltf
|
||
<John> ah yes, chapter three points to trainterminal_2A in the scene, cool
|
||
|
||
17.1. Default Data URI mimetype
|
||
|
||
The src-values work as expected (respecting mime-types), however:
|
||
|
||
The XR Fragment specification advices to bump the traditional default
|
||
browser-mimetype
|
||
|
||
text/plain;charset=US-ASCII
|
||
|
||
to a hashtagbib(tex)-friendly one:
|
||
|
||
text/plain;charset=utf-8;bib=^@
|
||
|
||
This indicates that:
|
||
|
||
* utf-8 is supported by default
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 29]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
* lines beginning with @ will not be rendered verbatim by default
|
||
(read more (https://github.com/coderofsalvation/
|
||
hashtagbibs#hashtagbib-mimetypes))
|
||
* the XRWG should expand bibs to BibTex occurring in text
|
||
(#contactjohn@todo@important e.g.)
|
||
|
||
By doing so, the XR Browser (applications-layer) can interpret
|
||
microformats (visual-meta (https://visual-meta.info) to connect text
|
||
further with its environment ( setup links between textual/spatial
|
||
objects automatically e.g.).
|
||
|
||
| for more info on this mimetype see bibs
|
||
| (https://github.com/coderofsalvation/hashtagbibs)
|
||
|
||
Advantages:
|
||
|
||
* auto-expanding of hashtagbibs
|
||
(https://github.com/coderofsalvation/hashtagbibs) associations
|
||
* out-of-the-box (de)multiplex human text and metadata in one go
|
||
(see the core principle (#core-principle))
|
||
* no network-overhead for metadata (see the core principle (#core-
|
||
principle))
|
||
* ensuring high FPS: HTML/RDF historically is too 'requesty'/'parsy'
|
||
for game studios
|
||
* rich send/receive/copy-paste everywhere by default, metadata being
|
||
retained (see the core principle (#core-principle))
|
||
* netto result: less webservices, therefore less servers, and
|
||
overall better FPS in XR
|
||
|
||
| This significantly expands expressiveness and portability of human
|
||
| tagged text, by *postponing machine-concerns to the end of the
|
||
| human text* in contrast to literal interweaving of content and
|
||
| markupsymbols (or extra network requests, webservices e.g.).
|
||
|
||
For all other purposes, regular mimetypes can be used (but are not
|
||
required by the spec).
|
||
|
||
17.2. URL and Data URI
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 30]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
+--------------------------------------------------------------+ +------------------------+
|
||
| | | author.com/article.txt |
|
||
| index.gltf | +------------------------+
|
||
| │ | | |
|
||
| ├── ◻ article_canvas | | Hello friends. |
|
||
| │ └ src: ://author.com/article.txt | | |
|
||
| │ | | @book{greatgatsby |
|
||
| └── ◻ note_canvas | | ... |
|
||
| └ src:`data:welcome human\n@book{sunday...}` | | } |
|
||
| | +------------------------+
|
||
| |
|
||
+--------------------------------------------------------------+
|
||
|
||
The enduser will only see welcome human and Hello friends rendered
|
||
verbatim (see mimetype). The beauty is that text in Data URI
|
||
automatically promotes rich copy-paste (retaining metadata). In both
|
||
cases, the text gets rendered immediately (onto a plane geometry,
|
||
hence the name '_canvas'). The XR Fragment-compatible browser can
|
||
let the enduser access visual-meta(data)-fields after interacting
|
||
with the object (contextmenu e.g.).
|
||
|
||
| additional tagging using bibs
|
||
| (https://github.com/coderofsalvation/hashtagbibs): to tag spatial
|
||
| object note_canvas with 'todo', the enduser can type or speak
|
||
| #note_canvas@todo
|
||
|
||
17.3. XR Text example parser
|
||
|
||
To prime the XRWG with text from plain text src-values, here's an
|
||
example XR Text (de)multiplexer in javascript (which supports inline
|
||
bibs & bibtex):
|
||
|
||
xrtext = {
|
||
|
||
expandBibs: (text) => {
|
||
let bibs = { regex: /(#[a-zA-Z0-9_+@\-]+(#)?)/g, tags: {}}
|
||
text.replace( bibs.regex , (m,k,v) => {
|
||
tok = m.substr(1).split("@")
|
||
match = tok.shift()
|
||
if( tok.length ) tok.map( (t) => bibs.tags[t] = `@${t}{${match},\n}` )
|
||
else if( match.substr(-1) == '#' )
|
||
bibs.tags[match] = `@{${match.replace(/#/,'')}}`
|
||
else bibs.tags[match] = `@${match}{${match},\n}`
|
||
})
|
||
return text.replace( bibs.regex, '') + Object.values(bibs.tags).join('\n')
|
||
},
|
||
|
||
decode: (str) => {
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 31]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
// bibtex: ↓@ ↓<tag|tag{phrase,|{ruler}> ↓property ↓end
|
||
let pat = [ /@/, /^\S+[,{}]/, /},/, /}/ ]
|
||
let tags = [], text='', i=0, prop=''
|
||
let lines = xrtext.expandBibs(str).replace(/\r?\n/g,'\n').split(/\n/)
|
||
for( let i = 0; i < lines.length && !String(lines[i]).match( /^@/ ); i++ )
|
||
text += lines[i]+'\n'
|
||
|
||
bibtex = lines.join('\n').substr( text.length )
|
||
bibtex.split( pat[0] ).map( (t) => {
|
||
try{
|
||
let v = {}
|
||
if( !(t = t.trim()) ) return
|
||
if( tag = t.match( pat[1] ) ) tag = tag[0]
|
||
if( tag.match( /^{.*}$/ ) ) return tags.push({ruler:tag})
|
||
if( tag.match( /}$/ ) ) return tags.push({k: tag.replace(/}$/,''), v: {}})
|
||
t = t.substr( tag.length )
|
||
t.split( pat[2] )
|
||
.map( kv => {
|
||
if( !(kv = kv.trim()) || kv == "}" ) return
|
||
v[ kv.match(/\s?(\S+)\s?=/)[1] ] = kv.substr( kv.indexOf("{")+1 )
|
||
})
|
||
tags.push( { k:tag, v } )
|
||
}catch(e){ console.error(e) }
|
||
})
|
||
return {text, tags}
|
||
},
|
||
|
||
encode: (text,tags) => {
|
||
let str = text+"\n"
|
||
for( let i in tags ){
|
||
let item = tags[i]
|
||
if( item.ruler ){
|
||
str += `@${item.ruler}\n`
|
||
continue;
|
||
}
|
||
str += `@${item.k}\n`
|
||
for( let j in item.v ) str += ` ${j} = {${item.v[j]}}\n`
|
||
str += `}\n`
|
||
}
|
||
return str
|
||
}
|
||
}
|
||
|
||
The above functions (de)multiplexe text/metadata, expands bibs,
|
||
(de)serialize bibtex and vice versa
|
||
|
||
| above can be used as a startingpoint for LLVM's to translate/
|
||
| steelman to a more formal form/language.
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 32]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
str = `
|
||
hello world
|
||
here are some hashtagbibs followed by bibtex:
|
||
|
||
#world
|
||
#hello@greeting
|
||
#another-section#
|
||
|
||
@{some-section}
|
||
@flap{
|
||
asdf = {23423}
|
||
}`
|
||
|
||
var {tags,text} = xrtext.decode(str) // demultiplex text & bibtex
|
||
tags.find( (t) => t.k == 'flap{' ).v.asdf = 1 // edit tag
|
||
tags.push({ k:'bar{', v:{abc:123} }) // add tag
|
||
console.log( xrtext.encode(text,tags) ) // multiplex text & bibtex back together
|
||
|
||
This expands to the following (hidden by default) BibTex appendix:
|
||
|
||
hello world
|
||
here are some hashtagbibs followed by bibtex:
|
||
|
||
@{some-section}
|
||
@flap{
|
||
asdf = {1}
|
||
}
|
||
@world{world,
|
||
}
|
||
@greeting{hello,
|
||
}
|
||
@{another-section}
|
||
@bar{
|
||
abc = {123}
|
||
}
|
||
|
||
| when an XR browser updates the human text, a quick scan for
|
||
| nonmatching tags (@book{nonmatchingbook e.g.) should be performed
|
||
| and prompt the enduser for deleting them.
|
||
|
||
18. Transclusion (broken link) resolution
|
||
|
||
In spirit of Ted Nelson's 'transclusion resolution', there's a soft-
|
||
mechanism to harden links & minimize broken links in various ways:
|
||
|
||
1. defining a different transport protocol (https vs ipfs or DAT) in
|
||
src or href values can make a difference
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 33]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
2. mirroring files on another protocol using (HTTP) errorcode tags
|
||
in src or href properties
|
||
3. in case of src: nesting a copy of the embedded object in the
|
||
placeholder object (embeddedObject) will not be replaced when the
|
||
request fails
|
||
|
||
| due to the popularity, maturity and extensiveness of HTTP codes
|
||
| for client/server communication, non-HTTP protocols easily map to
|
||
| HTTP codes (ipfs ERR_NOT_FOUND maps to 404 e.g.)
|
||
|
||
For example:
|
||
|
||
+────────────────────────────────────────────────────────+
|
||
│ │
|
||
│ index.gltf │
|
||
│ │ │
|
||
│ │ #: #-offlinetext │
|
||
│ │ │
|
||
│ ├── ◻ buttonA │
|
||
│ │ └ href: http://foo.io/campagne.fbx │
|
||
│ │ └ href@404: ipfs://foo.io/campagne.fbx │
|
||
│ │ └ href@400: #clienterrortext │
|
||
│ │ └ ◻ offlinetext │
|
||
│ │ │
|
||
│ └── ◻ embeddedObject <--------- the meshdata inside embeddedObject will (not)
|
||
│ └ src: https://foo.io/bar.gltf │ be flushed when the request (does not) succeed.
|
||
│ └ src@404: http://foo.io/bar.gltf │ So worstcase the 3D data (of the time of publishing index.gltf)
|
||
│ └ src@400: https://archive.org/l2kj43.gltf │ will be displayed.
|
||
│ │
|
||
+────────────────────────────────────────────────────────+
|
||
|
||
19. Topic-based index-less Webrings
|
||
|
||
As hashtags in URLs map to the XWRG, href-values can be used to
|
||
promote topic-based index-less webrings.
|
||
Consider 3D scenes linking to eachother using these href values:
|
||
|
||
* href: schoolA.edu/projects.gltf#math
|
||
* href: schoolB.edu/projects.gltf#math
|
||
* href: university.edu/projects.gltf#math
|
||
|
||
These links would all show visible links to math-tagged objects in
|
||
the scene.
|
||
To filter out non-related objects one could take it a step further
|
||
using filters:
|
||
|
||
* href: schoolA.edu/projects.gltf#math&-topics math
|
||
* href: schoolB.edu/projects.gltf#math&-courses math
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 34]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
* href: university.edu/projects.gltf#math&-theme math
|
||
|
||
| This would hide all object tagged with topic, courses or theme
|
||
| (including math) so that later only objects tagged with math will
|
||
| be visible
|
||
|
||
This makes spatial content multi-purpose, without the need to
|
||
separate content into separate files, or show/hide things using a
|
||
complex logiclayer like javascript.
|
||
|
||
20. URI Templates (RFC6570)
|
||
|
||
XR Fragments adopts Level1 URI *Fragment* expansion to provide safe
|
||
interactivity.
|
||
The following demonstrates a simple video player:
|
||
|
||
+─────────────────────────────────────────────+
|
||
│ │
|
||
│ foo.usdz │
|
||
│ │ │
|
||
│ │ │
|
||
│ ├── ◻ stopbutton │
|
||
│ │ ├ #: #-stopbutton │
|
||
│ │ └ href: #player=stop&-stopbutton │ (stop and hide stop-button)
|
||
│ │ │
|
||
│ └── ◻ plane │
|
||
│ ├ play: #t=l:0,10 │
|
||
│ ├ stop: #t=0,0 │
|
||
│ ├ href: #player=play&stopbutton │ (play and show stop-button)
|
||
│ └ src: cat.mp4#{player} │
|
||
│ │
|
||
│ │
|
||
+─────────────────────────────────────────────+
|
||
|
||
21. Additional scene metadata
|
||
|
||
XR Fragments does not aim to redefine the metadata-space or
|
||
accessibility-space by introducing its own cataloging-metadata
|
||
fields. Instead, it encourages browsers to scan nodes for the
|
||
following custom properties:
|
||
|
||
* SPDX (https://spdx.dev/) license information
|
||
* ARIA (https://www.w3.org/WAI/standards-guidelines/aria/)
|
||
attributes (aria-*: .....)
|
||
* Open Graph (https://ogp.me) attributes (og:*: .....)
|
||
* Dublin-Core (https://www.dublincore.org/specifications/dublin-
|
||
core/application-profile-guidelines/) attributes(dc:*: .....)
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 35]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
* BibTex (https://bibtex.eu/fields) when known bibtex-keys exist
|
||
with values enclosed in { and },
|
||
|
||
*ARIA* (aria-description) is the most important to support, as it
|
||
promotes accessibility and allows scene transcripts. Please start
|
||
aria-description with a verb to aid transcripts.
|
||
|
||
| Example: object 'tryceratops' with aria-description: is a huge
|
||
| dinosaurus standing on a #mountain generates transcript
|
||
| #tryceratops is a huge dinosaurus standing on a #mountain, where
|
||
| the hashtags are clickable XR Fragments (activating the visible-
|
||
| links in the XR browser).
|
||
|
||
Individual nodes can be enriched with such metadata, but most
|
||
importantly the scene node:
|
||
|
||
+================================+=========================+
|
||
| metadata key | example value |
|
||
+================================+=========================+
|
||
| aria-description, | An immersive experience |
|
||
| og:description, dc:description | about Triceratops (*) |
|
||
+--------------------------------+-------------------------+
|
||
| SPDX | CC0-1.0 |
|
||
+--------------------------------+-------------------------+
|
||
| dc:creator | John Doe |
|
||
+--------------------------------+-------------------------+
|
||
| dc:title, og:title | 'Triceratops` (*) |
|
||
+--------------------------------+-------------------------+
|
||
| og:site_name | https://xrfragment.org |
|
||
+--------------------------------+-------------------------+
|
||
| dc.publisher | NLNET |
|
||
+--------------------------------+-------------------------+
|
||
| dc.date | 2024-01-01 |
|
||
+--------------------------------+-------------------------+
|
||
| dc.identifier | XRFRAGMENT-001 |
|
||
+--------------------------------+-------------------------+
|
||
| journal (bibTeX) | {Future Of Text Vol 3}, |
|
||
+--------------------------------+-------------------------+
|
||
|
||
Table 13
|
||
|
||
| * = these are interchangable (only one needs to be defined)
|
||
|
||
There's no silver bullet when it comes to metadata, so one should
|
||
support where the metadata is/goes.
|
||
|
||
| These attributes can be scanned and presented during an href or
|
||
| src eye/mouse-over.
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 36]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
22. Accessibility interface
|
||
|
||
The addressibility of XR Fragments allows for unique 3D-to-text
|
||
transcripts, as well as an textual interface to navigate 3D content.
|
||
Spec:
|
||
<Br>
|
||
|
||
1. The enduser must be able to enable an accessibility-mode (which
|
||
persists across application/webpage restarts)
|
||
2. Accessibility-mode must contain a text-input for the user to
|
||
enter text
|
||
3. Accessibility-mode must contain a flexible textlog for the user
|
||
to read (via screenreader, screen, or TTS e.g.)
|
||
4. the textlog contains aria-descriptions, and its narration
|
||
(Screenreader e.g.) can be skipped (via 2-button navigation)
|
||
5. The back command should navigate back to the previous URL (alias
|
||
for browser-backbutton)
|
||
6. The forward command should navigate back to the next URL (alias
|
||
for browser-nextbutton)
|
||
7. A destination is a 3D node containing an href with a pos= XR
|
||
fragment
|
||
8. The go command should list all possible destinations
|
||
9. The go left command should move the camera around 0.3 meters to
|
||
the left
|
||
10. The go right command should move the camera around 0.3 meters to
|
||
the right
|
||
11. The go forward command should move the camera 0.3 meters forward
|
||
(direction of current rotation).
|
||
12. The rotate left command should rotate the camera 0.3 to the left
|
||
13. The rotate left command should rotate the camera 0.3 to the
|
||
right
|
||
14. The (dynamic) go abc command should navigate to #pos=scene2 in
|
||
case there's a 3D node with name abc and href value #pos=scene2
|
||
15. The look command should give an (contextual) 3D-to-text
|
||
transcript, by scanning the aria-description values of the
|
||
current pos= value (including its children)
|
||
16. The do command should list all possible href values which don't
|
||
contain an pos= XR Fragment
|
||
17. The (dynamic) do abc command should navigate/execute
|
||
https://.../... in case a 3D node exist with name abc and href
|
||
value https://.../...
|
||
|
||
22.1. Two-button navigation
|
||
|
||
For specific user-profiles, gyroscope/mouse/keyboard/audio/visuals
|
||
will not be available.
|
||
Therefore a 2-button navigation-interface is the bare minimum
|
||
interface:
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 37]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
1. objects with href metadata can be cycled via a key (tab on a
|
||
keyboard)
|
||
2. objects with href metadata can be activated via a key (enter on a
|
||
keyboard)
|
||
3. the TTS reads the href-value (and/or aria-description if
|
||
available)
|
||
|
||
23. Security Considerations
|
||
|
||
The only dynamic parts are W3C Media Fragments
|
||
(https://www.w3.org/TR/media-frags/) and URI Templates (RFC6570)
|
||
(https://www.rfc-editor.org/rfc/rfc6570).
|
||
The use of URI Templates is limited to pre-defined variables and
|
||
Level0 fragments-expansion only, which makes it quite safe.
|
||
In fact, it is much safer than relying on a scripting language
|
||
(javascript) which can change URN too.
|
||
|
||
24. FAQ
|
||
|
||
*Q:* Why is everything HTTP GET-based, what about POST/PUT/DELETE
|
||
HATEOS
|
||
*A:* Because it's out of scope: XR Fragment specifies a read-only way
|
||
to surf XR documents. These things belong in the application layer
|
||
(for example, an XR Hypermedia browser can decide to support
|
||
POST/PUT/DELETE requests for embedded HTML thru src values)
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 38]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
*Q:* Why isn't there support for scripting, URI Template Fragments
|
||
are so limited compared to WASM & javascript *A:* This is out of
|
||
scope as it unhyperifies hypermedia, and this is up to XR hypermedia
|
||
browser-extensions.
|
||
Historically scripting/Javascript seems to been able to turn webpages
|
||
from hypermedia documents into its opposite (hyperscripted
|
||
nonhypermedia documents).
|
||
In order to prevent this backward-movement (hypermedia tends to
|
||
liberate people from finnicky scripting) XR Fragment uses W3C Media
|
||
Fragments (https://www.w3.org/TR/media-frags/) and URI Templates
|
||
(RFC6570) (https://www.rfc-editor.org/rfc/rfc6570), to prevent
|
||
unhyperifying itself by hardcoupling to a particular markup or
|
||
scripting language.
|
||
XR Fragments supports filtering objects in a scene only, because in
|
||
the history of the javascript-powered web, showing/hiding document-
|
||
entities seems to be one of the most popular basic usecases.
|
||
Doing advanced scripting & networkrequests under the hood are
|
||
obviously interesting endavours, but this is something which should
|
||
not be hardcoupled with XR Fragments or hypermedia.
|
||
This perhaps belongs more to browser extensions.
|
||
Non-HTML Hypermedia browsers should make browser extensions the right
|
||
place, to 'extend' experiences, in contrast to code/javascript inside
|
||
hypermedia documents (this turned out as a hypermedia antipattern).
|
||
|
||
25. authors
|
||
|
||
* Leon van Kammen (@lvk@mastodon.online)
|
||
* Jens Finkhäuser (@jens@social.finkhaeuser.de)
|
||
|
||
26. IANA Considerations
|
||
|
||
This document has no IANA actions.
|
||
|
||
27. Acknowledgments
|
||
|
||
* NLNET (https://nlnet.nl)
|
||
* Future of Text (https://futureoftext.org)
|
||
* visual-meta.info (https://visual-meta.info)
|
||
* Michiel Leenaars
|
||
* Gerben van der Broeke
|
||
* Mauve
|
||
* Jens Finkhäuser
|
||
* Marc Belmont
|
||
* Tim Gerritsen
|
||
* Frode Hegland
|
||
* Brandel Zackernuk
|
||
* Mark Anderson
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 39]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
28. Appendix: Definitions
|
||
|
||
+=================+=============================================+
|
||
| definition | explanation |
|
||
+=================+=============================================+
|
||
| human | a sentient being who thinks fuzzy, absorbs, |
|
||
| | and shares thought (by plain text, not |
|
||
| | markuplanguage) |
|
||
+-----------------+---------------------------------------------+
|
||
| scene | a (local/remote) 3D scene or 3D file |
|
||
| | (index.gltf e.g.) |
|
||
+-----------------+---------------------------------------------+
|
||
| 3D object | an object inside a scene characterized by |
|
||
| | vertex-, face- and customproperty data. |
|
||
+-----------------+---------------------------------------------+
|
||
| URI | some resource at something somewhere via |
|
||
| | someprotocol (http://me.com/foo.glb#foo or |
|
||
| | e76f8efec8efce98e6f see interpeer.io |
|
||
| | (https://interpeer.io)) |
|
||
+-----------------+---------------------------------------------+
|
||
| URL | something somewhere via someprotocol |
|
||
| | (http://me.com/foo.glb) |
|
||
+-----------------+---------------------------------------------+
|
||
| URN | something at some domain (me.com/foo.glb) |
|
||
+-----------------+---------------------------------------------+
|
||
| metadata | custom properties of text, 3D Scene or |
|
||
| | Object(nodes), relevant to machines and a |
|
||
| | human minority (academics/developers) |
|
||
+-----------------+---------------------------------------------+
|
||
| XR fragment | URI Fragment with spatial hints like |
|
||
| | #pos=0,0,0&t=1,100 e.g. |
|
||
+-----------------+---------------------------------------------+
|
||
| the XRWG | wordgraph (collapses 3D scene to tags) |
|
||
+-----------------+---------------------------------------------+
|
||
| the hashbus | hashtags map to camera/scene-projections |
|
||
+-----------------+---------------------------------------------+
|
||
| spacetime | positions camera, triggers scene-preset/ |
|
||
| hashtags | time |
|
||
+-----------------+---------------------------------------------+
|
||
| teleportation | repositioning the enduser to a different |
|
||
| | position (or 3D scene/file) |
|
||
+-----------------+---------------------------------------------+
|
||
| sourceportation | teleporting the enduser to the original XR |
|
||
| | Document of an src embedded object. |
|
||
+-----------------+---------------------------------------------+
|
||
| placeholder | a 3D object which with src-metadata (which |
|
||
| object | will be replaced by the src-data.) |
|
||
+-----------------+---------------------------------------------+
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 40]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
| src | (HTML-piggybacked) metadata of a 3D object |
|
||
| | which instances content |
|
||
+-----------------+---------------------------------------------+
|
||
| href | (HTML-piggybacked) metadata of a 3D object |
|
||
| | which links to content |
|
||
+-----------------+---------------------------------------------+
|
||
| filter | URI Fragment(s) which show/hide object(s) |
|
||
| | in a scene based on name/tag/property |
|
||
| | (#cube&-price=>3) |
|
||
+-----------------+---------------------------------------------+
|
||
| visual-meta | visual-meta (https://visual.meta.info) data |
|
||
| | appended to text/books/papers which is |
|
||
| | indirectly visible/editable in XR. |
|
||
+-----------------+---------------------------------------------+
|
||
| requestless | metadata which never spawns new requests |
|
||
| metadata | (unlike RDF/HTML, which can cause |
|
||
| | framerate-dropping, hence not used a lot in |
|
||
| | games) |
|
||
+-----------------+---------------------------------------------+
|
||
| FPS | frames per second in spatial experiences |
|
||
| | (games,VR,AR e.g.), should be as high as |
|
||
| | possible |
|
||
+-----------------+---------------------------------------------+
|
||
| introspective | inward sensemaking ("I feel this belongs to |
|
||
| | that") |
|
||
+-----------------+---------------------------------------------+
|
||
| extrospective | outward sensemaking ("I'm fairly sure John |
|
||
| | is a person who lives in oklahoma") |
|
||
+-----------------+---------------------------------------------+
|
||
| ◻ | ascii representation of an 3D object/mesh |
|
||
+-----------------+---------------------------------------------+
|
||
| (un)obtrusive | obtrusive: wrapping human text/thought in |
|
||
| | XML/HTML/JSON obfuscates human text into a |
|
||
| | salad of machine-symbols and words |
|
||
+-----------------+---------------------------------------------+
|
||
| flat 3D object | a 3D object of which all verticies share a |
|
||
| | plane |
|
||
+-----------------+---------------------------------------------+
|
||
| BibTeX | simple tagging/citing/referencing standard |
|
||
| | for plaintext |
|
||
+-----------------+---------------------------------------------+
|
||
| BibTag | a BibTeX tag |
|
||
+-----------------+---------------------------------------------+
|
||
| (hashtag)bibs | an easy to speak/type/scan tagging SDL (see |
|
||
| | here (https://github.com/coderofsalvation/ |
|
||
| | hashtagbibs) which expands to BibTex/JSON/ |
|
||
| | XML |
|
||
+-----------------+---------------------------------------------+
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 41]
|
||
|
||
Internet-Draft XR Fragments June 2024
|
||
|
||
|
||
Table 14
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
van Kammen Expires 21 December 2024 [Page 42]
|