2023-09-01 14:20:02 +02:00
2024-02-08 14:05:44 +01:00
Jens & Leon Internet Engineering Task Force L.R. van Kammen
2024-04-25 21:06:03 +02:00
Internet-Draft 25 April 2024
2023-09-01 14:20:02 +02:00
Intended status: Informational
2023-09-04 21:20:59 +02:00
2023-09-01 14:20:02 +02:00
XR Fragments
draft-XRFRAGMENTS-leonvankammen-00
Abstract
2024-01-30 10:58:00 +01:00
This draft is a specification for 4D URI's & hypermediatic
2024-04-25 21:06:03 +02:00
(https://github.com/coderofsalvation/hypermediatic) navigation, to
enable a spatial web for hypermedia browsers with- or without a
network-connection.
2024-02-08 14:05:44 +01:00
The specification uses W3C Media Fragments (https://www.w3.org/TR/
media-frags/) and URI Templates (RFC6570) (https://www.rfc-
editor.org/rfc/rfc6570) to promote spatial addressibility, sharing,
2023-12-06 16:40:55 +01:00
navigation, filtering and databinding objects for (XR) Browsers.
XR Fragments allows us to better use existing metadata inside 3D
scene(files), by connecting it to proven technologies like URI
Fragments (https://en.wikipedia.org/wiki/URI_fragment).
2024-04-25 21:06:03 +02:00
XR Fragments views spatial webs thru the lens of 3D scene URI's,
rather than thru code(frameworks) or protocol-specific browsers
(webbrowser e.g.).
2023-09-05 19:14:10 +02:00
Almost every idea in this document is demonstrated at
https://xrfragment.org (https://xrfragment.org)
2023-09-01 14:20:02 +02:00
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
2024-04-25 21:06:03 +02:00
This Internet-Draft will expire on 27 October 2024.
2023-09-01 14:20:02 +02:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 1]
2023-09-01 14:20:02 +02:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2023-09-01 14:20:02 +02:00
2024-04-25 21:06:03 +02:00
Copyright Notice
Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved.
2023-09-01 14:20:02 +02:00
2023-09-18 11:03:18 +02:00
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
2023-09-05 19:14:10 +02:00
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
2023-09-01 14:20:02 +02:00
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
2023-12-06 16:40:55 +01:00
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2023-09-05 19:14:10 +02:00
2. Core principle . . . . . . . . . . . . . . . . . . . . . . . 3
2024-02-26 12:34:21 +01:00
3. Hypermediatic FeedbackLoop for XR browsers . . . . . . . . . 5
2024-02-29 15:07:20 +01:00
4. Conventions and Definitions . . . . . . . . . . . . . . . . . 8
4.1. XR Fragment URL Grammar . . . . . . . . . . . . . . . . . 8
2024-04-25 21:06:03 +02:00
5. Spatial Referencing 3D . . . . . . . . . . . . . . . . . . . 8
6. List of URI Fragments . . . . . . . . . . . . . . . . . . . . 9
6.1. List of metadata for 3D nodes . . . . . . . . . . . . . . 10
6.2. Fragment-to-metadata mapping . . . . . . . . . . . . . . 11
6.3. media fragments and datatypes . . . . . . . . . . . . . . 12
2024-02-29 15:07:20 +01:00
7. Navigating 3D . . . . . . . . . . . . . . . . . . . . . . . . 15
8. Top-level URL processing . . . . . . . . . . . . . . . . . . 16
2024-04-25 21:06:03 +02:00
9. Embedding XR content using src . . . . . . . . . . . . . . . 17
2024-02-29 15:07:20 +01:00
10. Navigating content href portals . . . . . . . . . . . . . . . 19
2024-04-25 21:06:03 +02:00
10.1. Walking surfaces . . . . . . . . . . . . . . . . . . . . 21
10.2. UX spec . . . . . . . . . . . . . . . . . . . . . . . . 21
2024-02-29 15:07:20 +01:00
10.3. Scaling instanced content . . . . . . . . . . . . . . . 21
2024-04-25 21:06:03 +02:00
11. XR Fragment: pos . . . . . . . . . . . . . . . . . . . . . . 22
12. XR Fragment: rot . . . . . . . . . . . . . . . . . . . . . . 22
13. XR Fragment: t . . . . . . . . . . . . . . . . . . . . . . . 22
14. XR audio/video integration . . . . . . . . . . . . . . . . . 23
15. XR Fragment filters . . . . . . . . . . . . . . . . . . . . . 23
15.1. including/excluding . . . . . . . . . . . . . . . . . . 24
2024-02-29 15:07:20 +01:00
15.2. Filter Parser . . . . . . . . . . . . . . . . . . . . . 24
2024-04-25 21:06:03 +02:00
16. Visible links . . . . . . . . . . . . . . . . . . . . . . . . 25
2024-02-29 15:07:20 +01:00
17. Text in XR (tagging,linking to spatial objects) . . . . . . . 25
17.1. Default Data URI mimetype . . . . . . . . . . . . . . . 29
17.2. URL and Data URI . . . . . . . . . . . . . . . . . . . . 30
17.3. XR Text example parser . . . . . . . . . . . . . . . . . 31
18. Transclusion (broken link) resolution . . . . . . . . . . . . 33
19. Topic-based index-less Webrings . . . . . . . . . . . . . . . 34
20. URI Templates (RFC6570) . . . . . . . . . . . . . . . . . . . 35
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 2]
2023-12-06 16:40:55 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2023-12-06 16:40:55 +01:00
2023-09-01 14:20:02 +02:00
2024-04-25 21:06:03 +02:00
21. Additional scene metadata . . . . . . . . . . . . . . . . . . 35
22. Security Considerations . . . . . . . . . . . . . . . . . . . 37
23. FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
24. authors . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
25. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 38
2024-02-29 15:07:20 +01:00
26. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 38
27. Appendix: Definitions . . . . . . . . . . . . . . . . . . . . 38
2024-02-26 12:34:21 +01:00
2023-09-01 14:20:02 +02:00
1. Introduction
2024-04-25 21:06:03 +02:00
How can we add more control to existing text and 3D scenes, without
2023-09-04 21:20:59 +02:00
introducing new dataformats?
2024-04-25 21:06:03 +02:00
Historically, there's many attempts to create the ultimate 3D
fileformat.
2023-12-06 16:40:55 +01:00
The lowest common denominator is: designers describing/tagging/naming
things using *plain text*.
XR Fragments exploits the fact that all 3D models already contain
such metadata:
2023-09-05 19:14:10 +02:00
2023-12-06 16:40:55 +01:00
*XR Fragments allows controlling of metadata in 3D scene(files) using
2024-01-30 10:58:00 +01:00
URI's*
2023-09-05 19:14:10 +02:00
2024-01-30 10:58:00 +01:00
It solves:
2023-10-12 17:04:46 +02:00
1. addressibility and hypermediatic
(https://github.com/coderofsalvation/hypermediatic) navigation of
3D scenes/objects: URI Fragments (https://en.wikipedia.org/wiki/
2024-02-23 11:51:05 +01:00
URI_fragment) using src/href spatial metadata
2024-01-30 10:58:00 +01:00
2. Interlinking text & spatial objects by collapsing space into a
Word Graph (XRWG) to show visible links (#visible-links)
2023-09-21 13:05:30 +02:00
3. unlocking spatial potential of the (originally 2D) hashtag (which
jumps to a chapter) for navigating XR documents
2024-02-23 11:51:05 +01:00
4. refraining from introducing scripting-engines for mundane tasks
(and preventing its inevitable security-headaches)
2024-02-29 15:07:20 +01:00
5. the gap between text an 3d objects: object-names directly map to
hashtags (=fragments), which allows 3D to text transcription.
2023-09-18 11:03:18 +02:00
2023-09-04 21:20:59 +02:00
| NOTE: The chapters in this document are ordered from highlevel to
| lowlevel (technical) as much as possible
2023-09-05 19:14:10 +02:00
2. Core principle
2023-09-04 21:20:59 +02:00
2024-01-30 10:58:00 +01:00
*XR Fragments allows controlling 3D models using URLs, based on
(non)existing metadata via URI's*
2023-12-06 16:40:55 +01:00
2023-09-21 13:05:30 +02:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 3]
2023-09-21 13:05:30 +02:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-01-30 10:58:00 +01:00
2024-04-25 21:06:03 +02:00
XR Fragments tries to seek to connect the world of text (semantical
web / RDF), and the world of pixels.
Instead of forcing authors to combine 3D/2D objects programmatically
(publishing thru a game-editor e.g.), XR Fragments *integrates all*
which allows a universal viewing experience.
2024-01-30 10:58:00 +01:00
+───────────────────────────────────────────────────────────────────────────────────────────────+
│ │
│ U R N │
│ U R L | │
│ | |-----------------+--------| │
│ +--------------------------------------------------| │
│ | │
│ + https://foo.com/some/foo/scene.glb#someview <-- http URI (=URL and has URN) │
│ | │
│ + ipfs://cfe0987ec9r9098ecr/cats.fbx#someview <-- an IPFS URI (=URL and has URN) │
│ │
│ ec09f7e9cf8e7f09c8e7f98e79c09ef89e000efece8f7ecfe9fe <-- an interpeer URI │
│ │
│ │
│ |------------------------+-------------------------| │
│ | │
│ U R I │
│ │
+───────────────────────────────────────────────────────────────────────────────────────────────+
Fact: our typical browser URL's are just *a possible implementation*
of URI's (for untapped humancentric potential of URI's see
interpeer.io (https://interpeer.io))
2023-12-06 16:40:55 +01:00
2024-01-30 10:58:00 +01:00
| XR Fragments does not look at XR (or the web) thru the lens of
| HTML or URLs.
| But approaches things from a higherlevel feedbackloop/hypermedia
| browser-perspective.
Below you can see how this translates back into good-old URLs:
2023-12-06 16:40:55 +01:00
2023-09-21 13:05:30 +02:00
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
2023-09-18 11:03:18 +02:00
2023-09-04 21:20:59 +02:00
2023-09-01 14:20:02 +02:00
2023-09-18 11:03:18 +02:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 4]
2023-09-18 11:03:18 +02:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-01-30 10:58:00 +01:00
+───────────────────────────────────────────────────────────────────────────────────────────────+
│ │
│ the soul of any URL: ://macro /meso ?micro #nano │
│ │
│ 2D URL: ://library.com /document ?search #chapter │
2024-02-26 12:34:21 +01:00
│ xrf:// │
2024-01-30 10:58:00 +01:00
│ 4D URL: ://park.com /4Dscene.fbx ─> ?other.glb ─> #view ───> hashbus │
│ │ #filter │ │
│ │ #tag │ │
│ │ (hypermediatic) #material │ │
│ │ ( feedback ) #animation │ │
│ │ ( loop ) #texture │ │
│ │ #variable │ │
│ │ │ │
│ XRWG <─────────────────────<─────────────+ │
│ │ │ │
│ └─ objects ──────────────>─────────────+ │
│ │
│ │
+───────────────────────────────────────────────────────────────────────────────────────────────+
2024-02-23 11:51:05 +01:00
| ?-linked and #-linked navigation are JUST one possible way to
2024-02-26 12:34:21 +01:00
| implement XR Fragments: the essential goal is to allow a
| Hypermediatic FeedbackLoop (HFL) between external and internal 4D
| navigation.
2023-09-18 11:03:18 +02:00
2024-02-29 15:07:20 +01:00
Traditional webbrowsers can become 4D document-ready by:
2024-02-26 12:34:21 +01:00
3. Hypermediatic FeedbackLoop for XR browsers
2023-09-18 11:03:18 +02:00
2024-02-26 12:34:21 +01:00
href metadata traditionally implies *click* AND *navigate*, however
2024-04-25 21:06:03 +02:00
XR Fragments adds stateless *click* (xrf://#....) or *navigate*
(xrf://#pos=...) as well (which allows many extra interactions which
otherwise need a scripting language). This is known as *hashbus*-
only events (see image above).
2023-09-04 21:20:59 +02:00
2024-02-26 12:34:21 +01:00
| Being able to use the same URI Fragment DSL for navigation (href:
| #foo) as well as interactions (href: xrf://#bar) greatly
| simplifies implementation, increases HFL, and reduces need for
| scripting languages.
2024-01-30 10:58:00 +01:00
2024-02-26 12:34:21 +01:00
This opens up the following benefits for traditional & future
webbrowsers:
2024-01-30 10:58:00 +01:00
2024-02-26 12:34:21 +01:00
* hypermediatic (https://github.com/coderofsalvation/hypermediatic)
loading/clicking 3D assets (gltf/fbx e.g.) natively (with or
without using HTML).
2024-01-30 10:58:00 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 5]
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-01-30 10:58:00 +01:00
2024-02-29 15:07:20 +01:00
* allowing 3D assets/nodes to publish XR Fragments to themselves/
eachother using the xrf:// hashbus
2024-02-26 12:34:21 +01:00
* collapsing the 3D scene to an wordgraph (for essential navigation
purposes) controllable thru a hash(tag)bus
* completely bypassing the security-trap of loading external scripts
(by loading 3D model-files, not HTML-javascriptable resources)
2024-01-30 10:58:00 +01:00
2024-02-26 12:34:21 +01:00
XR Fragments itself are hypermediatic
(https://github.com/coderofsalvation/hypermediatic) and HTML-
agnostic, though pseudo-XR Fragment browsers *can* be implemented on
top of HTML/Javascript.
2024-01-30 10:58:00 +01:00
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 6]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-02-26 12:34:21 +01:00
+=========+======================+=====================================+
|principle|XR 4D URL |HTML 2D URL |
+=========+======================+=====================================+
|the XRWG |wordgraph (collapses |Ctrl-F (find) |
| |3D scene to tags) | |
+---------+----------------------+-------------------------------------+
|the |hashtags alter |hashtags alter document positions |
|hashbus |camera/scene/object- | |
| |projections | |
+---------+----------------------+-------------------------------------+
|src |renders content and |renders content |
|metadata |offers sourceportation| |
+---------+----------------------+-------------------------------------+
|href |teleports to other XR |jumps to other HTML document |
|metadata |document | |
+---------+----------------------+-------------------------------------+
|href |triggers predefined |Media fragments |
|metadata |view | |
+---------+----------------------+-------------------------------------+
|href |triggers |n/a |
|metadata |camera/scene/object/ | |
| |projections | |
+---------+----------------------+-------------------------------------+
|href |draws visible |n/a |
|metadata |connection(s) for XRWG| |
| |'tag' | |
+---------+----------------------+-------------------------------------+
|href |filters certain |n/a |
|metadata |(in)visible objects | |
+---------+----------------------+-------------------------------------+
|href |href="xrf://#-foo&bar"|href="javascript:hideFooAndShowBar()`|
|metadata | | |
+---------+----------------------+-------------------------------------+
| |(this does not update |(this is non-standard, non- |
| |topLevel URI) |hypermediatic) |
+---------+----------------------+-------------------------------------+
Table 1
| An important aspect of HFL is that URI Fragments can be triggered
| without updating the top-level URI (default href-behaviour) thru
| their own 'bus' (xrf://#.....). This decoupling between
| navigation and interaction prevents non-standard things like
| (href:javascript:dosomething()).
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 7]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-02-26 12:34:21 +01:00
4. Conventions and Definitions
2023-09-04 21:20:59 +02:00
2023-09-07 15:53:32 +02:00
See appendix below in case certain terms are not clear.
2023-09-04 21:20:59 +02:00
2024-02-26 12:34:21 +01:00
4.1. XR Fragment URL Grammar
2024-01-30 10:58:00 +01:00
For typical HTTP-like browsers/applications:
2023-09-04 21:20:59 +02:00
2023-09-18 11:03:18 +02:00
reserved = gen-delims / sub-delims
gen-delims = "#" / "&"
sub-delims = "," / "="
| Example: ://foo.com/my3d.gltf#pos=1,0,0&prio=-5&t=0,100
2023-12-06 16:40:55 +01:00
+==========================+=================================+
| Demo | Explanation |
+==========================+=================================+
| pos=1,2,3 | vector/coordinate argument e.g. |
+--------------------------+---------------------------------+
| pos=1,2,3&rot=0,90,0&foo | combinators |
+--------------------------+---------------------------------+
2023-09-04 21:20:59 +02:00
2023-12-06 16:40:55 +01:00
Table 2
2023-09-05 19:14:10 +02:00
2023-09-18 11:03:18 +02:00
| this is already implemented in all browsers
2024-01-30 10:58:00 +01:00
Pseudo (non-native) browser-implementations (supporting XR Fragments
using HTML+JS e.g.) can use the ? search-operator to address outbound
content.
In other words, the URL updates to: https://me.com?https://me.com/
other.glb when navigating to https://me.com/other.glb from inside a
https://me.com WebXR experience e.g.
That way, if the link gets shared, the XR Fragments implementation at
https://me.com can load the latter (and still indicates which XR
Fragments entrypoint-experience/client was used).
2024-04-25 21:06:03 +02:00
5. Spatial Referencing 3D
XR Fragments assume the following objectname-to-URIFragment mapping:
van Kammen Expires 27 October 2024 [Page 8]
Internet-Draft XR Fragments April 2024
my.io/scene.fbx
+─────────────────────────────+
│ sky │ src: http://my.io/scene.fbx#sky (includes building,mainobject,floor)
│ +─────────────────────────+ │
│ │ building │ │ src: http://my.io/scene.fbx#building (includes mainobject,floor)
│ │ +─────────────────────+ │ │
│ │ │ mainobject │ │ │ src: http://my.io/scene.fbx#mainobject (includes floor)
│ │ │ +─────────────────+ │ │ │
│ │ │ │ floor │ │ │ │ src: http://my.io/scene.fbx#floor (just floor object)
│ │ │ │ │ │ │ │
│ │ │ +─────────────────+ │ │ │
│ │ +─────────────────────+ │ │
│ +─────────────────────────+ │
+─────────────────────────────+
| Every 3D fileformat supports named 3D object, and this name allows
| URLs (fragments) to reference them (and their children objects).
Clever nested design of 3D scenes allow great ways for re-using
content, and/or previewing scenes.
For example, to render a portal with a preview-version of the scene,
create an 3D object with:
* href: https://scene.fbx
* src: https://otherworld.gltf#mainobject
| It also allows *sourceportation*, which basically means the
| enduser can teleport to the original XR Document of an src
| embedded object, and see a visible connection to the particular
| embedded object. Basically an embedded link becoming an outbound
| link by activating it.
6. List of URI Fragments
2023-09-18 11:03:18 +02:00
2024-02-02 13:49:37 +01:00
+=======================+======================================+============+============+
|fragment |type |example |info |
+=======================+======================================+============+============+
|#pos |vector3 |#pos=0.5,0,0|positions |
| | | |camera (or |
| | | |XR floor) to|
| | | |xyz-coord |
| | | |0.5,0,0, |
+-----------------------+--------------------------------------+------------+------------+
|#rot |vector3 |#rot=0,90,0 |rotates |
2024-04-25 21:06:03 +02:00
| | | |camera to |
| | | |xyz-coord |
| | | |0.5,0,0 |
+-----------------------+--------------------------------------+------------+------------+
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 9]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-02-26 12:34:21 +01:00
|Media Fragments |media fragment |#t=0,2&loop |play (and |
2024-02-16 17:34:32 +01:00
|(https://www.w3.org/TR/|(#media%20fragments%20and%20datatypes)| |loop) 3D |
|media-frags/) | | |animation |
| | | |from 0 |
2024-02-02 13:49:37 +01:00
| | | |seconds till|
| | | |2 seconds |
+-----------------------+--------------------------------------+------------+------------+
| | | |but can also|
| | | |crop, |
| | | |animate & |
2024-02-16 17:34:32 +01:00
| | | |configure |
2024-02-02 13:49:37 +01:00
| | | |uv- |
| | | |coordinates/|
| | | |shader |
| | | |uniforms |
+-----------------------+--------------------------------------+------------+------------+
2024-02-01 13:28:17 +01:00
Table 3
2024-04-25 21:06:03 +02:00
6.1. List of metadata for 3D nodes
2023-12-06 16:40:55 +01:00
2024-02-23 11:51:05 +01:00
+======+========+============+===================+================+
| key | type | example | function | existing |
| | | (JSON) | | compatibility |
+======+========+============+===================+================+
| href | string | "href": | XR teleport | custom |
| | | "b.gltf" | | property in 3D |
| | | | | fileformats |
+------+--------+------------+-------------------+----------------+
| src | string | "src": | XR embed / | custom |
| | | "#cube" | teleport | property in 3D |
| | | | | fileformats |
+------+--------+------------+-------------------+----------------+
| tag | string | "tag": | tag object (for | custom |
| | | "cubes | filter-use / XRWG | property in 3D |
| | | geo" | highlighting) | fileformats |
+------+--------+------------+-------------------+----------------+
| # | string | "#": | trigger default | custom |
| | | "#mypreset | fragment on load | property in 3D |
| | | | | fileformats |
+------+--------+------------+-------------------+----------------+
Table 4
2023-09-18 11:03:18 +02:00
2024-04-25 21:06:03 +02:00
| Supported popular compatible 3D fileformats: .gltf, .obj, .fbx,
| .usdz, .json (THREE.js), .dae and so on.
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 10]
Internet-Draft XR Fragments April 2024
2023-09-18 11:03:18 +02:00
2024-04-25 21:06:03 +02:00
6.2. Fragment-to-metadata mapping
2024-02-01 13:28:17 +01:00
These are automatic fragment-to-metadata mappings, which only trigger
2024-02-08 14:05:44 +01:00
if the 3D scene metadata matches a specific identifier:
2023-09-18 11:03:18 +02:00
2024-02-08 14:05:44 +01:00
+================+======================================+=============+=====================+=======================+
| |fragment |type |example |info |
+================+======================================+=============+=====================+=======================+
|*PRESET* |#<preset> |string |#cubes |evaluates preset |
| | | | |(#foo&bar) defined in |
| | | | |3D Object metadata |
2024-02-23 11:51:05 +01:00
| | | | |(#cubes: #foo&bar e.g.)|
| | | | |while URL-browserbar |
| | | | |reflects #cubes. Only |
| | | | |works when metadata-key|
| | | | |starts with # |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
2024-02-16 17:34:32 +01:00
|*FOCUS* |#<tag_or_objectname> |string |#person |(and show) object(s) |
2024-02-08 14:05:44 +01:00
| | | | |with tag: person or |
| | | | |name person (XRWG |
| | | | |lookup) |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|*FILTERS* |#[!][-]<tag_or_objectname>[*] |string |#person (#-person) |will reset (!), show/ |
| | | | |focus or hide (-) focus|
| | | | |object(s) with tag: |
| | | | |person or name person |
| | | | |by looking up XRWG |
| | | | |(*=including children) |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|*CAMERASWITCH* |#<cameraname> |string |#cam01 |sets camera with name |
| | | | |cam01 as active camera |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|*MATERIALUPDATE*|#<tag_or_objectname>[*]=<materialname>|string=string|#car=metallic |sets material of car to|
| | | | |material with name |
| | | | |metallic (*=including |
| | | | |children) |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
2024-02-09 18:00:22 +01:00
| | | |#soldout*=halfopacity|set material of objects|
2024-02-08 14:05:44 +01:00
| | | | |tagged with product to |
| | | | |material with name |
| | | | |metallic |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
|*VARIABLE |#<variable>=<metadata-key> |string=string|#foo=bar |sets URI Template |
|UPDATE* | | | |(https://www.rfc- |
| | | | |editor.org/rfc/rfc6570)|
2024-04-25 21:06:03 +02:00
| | | | |variable foo to the |
| | | | |value #t=0 from |
| | | | |*existing* object |
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 11]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-02-08 14:05:44 +01:00
| | | | |metadata (bar:#t=0 |
| | | | |e.g.), This allows for |
| | | | |reactive URI Template |
| | | | |(https://www.rfc- |
| | | | |editor.org/rfc/rfc6570)|
| | | | |defined in object |
| | | | |metadata elsewhere |
| | | | |(src:://m.com/ |
| | | | |cat.mp4#{foo} e.g., to |
| | | | |play media using media |
| | | | |fragment URI |
2024-02-23 11:51:05 +01:00
| | | | |(https://www.w3.org/TR/|
| | | | |media-frags/#valid- |
| | | | |uri)). NOTE: metadata-|
| | | | |key should not start |
| | | | |with # |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
2024-02-16 17:34:32 +01:00
|*ANIMATION* |#<tag_or_objectname>=<animationname> |string=string|#people=walk |assign a different |
| | | |#people=noanim |animation to object(s) |
+----------------+--------------------------------------+-------------+---------------------+-----------------------+
2024-02-02 13:49:37 +01:00
Table 5
2024-04-25 21:06:03 +02:00
6.3. media fragments and datatypes
2024-02-02 13:49:37 +01:00
| NOTE: below the word 'play' applies to 3D animations embedded in
| the 3D scene(file) *but also* media defined in src-metadata like
| audio/video-files (mp3/mp4 e.g.)
2024-04-25 21:06:03 +02:00
+===========+======================+===============+================+
| type | syntax | example | info |
+===========+======================+===============+================+
| vector2 | x,y | 2,3.0 | 2-dimensional |
| | | | vector |
+-----------+----------------------+---------------+----------------+
| vector3 | x,y,z | 2,3.0,4 | 3-dimensional |
| | | | vector |
+-----------+----------------------+---------------+----------------+
| temporal | t=x | 0 | play from 0 |
| W3C media | | | seconds to |
| fragment | | | end (and |
| | | | stop) |
+-----------+----------------------+---------------+----------------+
| temporal | t=x,y | 0,2 | play from 0 |
| W3C media | | | seconds till |
| fragment | | | 2 seconds |
| | | | (and stop) |
+-----------+----------------------+---------------+----------------+
van Kammen Expires 27 October 2024 [Page 12]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
| temporal | s=x | 1 | set playback |
| W3C media | | | speed of |
| fragment | | | audio/ |
| * | | | video/3D anim |
+-----------+----------------------+---------------+----------------+
| temporal | [-]loop | loop | enable looped |
| W3C media | | | playback of |
| fragment | | | audio/ |
| * | | | video/3D anim |
+-----------+----------------------+---------------+----------------+
| | | -loop | disable |
| | | | looped |
| | | | playback |
| | | | (does not |
| | | | affect |
| | | | playbackstate |
| | | | of media) |
+-----------+----------------------+---------------+----------------+
| vector2 | uv=u,v,uspeed,vspeed | 0,0 | set uv offset |
| | | | instantly |
| | | | (default |
| | | | speed = 1,1) |
+-----------+----------------------+---------------+----------------+
| | | +0.5,+0.5 | scroll |
| | | | instantly by |
| | | | adding 0.5 to |
| | | | the current |
| | | | uv |
| | | | coordinates |
+-----------+----------------------+---------------+----------------+
| | | 0.2,1,0.1,0.1 | scroll (lerp) |
| | | | to uv |
| | | | coordinate |
| | | | 0,2,1 with |
| | | | 0.1 units per |
| | | | second |
+-----------+----------------------+---------------+----------------+
| | | 0,0,0,+0.1 | scroll v |
| | | | coordinates |
| | | | with 0.1 |
| | | | units per |
| | | | second |
| | | | (infinitely) |
+-----------+----------------------+---------------+----------------+
| | | +0.5,+0.5 | scroll |
| | | | instantly by |
| | | | adding 0.5 to |
| | | | the current |
van Kammen Expires 27 October 2024 [Page 13]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-02-23 11:51:05 +01:00
2024-04-25 21:06:03 +02:00
| | | | uv |
| | | | coordinates |
+-----------+----------------------+---------------+----------------+
| media | u:<uniform>=<string | float | vec2 |
| parameter | | | |
| (shader | | | |
| uniform) | | | |
+-----------+----------------------+---------------+----------------+
2024-02-16 17:34:32 +01:00
2024-04-25 21:06:03 +02:00
Table 6
2024-02-16 17:34:32 +01:00
2024-02-02 13:49:37 +01:00
| * = this is extending the W3C media fragments
2024-02-09 18:00:22 +01:00
| (https://www.w3.org/TR/media-frags/#mf-advanced) with (missing)
| playback/viewport-control. Normally #t=0,2 implies setting start/
| stop-values AND starting playback, whereas #s=0&loop allows
| pausing a video, speeding up/slowing down media, as well as
| enabling/disabling looping.
|
| The rationale for uv is that the xywh Media Fragment deals with
| rectangular media, which does not translate well to 3D models
| (which use triangular polygons, not rectangular) positioned by uv-
2024-02-23 11:51:05 +01:00
| coordinates. This also explains the absense of a scale or rotate
| primitive, which is challenged by this, as well as multiple
| origins (mesh- or texture).
2024-02-08 14:05:44 +01:00
2024-02-05 11:17:16 +01:00
Example URI's:
2024-02-02 13:49:37 +01:00
2024-02-09 18:00:22 +01:00
* https://images.org/credits.jpg#uv=0,0,0,+0.1 (infinite vertical
2024-02-08 14:05:44 +01:00
texturescrolling)
2024-02-23 11:51:05 +01:00
* https://video.org/organogram.mp4#t=0&loop&uv=0.1,0.1,0.3,0.3
2024-02-26 12:34:21 +01:00
(animated tween towards region in looped video)
2024-02-23 11:51:05 +01:00
* https://shaders.org/plasma.glsl#t=0&u:col2=0,1,0 (red-green shader
plasma starts playing from time-offset 0)
2024-02-02 13:49:37 +01:00
2024-02-26 12:34:21 +01:00
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 14]
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-26 12:34:21 +01:00
2024-02-02 13:49:37 +01:00
+──────────────────────────────────────────────────────────+
│ │
│ index.gltf#playall │
│ │ │
2024-02-08 14:05:44 +01:00
│ ├ # : #t=0&shared=play │ apply default XR Fragment on load (`t` plays global 3D animation timeline)
│ ├ play : #t=l:0,2 │ variable for [URI Templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570)
│ │ │
│ ├── ◻ plane (with material) │
2024-02-23 11:51:05 +01:00
│ │ └ #: #uv=0,0,0,+0.1 │ infinite texturescroll `v` of uv·coordinates with 0.1/fps
2024-02-02 13:49:37 +01:00
│ │ │
2024-02-08 14:05:44 +01:00
│ ├── ◻ plane │
2024-02-23 11:51:05 +01:00
│ │ └ src: foo.jpg#uv=0,0,0,+0.1 │ infinite texturescroll `v` of uv·coordinates with 0.1/fps
2024-02-08 14:05:44 +01:00
│ │ │
│ ├── ◻ media │
│ │ └ src: cat.mp4#t=l:2,10&uv=0.5,0.5 │ loop cat.mp4 (or mp3/wav/jpg) between 2 and 10 seconds (uv's shifted with 0.5,0.5)
│ │ │
│ └── ◻ wall │
│ ├ href: #color=blue │ updates uniform values (IFS shader e.g.)
│ ├ blue: t=0&u:col=0,0,1 │ variable for [Level1 URI Templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570)
│ └ src: ://a.com/art.glsl#{color}&{shared} │ .fs/.vs/.glsl/.wgsl etc shader [Level1 URI Template (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570)
2024-02-02 13:49:37 +01:00
│ │
│ │
+──────────────────────────────────────────────────────────+
2024-02-08 14:05:44 +01:00
> NOTE: URI Template variables are immutable and respect scope: in other words, the end-user cannot modify `blue` by entering an URL like `#blue=.....` in the browser URL, and `blue` is not accessible by the plane/media-object (however `{play}` would work).
2024-02-05 11:17:16 +01:00
2024-02-26 12:34:21 +01:00
7. Navigating 3D
2023-09-18 11:03:18 +02:00
2023-12-06 16:40:55 +01:00
+====================+=========+=============================+
| fragment | type | functionality |
+====================+=========+=============================+
| <b>#pos</b>=0,0,0 | vector3 | (re)position camera based |
| | or | on coordinates directly, or |
| | string | indirectly using objectname |
| | | (its worldposition) |
+--------------------+---------+-----------------------------+
2024-02-02 13:49:37 +01:00
| <b>#rot</b>=0,90,0 | vector3 | rotate camera |
+--------------------+---------+-----------------------------+
2023-09-18 11:03:18 +02:00
2024-02-09 18:00:22 +01:00
Table 7
2023-09-18 11:03:18 +02:00
2024-02-29 15:07:20 +01:00
» example implementation
2023-09-21 13:05:30 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/pos.js)
2024-02-29 15:07:20 +01:00
» discussion (https://github.com/coderofsalvation/xrfragment/
2023-09-21 13:05:30 +02:00
issues/5)
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 15]
Internet-Draft XR Fragments April 2024
2023-09-21 13:05:30 +02:00
1. the Y-coordinate of pos identifies the floorposition. This means
that desktop-projections usually need to add 1.5m (average person
height) on top (which is done automatically by VR/AR headsets).
2. set the position of the camera accordingly to the vector3 values
of #pos
3. rot sets the rotation of the camera (only for non-VR/AR headsets)
2024-04-25 21:06:03 +02:00
4. mediafragment t in the top-URL sets the playbackspeed and
animation-range of the global scene animation
2023-12-06 16:40:55 +01:00
5. after scene load: in case an href does not mention any pos-
coordinate, pos=0,0,0 will be assumed
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
Here's an ascii representation of a 3D scene-graph which contains 3D
2024-02-29 15:07:20 +01:00
objects ◻ and their metadata:
2023-10-12 17:04:46 +02:00
2023-09-21 13:05:30 +02:00
+────────────────────────────────────────────────────────+
│ │
│ index.gltf │
│ │ │
│ ├── ◻ buttonA │
│ │ └ href: #pos=1,0,1&t=100,200 │
│ │ │
│ └── ◻ buttonB │
│ └ href: other.fbx │ <── file─agnostic (can be .gltf .obj etc)
│ │
+────────────────────────────────────────────────────────+
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
An XR Fragment-compatible browser viewing this scene, allows the end-
user to interact with the buttonA and buttonB.
In case of buttonA the end-user will be teleported to another
location and time in the *current loaded scene*, but buttonB will
*replace the current scene* with a new one, like other.fbx, and
assume pos=0,0,0.
2023-09-07 14:06:50 +02:00
2024-02-26 12:34:21 +01:00
8. Top-level URL processing
2023-09-18 11:03:18 +02:00
2024-02-02 13:49:37 +01:00
| Example URL: ://foo/world.gltf#cube&pos=0,0,0
2023-09-18 11:03:18 +02:00
2024-02-02 13:49:37 +01:00
The URL-processing-flow for hypermedia browsers goes like this:
2023-12-06 16:40:55 +01:00
2023-09-21 13:30:14 +02:00
1. IF a #cube matches a custom property-key (of an object) in the 3D
file/scene (#cube: #......) <b>THEN</b> execute that
predefined_view.
2. IF scene operators (pos) and/or animation operator (t) are
present in the URL then (re)position the camera and/or animation-
range accordingly.
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 16]
Internet-Draft XR Fragments April 2024
2023-09-21 13:30:14 +02:00
3. IF no camera-position has been set in <b>step 1 or 2</b> update
the top-level URL with #pos=0,0,0 (example (https://github.com/co
derofsalvation/xrfragment/blob/main/src/3rd/js/three/
navigator.js#L31]]))
4. IF a #cube matches the name (of an object) in the 3D file/scene
then draw a line from the enduser('s heart) to that object (to
highlight it).
5. IF a #cube matches anything else in the XR Word Graph (XRWG) draw
wires to them (text or related objects).
2023-09-18 11:03:18 +02:00
2024-02-26 12:34:21 +01:00
9. Embedding XR content using src
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
src is the 3D version of the <a target="_blank"
href="https://www.w3.org/html/wiki/Elements/iframe">iframe</a>.
It instances content (in objects) in the current scene/asset.
2023-09-18 11:03:18 +02:00
2023-12-06 16:40:55 +01:00
+========+========+===================================================+
|fragment|type |example value |
+========+========+===================================================+
|src |string |#cube |
| |(uri, |#sometag |
| |hashtag/|#cube&-ball_inside_cube<br>#-sky&-rain<br>#- |
| |filter) |language&english<br>#price=>5<br>https://linux.org/|
| | |penguin.png` (https://linux.org/penguin.png`) |
| | |https://linux.world/distrowatch.gltf#t=1,100 |
| | |linuxapp://conference/nixworkshop/apply.gltf#- |
| | |cta&cta_apply |
| | |androidapp://page1?tutorial#pos=0,0,1&t1,100 |
| | |foo.mp3#0,0,0 |
+--------+--------+---------------------------------------------------+
2024-02-09 18:00:22 +01:00
Table 8
2023-12-06 16:40:55 +01:00
2024-02-29 15:07:20 +01:00
Here's an ascii representation of a 3D scene-graph with 3D objects ◻
which embeds remote & local 3D objects ◻ with/out using filters:
2024-02-16 17:34:32 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 17]
Internet-Draft XR Fragments April 2024
2023-09-21 13:05:30 +02:00
+────────────────────────────────────────────────────────+ +─────────────────────────+
│ │ │ │
│ index.gltf │ │ ocean.com/aquarium.fbx │
2023-12-06 16:40:55 +01:00
│ │ │ │ ├ room │
2023-09-21 13:05:30 +02:00
│ ├── ◻ canvas │ │ └── ◻ fishbowl │
│ │ └ src: painting.png │ │ ├─ ◻ bass │
│ │ │ │ └─ ◻ tuna │
│ ├── ◻ aquariumcube │ │ │
2023-12-06 16:40:55 +01:00
│ │ └ src: ://rescue.com/fish.gltf#fishbowl │ +─────────────────────────+
2023-09-21 13:05:30 +02:00
│ │ │
│ ├── ◻ bedroom │
│ │ └ src: #canvas │
│ │ │
│ └── ◻ livingroom │
│ └ src: #canvas │
│ │
+────────────────────────────────────────────────────────+
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
An XR Fragment-compatible browser viewing this scene, lazy-loads and
projects painting.png onto the (plane) object called canvas (which is
copy-instanced in the bed and livingroom).
Also, after lazy-loading ocean.com/aquarium.gltf, only the queried
2023-12-06 16:40:55 +01:00
objects fishbowl (and bass and tuna) will be instanced inside
aquariumcube.
2023-09-21 13:05:30 +02:00
Resizing will be happen accordingly to its placeholder object
aquariumcube, see chapter Scaling.
2023-09-18 11:03:18 +02:00
2023-12-06 16:40:55 +01:00
| Instead of cherrypicking a rootobject #fishbowl with src,
| additional filters can be used to include/exclude certain objects.
| See next chapter on filtering below.
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
*Specification*:
2023-09-07 14:06:50 +02:00
2023-12-06 16:40:55 +01:00
1. local/remote content is instanced by the src (filter) value (and
2023-09-22 10:08:28 +02:00
attaches it to the placeholder mesh containing the src property)
2023-12-06 16:40:55 +01:00
2. by default all objects are loaded into the instanced src (scene)
object (but not shown yet)
3. <b>local</b> src values (#... e.g.) starting with a non-negating
filter (#cube e.g.) will (deep)reparent that object (with name
cube) as the new root of the scene at position 0,0,0
4. <b>local</b> src values should respect (negative) filters
(#-foo&price=>3)
5. the instanced scene (from a src value) should be <b>scaled
2023-09-22 10:08:28 +02:00
accordingly</b> to its placeholder object or <b>scaled
relatively</b> based on the scale-property (of a geometry-less
placeholder, an 'empty'-object in blender e.g.). For more info
see Chapter Scaling.
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 18]
Internet-Draft XR Fragments April 2024
2023-12-06 16:40:55 +01:00
6. <b>external</b> src values should be served with appropriate
2023-09-22 10:08:28 +02:00
mimetype (so the XR Fragment-compatible browser will now how to
render it). The bare minimum supported mimetypes are:
2023-12-06 16:40:55 +01:00
7. src values should make its placeholder object invisible, and
2023-09-22 10:08:28 +02:00
only flush its children when the resolved content can
succesfully be retrieved (see broken links (#links))
2023-12-06 16:40:55 +01:00
8. <b>external</b> src values should respect the fallback link
2023-09-22 10:08:28 +02:00
mechanism (see broken links (#broken-links)
2023-12-06 16:40:55 +01:00
9. when the placeholder object is a 2D plane, but the mimetype is
2023-09-22 10:08:28 +02:00
3D, then render the spatial content on that plane via a stencil
buffer.
2023-12-06 16:40:55 +01:00
10. src-values are non-recursive: when linking to an external object
2023-09-22 10:08:28 +02:00
(src: foo.fbx#bar), then src-metadata on object bar should be
ignored.
2023-12-06 16:40:55 +01:00
11. an external src-value should always allow a sourceportation icon
within 3 meter: teleporting to the origin URI to which the
object belongs.
12. when only one object was cherrypicked (#cube e.g.), set its
2023-09-22 10:08:28 +02:00
position to 0,0,0
2023-12-06 16:40:55 +01:00
13. when the enduser clicks an href with #t=1,0,0 (play) will be
applied to all src mediacontent with a timeline (mp4/mp3 e.g.)
14. a non-euclidian portal can be rendered for flat 3D objects
(using stencil buffer e.g.) in case ofspatial src-values (an
object #world3 or URL world3.fbx e.g.).
2023-09-18 11:03:18 +02:00
2023-12-06 16:40:55 +01:00
* model/gltf-binary
2023-09-18 11:03:18 +02:00
* model/gltf+json
* image/png
* image/jpg
2023-12-06 16:40:55 +01:00
* text/plain;charset=utf-8
2023-09-21 13:30:14 +02:00
2024-02-29 15:07:20 +01:00
» example implementation
2023-09-22 10:08:28 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/src.js)
2024-02-29 15:07:20 +01:00
» example 3D asset
2023-09-21 13:05:30 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
assets/src.gltf#L192)
2024-02-29 15:07:20 +01:00
» discussion (https://github.com/coderofsalvation/xrfragment/
2023-09-21 13:05:30 +02:00
issues/4)
2023-09-07 14:06:50 +02:00
2024-02-26 12:34:21 +01:00
10. Navigating content href portals
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
navigation, portals & mutations
2023-09-07 14:06:50 +02:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 19]
Internet-Draft XR Fragments April 2024
2023-09-18 11:03:18 +02:00
+==========+==================+============================+
| fragment | type | example value |
+==========+==================+============================+
| href | string (uri or | #pos=1,1,0 |
| | predefined view) | #pos=1,1,0&rot=90,0,0 |
| | | ://somefile.gltf#pos=1,1,0 |
+----------+------------------+----------------------------+
2024-02-09 18:00:22 +01:00
Table 9
2023-09-18 11:03:18 +02:00
2023-10-12 17:04:46 +02:00
1. clicking an outbound ''external''- or ''file URI'' fully replaces
the current scene and assumes pos=0,0,0&rot=0,0,0 by default
(unless specified)
2023-09-18 11:03:18 +02:00
2. relocation/reorientation should happen locally for local URI's
(#pos=....)
3. navigation should not happen ''immediately'' when user is more
2024-04-25 21:06:03 +02:00
than 5 meter away from the portal/object containing the href (to
2023-09-18 11:03:18 +02:00
prevent accidental navigation e.g.)
2024-04-25 21:06:03 +02:00
4. URL navigation should always be reflected in the client URL-bar
(in case of javascript: see [here
2023-09-18 11:03:18 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/dev/src/3rd/
2024-04-25 21:06:03 +02:00
js/three/navigator.js) for an example navigator), and only update
the URL-bar after the scene (default fragment #) has been loaded.
2023-09-18 11:03:18 +02:00
2024-04-25 21:06:03 +02:00
5. In immersive XR mode, the navigator back/forward-buttons should
be always visible (using a wearable e.g., see [here
2023-09-21 13:05:30 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/dev/example/
aframe/sandbox/index.html#L26-L29) for an example wearable)
2024-04-25 21:06:03 +02:00
6. make sure that the ''back-button'' of the ''browser-history''
always refers to the previous position (see [here (https://github
.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/xrf/
href.js#L97))
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
7. ignore previous rule in special cases, like clicking an href
using camera-portal collision (the back-button could cause a
teleport-loop if the previous position is too close)
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
8. href-events should bubble upward the node-tree (from children to
ancestors, so that ancestors can also contain an href), however
only 1 href can be executed at the same time.
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
9. the end-user navigator back/forward buttons should repeat a back/
forward action until a pos=... primitive is found (the stateless
xrf:// href-values should not be pushed to the url-history)
2024-02-26 12:34:21 +01:00
2023-09-22 10:08:28 +02:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 20]
Internet-Draft XR Fragments April 2024
2023-09-22 10:08:28 +02:00
2024-02-16 17:34:32 +01:00
2024-02-29 15:07:20 +01:00
» example implementation
2023-09-21 13:05:30 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/href.js)
2024-02-29 15:07:20 +01:00
» example 3D asset
2023-09-21 13:05:30 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
assets/href.gltf#L192)
2024-02-29 15:07:20 +01:00
» discussion (https://github.com/coderofsalvation/xrfragment/
2023-09-21 13:05:30 +02:00
issues/1)
2024-02-26 12:34:21 +01:00
10.1. Walking surfaces
2023-12-06 16:40:55 +01:00
XR Fragment-compatible viewers can infer this data based scanning the
scene for:
1. materialless (nameless & textureless) mesh-objects (without src
and href)
| optionally the viewer can offer thumbstick, mouse or joystick
| teleport-tools for non-roomscale VR/AR setups.
2024-02-26 12:34:21 +01:00
10.2. UX spec
2023-09-21 13:05:30 +02:00
End-users should always have read/write access to:
1. the current (toplevel) <b>URL</b> (an URLbar etc)
2. URL-history (a <b>back/forward</b> button e.g.)
3. Clicking/Touching an href navigates (and updates the URL) to
another scene/file (and coordinate e.g. in case the URL contains
XR Fragments).
2024-02-26 12:34:21 +01:00
10.3. Scaling instanced content
2023-09-21 13:05:30 +02:00
Sometimes embedded properties (like src) instance new objects.
But what about their scale?
How does the scale of the object (with the embedded properties)
impact the scale of the referenced content?
| Rule of thumb: visible placeholder objects act as a '3D canvas'
| for the referenced scene (a plane acts like a 2D canvas for images
| e, a cube as a 3D canvas e.g.).
1. <b>IF</b> an embedded property (src e.g.) is set on an non-empty
placeholder object (geometry of >2 vertices):
* calculate the <b>bounding box</b> of the ''placeholder'' object
(maxsize=1.4 e.g.)
* hide the ''placeholder'' object (material e.g.)
* instance the src scene as a child of the existing object
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 21]
Internet-Draft XR Fragments April 2024
2023-12-06 16:40:55 +01:00
* calculate the <b>bounding box</b> of the instanced scene, and
scale it accordingly (to 1.4 e.g.)
2023-09-21 13:05:30 +02:00
| REASON: non-empty placeholder object can act as a protective
| bounding-box (for remote content of which might grow over time
| e.g.)
2. ELSE multiply the scale-vector of the instanced scene with the
scale-vector (a common property of a 3D node) of the
<b>placeholder</b> object.
2023-09-21 13:30:14 +02:00
| TODO: needs intermediate visuals to make things more obvious
2024-02-26 12:34:21 +01:00
11. XR Fragment: pos
2023-09-21 13:30:14 +02:00
2024-02-26 12:34:21 +01:00
12. XR Fragment: rot
2023-09-21 13:30:14 +02:00
2024-02-26 12:34:21 +01:00
13. XR Fragment: t
2023-09-21 13:05:30 +02:00
2023-12-06 16:40:55 +01:00
controls the animation(s) of the scene (or src resource which
contains a timeline)
2023-09-21 13:05:30 +02:00
2023-12-06 16:40:55 +01:00
| fragment | type | functionality | | <b>#t</b>=1,1,100 |
[vector3|vector] (default:`#t=1,0,0`) | speed,framestart,framestop |
2023-09-21 13:05:30 +02:00
2023-12-06 16:40:55 +01:00
* playposition is reset to framestart, when framestart or framestop
is greater than 0 |
2023-09-21 13:05:30 +02:00
2023-12-06 16:40:55 +01:00
| Example Value | Explanation | |-|-| | 1,1,100 | play loop between
frame 1 and 100 | | 1,1,0 | play once from frame 1 (oneshot) | |
1,0,0 | play (previously set looprange if any) | | 0,0,0 | pause | |
1,1,1 | play and auto-loop between begin and end of duration | |
-1,0,0 | reverse playback speed | | 2.3,0,0 | set (forward) playback
speed to 2.3 (no restart) | | -2.3,0,0 | set (reverse) playback speed
to -2.3 ( no restart)| | -2.3,100,0 | set (reverse) playback speed to
-2.3 restarting from frame 100 |
[[» example implementation|https://github.com/coderofsalvation/xrfrag
ment/blob/main/src/3rd/js/three/xrf/t.js]
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/t.js])]
[[» discussion|https://github.com/coderofsalvation/xrfragment/
issues/10] (https://github.com/coderofsalvation/xrfragment/
issues/10])]
2023-09-21 13:05:30 +02:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 22]
Internet-Draft XR Fragments April 2024
2024-02-26 12:34:21 +01:00
14. XR audio/video integration
2024-02-16 17:34:32 +01:00
2024-02-26 12:34:21 +01:00
To play global audio/video items:
2023-09-22 10:08:28 +02:00
2023-12-06 16:40:55 +01:00
1. add a src: foo.mp3 or src: bar.mp4 metadata to a 3D object (cube
e.g.)
2. to disable auto-play and global timeline ([[#t=|t]]) control:
hardcode a [[#t=|t]] XR Fragment: (src: bar.mp3#t=0,0,0 e.g.)
3. to play it, add href: #cube somewhere else
4. when the enduser clicks the href, #t=1,0,0 (play) will be applied
to the src value
5. to play a single animation, add href: #animationname=1,0,0
somewhere else
2023-09-21 13:30:14 +02:00
2023-12-06 16:40:55 +01:00
| NOTE: hardcoded framestart/framestop uses sampleRate/fps of
| embedded audio/video, otherwise the global fps applies. For more
| info see [[#t|t]].
2024-02-26 12:34:21 +01:00
15. XR Fragment filters
2023-12-06 16:40:55 +01:00
Include, exclude, hide/shows objects using space-separated strings:
+====================+===========================================+
| example | outcome |
+====================+===========================================+
| #-sky | show everything except object named sky |
+--------------------+-------------------------------------------+
| #-language&english | hide everything with tag language, but |
| | show all tag english objects |
+--------------------+-------------------------------------------+
| #-price&price=>10 | hide all objects with property price, |
| | then only show object with price above 10 |
+--------------------+-------------------------------------------+
2024-02-09 18:00:22 +01:00
Table 10
2023-12-06 16:40:55 +01:00
It's simple but powerful syntax which allows filtering the scene
using searchengine prompt-style feeling:
2023-09-21 13:30:14 +02:00
2023-12-06 16:40:55 +01:00
1. filters are a way to traverse a scene, and filter objects based
on their name, tag- or property-values.
2023-09-18 11:03:18 +02:00
2023-12-06 16:40:55 +01:00
* see an (outdated) example video here
(https://coderofsalvation.github.io/xrfragment.media/queries.mp4)
which used a dedicated q= variable (now deprecated and usable
directly)
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 23]
Internet-Draft XR Fragments April 2024
2024-02-26 12:34:21 +01:00
15.1. including/excluding
2023-12-06 16:40:55 +01:00
By default, selectors work like photoshop-layers: they scan for
matching layer(name/properties) within the scene-graph. Each matched
object (not their children) will be toggled (in)visible when
selecting.
+==========+==============================================+
| operator | info |
+==========+==============================================+
| - | hides object(s) (#-myobject&-objects e.g. |
+----------+----------------------------------------------+
| = | indicates an object-embedded custom property |
| | key/value (#price=4&category=foo e.g.) |
+----------+----------------------------------------------+
| => =< | compare float or int number (#price=>4 e.g.) |
+----------+----------------------------------------------+
| * | deepselect: automatically select children of |
| | selected object, including local (nonremote) |
| | embedded objects (starting with #) |
+----------+----------------------------------------------+
2024-02-09 18:00:22 +01:00
Table 11
2023-12-06 16:40:55 +01:00
| NOTE 1: after an external embedded object has been instanced (src:
| https://y.com/bar.fbx#room e.g.), filters do not affect them
| anymore (reason: local tag/name collisions can be mitigated
| easily, but not in case of remote content).
|
| NOTE 2: depending on the used 3D framework, toggling objects
| (in)visible should happen by enabling/disableing writing to the
| colorbuffer (to allow children being still visible while their
| parents are invisible).
2024-02-29 15:07:20 +01:00
» example implementation
2023-09-18 11:03:18 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
2024-02-29 15:07:20 +01:00
three/xrf/q.js) » example 3D asset
2023-09-18 11:03:18 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
2024-02-29 15:07:20 +01:00
assets/filter.gltf#L192) » discussion
2023-09-21 13:05:30 +02:00
(https://github.com/coderofsalvation/xrfragment/issues/3)
2023-09-18 11:03:18 +02:00
2024-02-26 12:34:21 +01:00
15.2. Filter Parser
2023-09-07 15:53:32 +02:00
2023-12-06 16:40:55 +01:00
Here's how to write a filter parser:
2023-09-07 15:53:32 +02:00
2023-12-06 16:40:55 +01:00
1. create an associative array/object to store filter-arguments as
2023-09-21 13:05:30 +02:00
objects
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 24]
Internet-Draft XR Fragments April 2024
2023-12-06 16:40:55 +01:00
2. detect object id's & properties foo=1 and foo (reference regex=
~/^.*=[><=]?/ )
3. detect excluders like -foo,-foo=1,-.foo,-/foo (reference regex=
/^-/ )
4. detect root selectors like /foo (reference regex= /^[-]?\// )
5. detect number values like foo=1 (reference regex= /^[0-9\.]+$/ )
6. detect operators so you can easily strip keys (reference regex=
/(^-|\*$)/ )
7. detect exclude keys like -foo (reference regex= /^-/ )
8. for every filter token split string on =
9. and we set root to true or false (true=/ root selector is
2023-09-21 13:05:30 +02:00
present)
2023-12-06 16:40:55 +01:00
10. therefore we we set show to true or false (false=excluder -)
2023-09-07 15:53:32 +02:00
2023-12-06 16:40:55 +01:00
| An example filter-parser (which compiles to many languages) can be
2023-09-21 13:05:30 +02:00
| found here
| (https://github.com/coderofsalvation/xrfragment/blob/main/src/
2023-12-06 16:40:55 +01:00
| xrfragment/Filter.hx)
2023-09-07 15:53:32 +02:00
2024-02-26 12:34:21 +01:00
16. Visible links
2023-09-21 13:30:14 +02:00
When predefined views, XRWG fragments and ID fragments (#cube or
#mytag e.g.) are triggered by the enduser (via toplevel URL or
clicking href):
1. draw a wire from the enduser (preferabbly a bit below the camera,
heartposition) to object(s) matching that ID (objectname)
2023-09-21 13:05:30 +02:00
2. draw a wire from the enduser (preferabbly a bit below the camera,
heartposition) to object(s) matching that tag value
3. draw a wire from the enduser (preferabbly a bit below the camera,
heartposition) to object(s) containing that in their src or href
value
2023-09-21 13:30:14 +02:00
The obvious approach for this, is to consult the XRWG (example
(https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
src/3rd/js/XRWG.js)), which basically has all these things already
collected/organized for you during scene-load.
*UX*
4. do not update the wires when the enduser moves, leave them as is
5. offer a control near the back/forward button which allows the
user to (turn off) control the correlation-intensity of the XRWG
2023-09-21 13:05:30 +02:00
2024-02-26 12:34:21 +01:00
17. Text in XR (tagging,linking to spatial objects)
2023-09-09 11:30:03 +02:00
2023-09-18 11:03:18 +02:00
How does XR Fragments interlink text with objects?
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 25]
Internet-Draft XR Fragments April 2024
2023-09-18 11:03:18 +02:00
| The XR Fragments does this by collapsing space into a *Word Graph*
2023-09-21 13:30:14 +02:00
| (the *XRWG* example
| (https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
| src/3rd/js/XRWG.js)), augmented by Bib(s)Tex.
2023-09-18 11:03:18 +02:00
Instead of just throwing together all kinds media types into one
2023-09-21 13:05:30 +02:00
experience (games), what about their tagged/semantical relationships?
Perhaps the following question is related: why is HTML adopted less
2024-04-25 21:06:03 +02:00
in games outside the browser?
2023-09-09 11:30:03 +02:00
2023-09-18 11:03:18 +02:00
Hence:
2023-09-09 11:30:03 +02:00
2024-04-25 21:06:03 +02:00
1. XR Fragments promotes (de)serializing a scene to a (lowercase)
XRWG (example
(https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
src/3rd/js/XRWG.js))
2. XR Fragments primes the XRWG, by collecting words from the tag
and name-property of 3D objects.
3. XR Fragments primes the XRWG, by collecting words from *optional*
metadata *at the end of content* of text (see default mimetype &
Data URI)
4. The XRWG should be recalculated when textvalues (in src) change
5. HTML/RDF/JSON is still great, but is beyond the XRWG-scope (they
fit better in the application-layer, or as embedded src content)
6. Applications don't have to be able to access the XRWG
programmatically, as they can easily generate one themselves by
traversing the scene-nodes.
7. The XR Fragment focuses on fast and easy-to-generate end-user
controllable word graphs (instead of complex implementations that
try to defeat word ambiguity)
8. Tags are the scope for now (supporting https://github.com/WICG/
scroll-to-text-fragment (https://github.com/WICG/scroll-to-text-
fragment) will be considered)
Example of generating BiBTex out of the XRWG and textdata with
hashtags:
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 26]
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2023-09-21 13:30:14 +02:00
2023-09-21 13:05:30 +02:00
http://y.io/z.fbx | Derived XRWG (expressed as BibTex)
2023-09-18 11:03:18 +02:00
----------------------------------------------------------------------------+--------------------------------------
| @house{castle,
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | url = {https://y.io/z.fbx#castle}
| Chapter one | | / \ | | }
| | | / \ | | @baroque{castle,
| John built houses in baroque style. | | / \ | | url = {https://y.io/z.fbx#castle}
| | | |_____| | | }
2024-04-25 21:06:03 +02:00
| | +-----│-----+ | @baroque{john}
2023-09-18 11:03:18 +02:00
| | │ |
| | ├─ name: castle |
| | └─ tag: house baroque |
+----------------------------------------+ |
[3D mesh ] |
| O ├─ name: john |
| /|\ | |
| / \ | |
+--------+ |
| the #john@baroque-bib associates both text John and objectname
| john, with tag baroque
2024-04-25 21:06:03 +02:00
Another example of deriving a graphdata from the XRWG:
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
http://y.io/z.fbx | Derived XRWG (expressed as BibTex)
2023-09-18 11:03:18 +02:00
----------------------------------------------------------------------------+--------------------------------------
|
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | @house{castle,
| Chapter one | | / \ | | url = {https://y.io/z.fbx#castle}
| | | / \ | | }
| John built houses in baroque style. | | / \ | | @baroque{castle,
| | | |_____| | | url = {https://y.io/z.fbx#castle}
| #john@baroque | +-----│-----+ | }
| @baroque{john} | │ | @baroque{john}
| | ├─ name: castle |
| | └─ tag: house baroque |
+----------------------------------------+ | @house{baroque}
[3D mesh ] | @todo{baroque}
+-[remotestorage.io / localstorage]------+ | O + name: john |
| #baroque@todo@house | | /|\ | |
| ... | | / \ | |
+----------------------------------------+ +--------+ |
| both #john@baroque-bib and BibTex @baroque{john} result in the
| same XRWG, however on top of that 2 tages (house and todo) are now
| associated with text/objectname/tag 'baroque'.
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 27]
Internet-Draft XR Fragments April 2024
2023-09-21 13:05:30 +02:00
As seen above, the XRWG can expand bibs
(https://github.com/coderofsalvation/hashtagbibs) (and the whole
scene) to BibTeX.
This allows hasslefree authoring and copy-paste of associations *for
and by humans*, but also makes these URLs possible:
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
+==================+======================================+
| URL example | Result |
+==================+======================================+
| https://my.com/ | draws lines between mesh john, 3D |
| foo.gltf#baroque | mesh castle, text John built(..) |
+------------------+--------------------------------------+
| https://my.com/ | draws lines between mesh john, and |
| foo.gltf#john | the text John built (..) |
+------------------+--------------------------------------+
| https://my.com/ | draws lines between mesh castle, and |
| foo.gltf#house | other objects with tag house or todo |
+------------------+--------------------------------------+
2023-09-18 11:03:18 +02:00
2024-02-09 18:00:22 +01:00
Table 12
2023-09-18 11:03:18 +02:00
| hashtagbibs (https://github.com/coderofsalvation/hashtagbibs)
| potentially allow the enduser to annotate text/objects by
| *speaking/typing/scanning associations*, which the XR Browser
| saves to remotestorage (or localStorage per toplevel URL). As
| well as, referencing BibTags per URI later on: https://y.io/
| z.fbx#@baroque@todo e.g.
The XRWG allows XR Browsers to show/hide relationships in realtime at
various levels:
* wordmatch *inside* src text
* wordmatch *inside* href text
* wordmatch object-names
* wordmatch object-tagnames
2023-09-09 11:30:03 +02:00
2023-09-21 13:05:30 +02:00
Spatial wires can be rendered between words/objects etc.
2023-09-18 11:03:18 +02:00
Some pointers for good UX (but not necessary to be XR Fragment
compatible):
9. The XR Browser needs to adjust tag-scope based on the endusers
2023-09-06 15:13:36 +02:00
needs/focus (infinite tagging only makes sense when environment
is scaled down significantly)
2023-09-18 11:03:18 +02:00
10. The XR Browser should always allow the human to view/edit the
2023-09-06 15:13:36 +02:00
metadata, by clicking 'toggle metadata' on the 'back'
(contextmenu e.g.) of any XR text, anywhere anytime.
2023-09-18 11:03:18 +02:00
11. respect multi-line BiBTeX metadata in text because of the core
principle (#core-principle)
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 28]
Internet-Draft XR Fragments April 2024
2023-09-18 11:03:18 +02:00
12. Default font (unless specified otherwise) is a modern monospace
font, for maximized tabular expressiveness (see the core
principle (#core-principle)).
13. anti-pattern: hardcoupling an XR Browser with a mandatory
*markup/scripting-language* which departs from onubtrusive plain
text (HTML/VRML/Javascript) (see the core principle (#core-
principle))
14. anti-pattern: limiting human introspection, by abandoning plain
text as first tag citizen.
| The simplicity of appending metadata (and leveling the metadata-
2023-09-09 11:30:03 +02:00
| playfield between humans and machines) is also demonstrated by
| visual-meta (https://visual-meta.info) in greater detail.
2023-09-05 19:14:10 +02:00
2023-09-18 11:03:18 +02:00
Fictional chat:
<John> Hey what about this: https://my.com/station.gltf#pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> I'm checking it right now
<Sarah> I don't see everything..where's our text from yesterday?
<John> Ah wait, that's tagged with tag 'draft' (and hidden)..hold on, try this:
<John> https://my.com/station.gltf#.draft&pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> how about we link the draft to the upcoming YELLO-event?
<John> ok I'm adding #draft@YELLO
<Sarah> Yesterday I also came up with other usefull assocations between other texts in the scene:
#event#YELLO
#2025@YELLO
<John> thanks, added.
<Sarah> Btw. I stumbled upon this spatial book which references station.gltf in some chapters:
<Sarah> https://thecommunity.org/forum/foo/mytrainstory.txt
<John> interesting, I'm importing mytrainstory.txt into station.gltf
<John> ah yes, chapter three points to trainterminal_2A in the scene, cool
2024-02-26 12:34:21 +01:00
17.1. Default Data URI mimetype
2023-09-04 21:20:59 +02:00
The src-values work as expected (respecting mime-types), however:
2023-09-02 21:44:57 +02:00
2024-02-08 14:05:44 +01:00
The XR Fragment specification advices to bump the traditional default
browser-mimetype
2023-09-02 21:44:57 +02:00
text/plain;charset=US-ASCII
2023-09-08 17:01:14 +02:00
to a hashtagbib(tex)-friendly one:
2023-09-04 21:20:59 +02:00
2023-09-06 15:13:36 +02:00
text/plain;charset=utf-8;bib=^@
2023-09-04 21:20:59 +02:00
2023-09-08 17:01:14 +02:00
This indicates that:
2023-09-04 21:20:59 +02:00
2023-09-08 17:01:14 +02:00
* utf-8 is supported by default
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 29]
Internet-Draft XR Fragments April 2024
2023-09-18 11:03:18 +02:00
* lines beginning with @ will not be rendered verbatim by default
(read more (https://github.com/coderofsalvation/
hashtagbibs#hashtagbib-mimetypes))
* the XRWG should expand bibs to BibTex occurring in text
(#contactjohn@todo@important e.g.)
By doing so, the XR Browser (applications-layer) can interpret
microformats (visual-meta (https://visual-meta.info) to connect text
further with its environment ( setup links between textual/spatial
objects automatically e.g.).
2023-09-02 21:44:57 +02:00
2023-09-08 17:01:14 +02:00
| for more info on this mimetype see bibs
| (https://github.com/coderofsalvation/hashtagbibs)
2023-09-02 21:44:57 +02:00
2023-09-08 17:01:14 +02:00
Advantages:
2023-09-18 11:03:18 +02:00
* auto-expanding of hashtagbibs
(https://github.com/coderofsalvation/hashtagbibs) associations
2023-09-08 17:01:14 +02:00
* out-of-the-box (de)multiplex human text and metadata in one go
(see the core principle (#core-principle))
2023-09-09 11:30:03 +02:00
* no network-overhead for metadata (see the core principle (#core-
principle))
* ensuring high FPS: HTML/RDF historically is too 'requesty'/'parsy'
for game studios
2023-09-08 17:01:14 +02:00
* rich send/receive/copy-paste everywhere by default, metadata being
retained (see the core principle (#core-principle))
* netto result: less webservices, therefore less servers, and
overall better FPS in XR
2023-09-04 21:20:59 +02:00
| This significantly expands expressiveness and portability of human
2023-09-06 15:13:36 +02:00
| tagged text, by *postponing machine-concerns to the end of the
| human text* in contrast to literal interweaving of content and
2023-09-04 21:20:59 +02:00
| markupsymbols (or extra network requests, webservices e.g.).
For all other purposes, regular mimetypes can be used (but are not
required by the spec).
2024-02-26 12:34:21 +01:00
17.2. URL and Data URI
2024-02-16 17:34:32 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 30]
Internet-Draft XR Fragments April 2024
2023-09-02 21:44:57 +02:00
+--------------------------------------------------------------+ +------------------------+
| | | author.com/article.txt |
| index.gltf | +------------------------+
| │ | | |
| ├── ◻ article_canvas | | Hello friends. |
| │ └ src: ://author.com/article.txt | | |
2023-09-18 11:03:18 +02:00
| │ | | @book{greatgatsby |
2023-09-04 21:20:59 +02:00
| └── ◻ note_canvas | | ... |
2023-09-18 11:03:18 +02:00
| └ src:`data:welcome human\n@book{sunday...}` | | } |
2023-09-04 21:20:59 +02:00
| | +------------------------+
2023-09-02 21:44:57 +02:00
| |
+--------------------------------------------------------------+
2023-09-04 21:20:59 +02:00
The enduser will only see welcome human and Hello friends rendered
2023-09-18 11:03:18 +02:00
verbatim (see mimetype). The beauty is that text in Data URI
2023-09-09 11:30:03 +02:00
automatically promotes rich copy-paste (retaining metadata). In both
cases, the text gets rendered immediately (onto a plane geometry,
hence the name '_canvas'). The XR Fragment-compatible browser can
let the enduser access visual-meta(data)-fields after interacting
with the object (contextmenu e.g.).
2023-09-02 21:44:57 +02:00
2023-09-07 14:06:50 +02:00
| additional tagging using bibs
2023-09-08 17:01:14 +02:00
| (https://github.com/coderofsalvation/hashtagbibs): to tag spatial
2023-09-07 14:06:50 +02:00
| object note_canvas with 'todo', the enduser can type or speak
2023-09-18 11:03:18 +02:00
| #note_canvas@todo
2023-09-09 11:30:03 +02:00
2024-02-26 12:34:21 +01:00
17.3. XR Text example parser
2023-09-08 17:01:14 +02:00
2023-09-18 11:03:18 +02:00
To prime the XRWG with text from plain text src-values, here's an
example XR Text (de)multiplexer in javascript (which supports inline
bibs & bibtex):
2023-09-05 19:14:10 +02:00
2023-09-08 17:01:14 +02:00
xrtext = {
expandBibs: (text) => {
let bibs = { regex: /(#[a-zA-Z0-9_+@\-]+(#)?)/g, tags: {}}
text.replace( bibs.regex , (m,k,v) => {
tok = m.substr(1).split("@")
match = tok.shift()
if( tok.length ) tok.map( (t) => bibs.tags[t] = `@${t}{${match},\n}` )
else if( match.substr(-1) == '#' )
bibs.tags[match] = `@{${match.replace(/#/,'')}}`
else bibs.tags[match] = `@${match}{${match},\n}`
})
return text.replace( bibs.regex, '') + Object.values(bibs.tags).join('\n')
},
decode: (str) => {
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 31]
Internet-Draft XR Fragments April 2024
2023-09-08 17:01:14 +02:00
// bibtex: ↓@ ↓<tag|tag{phrase,|{ruler}> ↓property ↓end
let pat = [ /@/, /^\S+[,{}]/, /},/, /}/ ]
let tags = [], text='', i=0, prop=''
let lines = xrtext.expandBibs(str).replace(/\r?\n/g,'\n').split(/\n/)
for( let i = 0; i < lines.length && !String(lines[i]).match( /^@/ ); i++ )
text += lines[i]+'\n'
bibtex = lines.join('\n').substr( text.length )
bibtex.split( pat[0] ).map( (t) => {
try{
let v = {}
if( !(t = t.trim()) ) return
if( tag = t.match( pat[1] ) ) tag = tag[0]
if( tag.match( /^{.*}$/ ) ) return tags.push({ruler:tag})
2023-09-18 11:03:18 +02:00
if( tag.match( /}$/ ) ) return tags.push({k: tag.replace(/}$/,''), v: {}})
2023-09-08 17:01:14 +02:00
t = t.substr( tag.length )
t.split( pat[2] )
.map( kv => {
if( !(kv = kv.trim()) || kv == "}" ) return
v[ kv.match(/\s?(\S+)\s?=/)[1] ] = kv.substr( kv.indexOf("{")+1 )
})
tags.push( { k:tag, v } )
}catch(e){ console.error(e) }
})
return {text, tags}
2023-09-04 21:20:59 +02:00
},
2023-09-06 15:13:36 +02:00
encode: (text,tags) => {
let str = text+"\n"
for( let i in tags ){
let item = tags[i]
if( item.ruler ){
str += `@${item.ruler}\n`
continue;
}
str += `@${item.k}\n`
for( let j in item.v ) str += ` ${j} = {${item.v[j]}}\n`
str += `}\n`
}
return str
2023-09-04 21:20:59 +02:00
}
}
2023-09-07 14:06:50 +02:00
The above functions (de)multiplexe text/metadata, expands bibs,
2023-09-18 11:03:18 +02:00
(de)serialize bibtex and vice versa
2023-09-04 21:20:59 +02:00
| above can be used as a startingpoint for LLVM's to translate/
2023-09-06 15:13:36 +02:00
| steelman to a more formal form/language.
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 32]
Internet-Draft XR Fragments April 2024
2023-09-06 15:13:36 +02:00
str = `
hello world
2023-09-08 17:01:14 +02:00
here are some hashtagbibs followed by bibtex:
#world
#hello@greeting
#another-section#
2023-09-06 15:13:36 +02:00
@{some-section}
@flap{
asdf = {23423}
}`
var {tags,text} = xrtext.decode(str) // demultiplex text & bibtex
tags.find( (t) => t.k == 'flap{' ).v.asdf = 1 // edit tag
tags.push({ k:'bar{', v:{abc:123} }) // add tag
console.log( xrtext.encode(text,tags) ) // multiplex text & bibtex back together
2023-09-09 11:30:03 +02:00
This expands to the following (hidden by default) BibTex appendix:
2023-09-08 17:01:14 +02:00
hello world
here are some hashtagbibs followed by bibtex:
2023-09-07 14:06:50 +02:00
@{some-section}
@flap{
asdf = {1}
2023-09-06 15:13:36 +02:00
}
2023-09-08 17:01:14 +02:00
@world{world,
}
@greeting{hello,
}
@{another-section}
2023-09-07 14:06:50 +02:00
@bar{
abc = {123}
2023-09-06 15:13:36 +02:00
}
2023-09-02 21:44:57 +02:00
2023-09-08 17:01:14 +02:00
| when an XR browser updates the human text, a quick scan for
| nonmatching tags (@book{nonmatchingbook e.g.) should be performed
| and prompt the enduser for deleting them.
2024-02-26 12:34:21 +01:00
18. Transclusion (broken link) resolution
2023-09-07 14:06:50 +02:00
2023-10-12 17:04:46 +02:00
In spirit of Ted Nelson's 'transclusion resolution', there's a soft-
mechanism to harden links & minimize broken links in various ways:
2023-09-07 14:06:50 +02:00
2023-09-22 10:08:28 +02:00
1. defining a different transport protocol (https vs ipfs or DAT) in
src or href values can make a difference
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 33]
Internet-Draft XR Fragments April 2024
2023-10-12 17:04:46 +02:00
2. mirroring files on another protocol using (HTTP) errorcode tags
in src or href properties
2023-09-22 10:08:28 +02:00
3. in case of src: nesting a copy of the embedded object in the
placeholder object (embeddedObject) will not be replaced when the
request fails
2023-10-12 17:04:46 +02:00
| due to the popularity, maturity and extensiveness of HTTP codes
| for client/server communication, non-HTTP protocols easily map to
| HTTP codes (ipfs ERR_NOT_FOUND maps to 404 e.g.)
2023-09-22 10:08:28 +02:00
For example:
+────────────────────────────────────────────────────────+
│ │
│ index.gltf │
│ │ │
2024-02-08 14:05:44 +01:00
│ │ #: #-offlinetext │
2023-09-22 10:08:28 +02:00
│ │ │
│ ├── ◻ buttonA │
│ │ └ href: http://foo.io/campagne.fbx │
2023-10-12 17:04:46 +02:00
│ │ └ href@404: ipfs://foo.io/campagne.fbx │
2024-02-08 14:05:44 +01:00
│ │ └ href@400: #clienterrortext │
2023-09-22 10:08:28 +02:00
│ │ └ ◻ offlinetext │
│ │ │
│ └── ◻ embeddedObject <--------- the meshdata inside embeddedObject will (not)
│ └ src: https://foo.io/bar.gltf │ be flushed when the request (does not) succeed.
2023-10-12 17:04:46 +02:00
│ └ src@404: http://foo.io/bar.gltf │ So worstcase the 3D data (of the time of publishing index.gltf)
│ └ src@400: https://archive.org/l2kj43.gltf │ will be displayed.
2023-09-22 10:08:28 +02:00
│ │
+────────────────────────────────────────────────────────+
2024-02-26 12:34:21 +01:00
19. Topic-based index-less Webrings
2023-10-12 17:04:46 +02:00
As hashtags in URLs map to the XWRG, href-values can be used to
promote topic-based index-less webrings.
Consider 3D scenes linking to eachother using these href values:
* href: schoolA.edu/projects.gltf#math
* href: schoolB.edu/projects.gltf#math
* href: university.edu/projects.gltf#math
These links would all show visible links to math-tagged objects in
the scene.
To filter out non-related objects one could take it a step further
2023-12-06 16:40:55 +01:00
using filters:
2023-10-12 17:04:46 +02:00
2023-12-06 16:40:55 +01:00
* href: schoolA.edu/projects.gltf#math&-topics math
* href: schoolB.edu/projects.gltf#math&-courses math
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 34]
Internet-Draft XR Fragments April 2024
2023-12-06 16:40:55 +01:00
* href: university.edu/projects.gltf#math&-theme math
2023-10-12 17:04:46 +02:00
| This would hide all object tagged with topic, courses or theme
| (including math) so that later only objects tagged with math will
| be visible
This makes spatial content multi-purpose, without the need to
separate content into separate files, or show/hide things using a
complex logiclayer like javascript.
2024-02-26 12:34:21 +01:00
20. URI Templates (RFC6570)
2024-02-08 14:05:44 +01:00
XR Fragments adopts Level1 URI *Fragment* expansion to provide safe
interactivity.
The following demonstrates a simple video player:
+─────────────────────────────────────────────+
│ │
│ foo.usdz │
│ │ │
│ │ │
│ ├── ◻ stopbutton │
│ │ ├ #: #-stopbutton │
│ │ └ href: #player=stop&-stopbutton │ (stop and hide stop-button)
│ │ │
│ └── ◻ plane │
│ ├ play: #t=l:0,10 │
│ ├ stop: #t=0,0 │
│ ├ href: #player=play&stopbutton │ (play and show stop-button)
│ └ src: cat.mp4#{player} │
│ │
│ │
+─────────────────────────────────────────────+
2024-02-29 15:07:20 +01:00
21. Additional scene metadata
XR Fragments does not aim to redefine the metadata-space by
introducing its own cataloging-metadata fields. Instead, it
encourages browsers to scan nodes for the following custom
properties:
* SPDX (https://spdx.dev/) license information
* ARIA (https://www.w3.org/WAI/standards-guidelines/aria/)
attributes (aria-*: .....)
* Open Graph (https://ogp.me) attributes (og:*: .....)
* Dublin-Core (https://www.dublincore.org/specifications/dublin-
core/application-profile-guidelines/) attributes(dc:*: .....)
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 35]
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
* BibTex (https://bibtex.eu/fields) when known bibtex-keys exist
with values enclosed in { and },
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
*ARIA* (aria-description) is the most important to support, as it
promotes accessibility and allows scene transcripts. Please start
aria-description with a verb to aid transcripts.
2024-02-29 15:07:20 +01:00
| Example: object 'tryceratops' with aria-description: is a huge
| dinosaurus standing on a #mountain generates transcript
| #tryceratops is a huge dinosaurus standing on a #mountain, where
| the hashtags are clickable XR Fragments (activating the visible-
| links in the XR browser).
Individual nodes can be enriched with such metadata, but most
importantly the scene node:
+================================+=========================+
| metadata key | example value |
+================================+=========================+
| aria-description, | An immersive experience |
| og:description, dc:description | about Triceratops (*) |
+--------------------------------+-------------------------+
| SPDX | CC0-1.0 |
+--------------------------------+-------------------------+
| dc:creator | John Doe |
+--------------------------------+-------------------------+
| dc:title, og:title | 'Triceratops` (*) |
+--------------------------------+-------------------------+
| og:site_name | https://xrfragment.org |
+--------------------------------+-------------------------+
| dc.publisher | NLNET |
+--------------------------------+-------------------------+
| dc.date | 2024-01-01 |
+--------------------------------+-------------------------+
| dc.identifier | XRFRAGMENT-001 |
+--------------------------------+-------------------------+
| journal (bibTeX) | {Future Of Text Vol 3}, |
+--------------------------------+-------------------------+
Table 13
| * = these are interchangable (only one needs to be defined)
There's no silver bullet when it comes to metadata, so one should
support where the metadata is/goes.
| These attributes can be scanned and presented during an href or
| src eye/mouse-over.
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 36]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
22. Security Considerations
2024-02-08 14:05:44 +01:00
The only dynamic parts are W3C Media Fragments
(https://www.w3.org/TR/media-frags/) and URI Templates (RFC6570)
(https://www.rfc-editor.org/rfc/rfc6570).
The use of URI Templates is limited to pre-defined variables and
Level0 fragments-expansion only, which makes it quite safe.
In fact, it is much safer than relying on a scripting language
(javascript) which can change URN too.
2024-02-29 15:07:20 +01:00
23. FAQ
2023-09-06 15:13:36 +02:00
2023-09-21 13:05:30 +02:00
*Q:* Why is everything HTTP GET-based, what about POST/PUT/DELETE
HATEOS
*A:* Because it's out of scope: XR Fragment specifies a read-only way
to surf XR documents. These things belong in the application layer
(for example, an XR Hypermedia browser can decide to support
POST/PUT/DELETE requests for embedded HTML thru src values)
2023-09-01 14:20:02 +02:00
2024-02-08 14:05:44 +01:00
*Q:* Why isn't there support for scripting, URI Template Fragments
are so limited compared to WASM & javascript *A:* This is out of
scope as it unhyperifies hypermedia, and this is up to XR hypermedia
browser-extensions.
2023-10-12 17:04:46 +02:00
Historically scripting/Javascript seems to been able to turn webpages
from hypermedia documents into its opposite (hyperscripted
nonhypermedia documents).
In order to prevent this backward-movement (hypermedia tends to
2024-02-08 14:05:44 +01:00
liberate people from finnicky scripting) XR Fragment uses W3C Media
Fragments (https://www.w3.org/TR/media-frags/) and URI Templates
(RFC6570) (https://www.rfc-editor.org/rfc/rfc6570), to prevent
unhyperifying itself by hardcoupling to a particular markup or
scripting language.
2023-10-12 17:04:46 +02:00
XR Fragments supports filtering objects in a scene only, because in
the history of the javascript-powered web, showing/hiding document-
entities seems to be one of the most popular basic usecases.
Doing advanced scripting & networkrequests under the hood are
obviously interesting endavours, but this is something which should
2024-02-08 14:05:44 +01:00
not be hardcoupled with XR Fragments or hypermedia.
This perhaps belongs more to browser extensions.
2023-10-12 17:04:46 +02:00
Non-HTML Hypermedia browsers should make browser extensions the right
place, to 'extend' experiences, in contrast to code/javascript inside
hypermedia documents (this turned out as a hypermedia antipattern).
2023-09-21 13:05:30 +02:00
2024-02-29 15:07:20 +01:00
24. authors
2024-02-08 14:05:44 +01:00
* Leon van Kammen (@lvk@mastodon.online)
2024-02-29 15:07:20 +01:00
* Jens Finkhäuser (@jens@social.finkhaeuser.de)
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 37]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-02-08 14:05:44 +01:00
2024-02-29 15:07:20 +01:00
25. IANA Considerations
2023-09-08 17:01:14 +02:00
2023-09-18 11:03:18 +02:00
This document has no IANA actions.
2023-09-08 17:01:14 +02:00
2024-02-29 15:07:20 +01:00
26. Acknowledgments
2023-09-08 17:01:14 +02:00
2023-09-09 11:30:03 +02:00
* NLNET (https://nlnet.nl)
* Future of Text (https://futureoftext.org)
2023-09-07 15:53:32 +02:00
* visual-meta.info (https://visual-meta.info)
2023-12-06 16:40:55 +01:00
* Michiel Leenaars
* Gerben van der Broeke
* Mauve
2024-02-29 15:07:20 +01:00
* Jens Finkhäuser
2023-12-06 16:40:55 +01:00
* Marc Belmont
* Tim Gerritsen
* Frode Hegland
* Brandel Zackernuk
* Mark Anderson
2024-02-29 15:07:20 +01:00
27. Appendix: Definitions
2023-09-21 13:05:30 +02:00
2024-01-30 10:58:00 +01:00
+=================+=============================================+
| definition | explanation |
+=================+=============================================+
| human | a sentient being who thinks fuzzy, absorbs, |
| | and shares thought (by plain text, not |
| | markuplanguage) |
+-----------------+---------------------------------------------+
| scene | a (local/remote) 3D scene or 3D file |
| | (index.gltf e.g.) |
+-----------------+---------------------------------------------+
| 3D object | an object inside a scene characterized by |
| | vertex-, face- and customproperty data. |
+-----------------+---------------------------------------------+
| URI | some resource at something somewhere via |
| | someprotocol (http://me.com/foo.glb#foo or |
| | e76f8efec8efce98e6f see interpeer.io |
| | (https://interpeer.io)) |
+-----------------+---------------------------------------------+
| URL | something somewhere via someprotocol |
| | (http://me.com/foo.glb) |
+-----------------+---------------------------------------------+
2024-02-08 14:05:44 +01:00
| URN | something at some domain (me.com/foo.glb) |
+-----------------+---------------------------------------------+
2024-01-30 10:58:00 +01:00
| metadata | custom properties of text, 3D Scene or |
| | Object(nodes), relevant to machines and a |
| | human minority (academics/developers) |
+-----------------+---------------------------------------------+
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 38]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
2024-01-30 10:58:00 +01:00
| XR fragment | URI Fragment with spatial hints like |
| | #pos=0,0,0&t=1,100 e.g. |
+-----------------+---------------------------------------------+
| the XRWG | wordgraph (collapses 3D scene to tags) |
+-----------------+---------------------------------------------+
| the hashbus | hashtags map to camera/scene-projections |
+-----------------+---------------------------------------------+
| spacetime | positions camera, triggers scene-preset/ |
| hashtags | time |
+-----------------+---------------------------------------------+
| teleportation | repositioning the enduser to a different |
| | position (or 3D scene/file) |
+-----------------+---------------------------------------------+
| sourceportation | teleporting the enduser to the original XR |
| | Document of an src embedded object. |
+-----------------+---------------------------------------------+
| placeholder | a 3D object which with src-metadata (which |
| object | will be replaced by the src-data.) |
+-----------------+---------------------------------------------+
| src | (HTML-piggybacked) metadata of a 3D object |
| | which instances content |
+-----------------+---------------------------------------------+
| href | (HTML-piggybacked) metadata of a 3D object |
| | which links to content |
+-----------------+---------------------------------------------+
| filter | URI Fragment(s) which show/hide object(s) |
| | in a scene based on name/tag/property |
| | (#cube&-price=>3) |
+-----------------+---------------------------------------------+
| visual-meta | visual-meta (https://visual.meta.info) data |
| | appended to text/books/papers which is |
| | indirectly visible/editable in XR. |
+-----------------+---------------------------------------------+
| requestless | metadata which never spawns new requests |
| metadata | (unlike RDF/HTML, which can cause |
| | framerate-dropping, hence not used a lot in |
| | games) |
+-----------------+---------------------------------------------+
| FPS | frames per second in spatial experiences |
| | (games,VR,AR e.g.), should be as high as |
| | possible |
+-----------------+---------------------------------------------+
| introspective | inward sensemaking ("I feel this belongs to |
| | that") |
+-----------------+---------------------------------------------+
| extrospective | outward sensemaking ("I'm fairly sure John |
| | is a person who lives in oklahoma") |
+-----------------+---------------------------------------------+
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 39]
2024-02-29 15:07:20 +01:00
2024-04-25 21:06:03 +02:00
Internet-Draft XR Fragments April 2024
2024-02-29 15:07:20 +01:00
| ◻ | ascii representation of an 3D object/mesh |
2024-01-30 10:58:00 +01:00
+-----------------+---------------------------------------------+
| (un)obtrusive | obtrusive: wrapping human text/thought in |
| | XML/HTML/JSON obfuscates human text into a |
| | salad of machine-symbols and words |
+-----------------+---------------------------------------------+
| flat 3D object | a 3D object of which all verticies share a |
| | plane |
+-----------------+---------------------------------------------+
| BibTeX | simple tagging/citing/referencing standard |
| | for plaintext |
+-----------------+---------------------------------------------+
| BibTag | a BibTeX tag |
+-----------------+---------------------------------------------+
| (hashtag)bibs | an easy to speak/type/scan tagging SDL (see |
| | here (https://github.com/coderofsalvation/ |
| | hashtagbibs) which expands to BibTex/JSON/ |
| | XML |
+-----------------+---------------------------------------------+
2023-09-21 13:05:30 +02:00
2024-02-29 15:07:20 +01:00
Table 14
2023-12-06 16:40:55 +01:00
2024-02-08 14:05:44 +01:00
2024-02-26 12:34:21 +01:00
2024-04-25 21:06:03 +02:00
van Kammen Expires 27 October 2024 [Page 40]
