An open specification for hyperlinking & deeplinking 3D fileformats.
This is a specification for interactive URI-controllable 3D files, to enable spatial webs for local-first XR hypermedia browsers.<br/>
XR Fragments allows us to better use implicit metadata inside 3D scene(files), by mapping it to proven technologies like [URI Fragments](https://en.wikipedia.org/wiki/URI_fragment).<br>
> Almost every idea in this document is demonstrated at [https://xrfragment.org](https://xrfragment.org)
{mainmatter}
# Quick reference
1. [Abstract](#abstract)
1. [Index](#index)
1. [Introduction](#introduction)
1. [How does it work](#how-does-it-work)
1. [What does it solve](#what-does-it-solve)
1. [Hypermediatic Feedback Loop for XR Browsers](#hfl-hypermediatic-feedback-loop-for-xr-browsers)
1. [Conventions and Definitions](#conventions-and-definitions)
1. [XR Fragment URL Grammar](#xr-fragment-url-grammar)
> Aspect 1 was pioneered and implemented by JanusXR / James Baicoianu via [janusweb](https://github.com/jbaicoianu/janusweb) well before the formal specification was established. As a key architect of these early concepts, James’s work laid the groundwork for the standard we use today. We owe him a great deal of thanks for his early contributions.
1. addressibility and [hypermediatic](https://github.com/coderofsalvation/hypermediatic) navigation of 3D scenes/objects: [URI Fragments](https://en.wikipedia.org/wiki/URI_fragment) using src/href spatial metadata
1. unlocking spatial potential of the (originally 2D) hashtag (which jumps to a chapter) for navigating XR documents
1. refraining from introducing scripting-engines for mundane tasks (and preventing its inevitable security-headaches)
1. the gap between text an 3d objects: object-names directly map to hashtags (=fragments), which allows 3D to text transcription.
XR Fragments views XR experiences through the lens of 3D deeplinked URI's, rather than thru code(frameworks) or protocol-specific browsers (webbrowser e.g.).
To aid adoption, the standard comprises of various (optional) support-levels, which incorporate existing standards like [W3C Media Fragments](https://www.w3.org/TR/media-frags/) and [URI Templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570) to promote spatial addressibility, sharing, navigation, filtering and databinding objects for (XR) Browsers.<br>
> XR Fragments is in a sense, a <b>heuristical 3D format</b> or meta-format, which leverages heuristic rules derived from any 3D scene or well-established 3D file formats, to extract meaningful features from scene hierarchies.<br>
These heuristics, enable features that are both meaningful and consistent across different scene representations, allowing <b>higher interop</b> between fileformats, 3D editors, viewers and game-engines.
`href` metadata traditionally implies **click** AND **navigate**, however XR Fragments separates **click** via the `xrf://` scheme, which does not change the top-level URL-adress (of the browser).
This allows for many extra interactions via URLs, which otherwise needs a scripting language.
> Being able to use the same URI Fragment DSL for navigation (`href: #foo`) as well as interactions (`href: xrf://#foo`) greatly simplifies implementation, increases HFL, and reduces need for scripting languages.
* [hypermediatic](https://github.com/coderofsalvation/hypermediatic) loading/clicking 3D assets (gltf/fbx e.g.) natively (with or without using HTML).
* potentially allowing 3D assets/nodes to publish XR Fragments to themselves/eachother using the `xrf://` hash-events (`xrf://#person=walk` to trigger `walk`-animation for object `person`)
* potentially collapsing the 3D scene to an wordgraph (for essential navigation purposes) controllable thru a hash(tag)bus
* completely bypassing the security-trap of loading external scripts (by loading 3D model-files, not HTML-javascriptable resources)
XR Fragments itself are [hypermediatic](https://github.com/coderofsalvation/hypermediatic) and HTML-agnostic, though pseudo-XR Fragment browsers **can** be implemented on top of HTML/Javascript.
> An important aspect is that URI Fragments can be triggered without updating the top-level URI (default href-behaviour) thru their own 'bus' (`xrf://#.....`). This decoupling between navigation and interaction prevents non-standard things like (`href`:`javascript:dosomething()`).
Compatible 3D fileformats: [glTF](https://www.khronos.org/gltf/), [usdz](https://openusd.org/release/spec_usdz.html), [obj](https://en.wikipedia.org/wiki/Wavefront_.obj_file), [collada](https://www.khronos.org/collada), [THREE.json](https://github.com/mrdoob/three.js/wiki/JSON-Object-Scene-format-4), [X3D](https://en.wikipedia.org/wiki/X3D) e.g.
> NOTE: Linux's `setfattr/getfattr` is `xattr` on mac, and `Set-Content/Get-content` on Windows. See [pxattr](https://www.lesbonscomptes.com/pxattr/index.html) for lowlevel access.
This will make object button clickable, and teleport the user to object roomB.
<br>
> So after loading experience.glb the existence of experience.json is detected, to apply the explicit metadata. <br>The sidecar will define (or override already existing) extras, which can be handy for multi-user platforms (offer 3D scene customization/personalization to users).
XR Fragments tries to seek to connect the world of text (semantical web / RDF), and the world of pixels.<br>
Instead of forcing authors to combine 3D/2D objects programmatically (publishing thru a game-editor e.g.), XR Fragments **integrates all** which allows a universal viewing experience.<br>
Fact: our typical browser URL's are just **a possible implementation** of URI's (for untapped humancentric potential of URI's [see interpeer.io](https://interpeer.io) or [NextGraph](https://nextgraph.org) )
> XR Fragments does not look at XR (or the web) thru the lens of HTML or URLs.<br>But approaches things from a higherlevel local-first 3D hypermedia browser-perspective.
> ?-linked and #-linked navigation are JUST one possible way to implement XR Fragments: the essential goal is to allow a Hypermediatic FeedbackLoop (HFL) between external and internal 4D navigation.
| level0+1 | hover 3D file [href](#via-href-metadata) | show the preview PNG thumbnail (if any). |
| level0+1 | launch 3D file [href](#via-href-metadata) | replace the current scene with a new 3D file (`href: other.glb` e.g.) |
| level2 | click internal 3D file [href](#via-href-metadata) (`#roomB` e.g.) | teleport the camera to the origin of object(name `roomB`). See [[teleport camera]].|
| level2 | click external 3D file [href](#via-href-metadata) (`foo.glb` e.g.) | replace the current scene with a new 3D file (`href: other.glb` e.g.) |
| level2 | hover external 3D file [href](#via-href-metadata) | show the preview PNG thumbnail (if any sidecar, see level0) |
> NOTE: hash-events links (`xrf://#foo&bar`) don't change the toplevel URL, which makes it ideal for interactions (in contrast to typical `#roomC` navigation, which benefit back/forward browser-buttons), see <a href="#hash-events">hash-events</a> for more info.
> **default camera (spawn) location**: When a 3D file is loaded, it will position the user/camera at xyz: 0,0,0 OR to the location of (an object called) ''spawn''.
> How to know whether a 3D file is a full scene or just a prop? <br>When should a 3D file hide its skybox?<br>How does a file know whether its embedded or not?<br>How to hide an object in AR-mode only?<br> How to ''hide'' an object by default, after loading a scene?
Any element without any of these class-values is considered visible in all layers. Once class-value(s) are detected it'll work as a layer-mask: `class: 'vr ar'` will be visible in desktop-,VR- and AR-mode.
> **NOTE**: there's is no CSS: class attributes without CSS means **no cascade inheritance**; above class-definitions will apply to the element itself (and its children, which can override by defining their own XR layer class-definitions).
> **Examples:** `#+menu` to show a object, `#-menu` to hide a menu, `#!menu` to teleport a menu, `#*block` to clone a grabbable block, `#|object` to share an object
This tiny but powerful symbol allows incredible interactive possibilities, by carefully positioning re-usable objects outside of a scene (below the usercamera's floor e.g.).
> **NOTE**: level2 teleportation links, as well as instancing mitigates the 'broken embedded image'-issue of HTML: **always** attaching the href-values to **a 3D (preview) object** (that way broken links will not break the design).
**Example:** clicking a 3D button with title 'menu' and [href](#href)-value `xrf:menu.glb?instance#t=4,5` would instance a 3D menu (`menu.glb`) in front of the user, and loop its animation between from 4-5 seconds (`t=4,5`)
> NOTE: to hide a skybox when importing/loading a 3D file (force AR) is possible by linking to ``https://my.org/foo.glb#-skybox` or `https://my.org/foo.glb#-skyboxmaterial`
> This URL can be fed straight into [Web Share API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Share_API) or [xdg-open](https://www.freedesktop.org/wiki/Software/xdg-utils/)
Prefixing the `xrf:` to [href](#href)-values **will prevent** [level2](#📜%20level2:%20explicit%20links) [href](#href)-values from changing the top-Level URL.
1. IF scene operators and/or animation operator (`t`) are present in the URL then (re)position the camera (to `room1`) and/or animation-range (`10`) accordingly.
2. IF no camera-position has been set in <b>step 1 or 2</b> assume `0,0,0` as camera coordinate (XR: add user-height) ([example](https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/navigator.js#L31]]))
3. IF a camera-object exists with name `cam` assume that user(camera) position
3. navigation should not happen ''immediately'' when user is more than 5 meter away from the portal/object containing the href (to prevent accidental navigation e.g.)
4. URL navigation should always be reflected in the client URL-bar (in case of javascript: see [[here](https://github.com/coderofsalvation/xrfragment/blob/dev/src/3rd/js/three/navigator.js) for an example navigator), and only update the URL-bar after the scene (default fragment `#`) has been loaded.
5. In immersive XR mode, the navigator back/forward-buttons should be always visible (using a wearable e.g., see [[here](https://github.com/coderofsalvation/xrfragment/blob/dev/example/aframe/sandbox/index.html#L26-L29) for an example wearable)
6. make sure that the ''back-button'' of the ''browser-history'' always refers to the previous position (see [[here](https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/xrf/href.js#L97))
7. ignore previous rule in special cases, like clicking an `href` using camera-portal collision (the back-button could cause a teleport-loop if the previous position is too close)
8. href-events should bubble upward the node-tree (from children to ancestors, so that ancestors can also conain an href), however only 1 href can be executed at the same time.
9. the end-user navigator back/forward buttons should repeat a back/forward action until a `#...` primitive is found (the stateless xrf:// href-values should not be pushed to the url-history)
* [RDF/JSON-LD](https://json-ld.org) like [this example](https://mvmd.org/standards/gltf/) or via glTF's [KHR_xmp_json_ld extension](https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Khronos/KHR_xmp_json_ld)
> Example: object 'tryceratops' with `aria-description: is a huge dinosaurus standing on a #mountain` generates transcript `#tryceratops is a huge dinosaurus standing on a #mountain`, where the hashtags are clickable XR Fragments (activating the visible-links in the XR browser).
1. The enduser must be able to enable an accessibility-mode (which persists across application/webpage restarts)
2. Accessibility-mode must contain a text-input for the user to enter text
3. Accessibility-mode must contain a flexible textlog for the user to read (via screenreader, screen, or TTS e.g.)
4. the textlog contains `aria-descriptions`, and its narration (Screenreader e.g.) can be skipped (via 2-button navigation)
5. The `back` command should navigate back to the previous URL (alias for browser-backbutton)
6. The `forward` command should navigate back to the next URL (alias for browser-nextbutton)
7. A destination is a 3D node containing an `href` with a `#...` XR fragment (which matches a 3d object name)
8. The `go` command should list all possible destinations
9. The `go left` command should move the camera around 0.3 meters to the left
10. The `go right` command should move the camera around 0.3 meters to the right
11. The `go forward` command should move the camera 0.3 meters forward (direction of current rotation).
12. The `rotate left` command should rotate the camera 0.3 to the left
13. The `rotate left` command should rotate the camera 0.3 to the right
14. The (dynamic) `go abc` command should navigate to `#scene2` in case there's a 3D node with name `abc` and `href` value `#scene2`
15. The `look` command should give an (contextual) 3D-to-text transcript, by scanning the `aria-description` values of the current `#...` (3D object) value (including its children)
16. The `do` command should list all possible `href` values which don't contain an `#...` XR Fragment
17. The (dynamic) `do abc` command should navigate/execute `https://.../...` in case a 3D node exist with name `abc` and `href` value `https://.../...`
3D engines (games/apps) can render these like clickable portals, for example JanusXR-style portals:
> NOTE: for more customizability [JML <link>](https://coderofsalvation.github.io/janus-guide/#/build/room?id=link) will allow for positioning/thumbnailed portals:
This way the application will load `https://3d.john.foo.com/model/scene.glb` when the user types `john@foo.com` into the user field.
## via Text (URI)
As a fallback, for an application to trigger loading a 3D scene is by detecting URI's of 3D scene-files any text:
* `foo.glb` (or any other popular 3D extension)
* `https://foo.com/scene.glb` (or any other popular protocol)
This way, the application can highlight the link whenever it detects the URI (in a text-file or text-section of a 3D model)
## Overlap with fileformat-specific extensions
Some 3D scene-fileformats have support for extensions.
What if the functionality of those overlap?
For example, GLTF has the `OMI_LINK` extension which might overlap with XR Fragment's `href`:
> Priority Order and Precedence, otherwise fallback applies
1.**Extensions Take Precedence**: Since glTF-specific extensions are designed with the format’s
specific needs and optimizations in mind, they should take precedence over extras metadata
in cases where both contain overlapping functionality.
This approach aligns with the idea that extensions are more likely to be interpreted uniformly by glTF-compatible software.
2. **Fallback Fall-through Mechanism**:
If a glTF implementation does not support a particular extension, the (XRF) extras field can serve as a fallback. This way, metadata provided in extras can still be useful for applications that don't handle certain extensions.
> **Example 1** In case of the OMI_LINK glTF extension (`href: https://nlnet.nl`) and an XR Fragment (`href: #otherroom` or `href: otherplanet.glb`), it is clear that `https://nlnet.nl` should open in a browsertab, whereas the XR Fragment links should teleport the user. If the OMI_LINK contains an XR Fragment (`#room1` e.g.) a teleport should be performed only (and other [overlapping] metadata should be ignored).
> **Example 2** If an Extensions uses XR Fragments in URI's (`href: #otherroom` or `href: xrf://-walls` in OMI_LINK e.g.), then perform them according to XR Fragment spec (teleport user). But only once: ignore further overlapping metadata for that usecase.
# Vendor Prefixes
Vendor-specific metadata in a 3D scenefiles, are similar to vendor-specific [CSS-prefixes](https://en.wikipedia.org/wiki/CSS#Vendor_prefixes) (`-moz-opacity: 0.2` e.g.).
This allows popular 3D engines/frameworks, to initialize specific features when loading a scene/object, in a progressive enhanced way.
Vendor Prefixes allows embedding 3D engines/framework-specific features a 3D file via metadata:
| what | XR metadata | Lowest common denominator |
> Why? Because not all XR interactions can/should be solved/standardized by embedding XR Fragments into any 3D file.
The lowest common denominator between 3D engines is the 'entity'-part of their entity-component-system (ECS). The 'component'-part can be progressively enhanced via vendor prefixes.
For example, the following metadata can be added to a .glb file, to make an object grabbable in AFRAME:
> This 'separating of mechanism from policy' (unix rule) does **somewhat** break portability of an XR experience, but still prevents (E-waste of) handcoded virtual worlds. It allows for (XR experience) metadata to survive in future 3D engines and scene-fileformats.
The only dynamic parts are [W3C Media Fragments](https://www.w3.org/TR/media-frags/) and [URI Templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570).<br>
The use of URI Templates is limited to pre-defined variables and Level0 fragments-expansion only, which makes it quite safe.<br>
n fact, it is much safer than relying on a scripting language (javascript) which can change URN too.
**Q:** Why is everything HTTP GET-based, what about POST/PUT/DELETE HATEOS<br>
**A:** Because it's out of scope: XR Fragment specifies a read-only way to surf XR documents. These things belong in the application layer (for example, an XR Hypermedia browser can decide to support POST/PUT/DELETE requests for embedded HTML thru `src` values)
**Q:** Why isn't there support for scripting, URI Template Fragments are so limited compared to WASM & javascript
**A:** This is out of scope as it unhyperifies hypermedia, and this is up to XR hypermedia browser-extensions.<br> Historically scripting/Javascript seems to been able to turn webpages from hypermedia documents into its opposite (hyperscripted nonhypermedia documents).<br>In order to prevent this backward-movement (hypermedia tends to liberate people from finnicky scripting) XR Fragment uses [W3C Media Fragments](https://www.w3.org/TR/media-frags/) and [URI Templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570), to prevent unhyperifying itself by hardcoupling to a particular markup or scripting language. <br>
XR Fragments supports filtering objects in a scene only, because in the history of the javascript-powered web, showing/hiding document-entities seems to be one of the most popular basic usecases.<br>
Doing advanced scripting & networkrequests under the hood are obviously interesting endavours, but this is something which should not be hardcoupled with XR Fragments or hypermedia.<br>This perhaps belongs more to browser extensions.<br>
Non-HTML Hypermedia browsers should make browser extensions the right place, to 'extend' experiences, in contrast to code/javascript inside hypermedia documents (this turned out as a hypermedia antipattern).
|human | a sentient being who thinks fuzzy, absorbs, and shares thought (by plain text, not markuplanguage) |
|scene | a (local/remote) 3D scene or 3D file (index.gltf e.g.) |
|3D object | an object inside a scene characterized by vertex-, face- and customproperty data. |
|URI | some resource at something somewhere via someprotocol (`http://me.com/foo.glb#foo` or `e76f8efec8efce98e6f` [see interpeer.io](https://interpeer.io))|
|URL | something somewhere via someprotocol (`http://me.com/foo.glb`) |
|URN | something at some domain (`me.com/foo.glb`) |
|metadata | custom properties of text, 3D Scene or Object(nodes), relevant to machines and a human minority (academics/developers) |
|XR fragment | URI Fragment with spatial hints (which match the name of a 3D object-, camera-, animation-object) |
|the hash-events | hashtags map to camera/scene-projections |
|teleportation | repositioning the enduser to a different position (or 3D scene/file) |
|sourceportation | teleporting the enduser to the original XR Document of an `src` embedded object. |
|placeholder object | a 3D object which with src-metadata (which will be replaced by the src-data.) |
|src | (HTML-piggybacked) metadata of a 3D object which instances content |
|href | (HTML-piggybacked) metadata of a 3D object which links to content |
|filter | URI Fragment(s) which show/hide object(s) in a scene based on name/tag/property (`#cube&-price=>3`) |
|requestless metadata | metadata which never spawns new requests (unlike RDF/HTML, which can cause framerate-dropping, hence not used a lot in games) |
|FPS | frames per second in spatial experiences (games,VR,AR e.g.), should be as high as possible |
|introspective | inward sensemaking ("I feel this belongs to that") |
|extrospective | outward sensemaking ("I'm fairly sure John is a person who lives in oklahoma") |
|`◻` | ascii representation of an 3D object/mesh |
|(un)obtrusive | obtrusive: wrapping human text/thought in XML/HTML/JSON obfuscates human text into a salad of machine-symbols and words |
|flat 3D object | a 3D object of which all verticies share a plane |