coderofsalvation/xrfragment-haxe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
xrfragment-haxe/doc/RFC_XR_Fragments.xml

1248 lines
66 KiB
XML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<?xml version="1.0" encoding="utf-8"?>
<!-- name="GENERATOR" content="github.com/mmarkdown/mmark Mmark Markdown Processor - mmark.miek.nl" -->
<rfc version="3" ipr="trust200902" docName="draft-XRFRAGMENTS-leonvankammen-00" submissionType="IETF" category="info" xml:lang="en" xmlns:xi="http://www.w3.org/2001/XInclude" indexInclude="true" consensus="true">
<front>
<title>XR Fragments</title><seriesInfo value="draft-XRFRAGMENTS-leonvankammen-00" stream="IETF" status="informational" name="XR-Fragments"></seriesInfo>
<author initials="L.R." surname="van Kammen" fullname="L.R. van Kammen"><organization></organization><address><postal><street></street>
</postal></address></author><date/>
<area>Internet</area>
<workgroup>Jens &amp; Leon Internet Engineering Task Force</workgroup>
<abstract>
<t>An open specification for hyperlinking &amp; deeplinking 3D fileformats.
This draft is a specification for interactive URI-controllable 3D files, enabling <eref target="https://github.com/coderofsalvation/hypermediatic">hypermediatic</eref> navigation, to enable a spatial web for hypermedia browsers with- or without a network-connection.<br />
XR Fragments allows us to better use implicit metadata inside 3D scene(files), by mapping it to proven technologies like <eref target="https://en.wikipedia.org/wiki/URI_fragment">URI Fragments</eref>.<br />
XR Fragments views XR experiences thru the lens of 3D deeplinked URI's, rather than thru code(frameworks) or protocol-specific browsers (webbrowser e.g.).
The standard comprises of various (optional) support levels, which also include <eref target="https://www.w3.org/TR/media-frags/">W3C Media Fragments</eref> and <eref target="https://www.rfc-editor.org/rfc/rfc6570">URI Templates (RFC6570)</eref> to promote spatial addressibility, sharing, navigation, filtering and databinding objects for (XR) Browsers.<br />
</t>
<t>XR Fragments is in a sense, a &lt;b&gt;heuristical 3D format&lt;/b&gt; or meta-format, which leverages heuristic rules derived from any 3D scene or well-established 3D file formats, to extract meaningful features from scene hierarchies.<br />
These heuristics, enable features that are both meaningful and consistent across different scene representations, allowing &lt;b&gt;higher interop&lt;/b&gt; between fileformats, 3D editors, viewers and game-engines.</t>
<t>Almost every idea in this document is demonstrated at <eref target="https://xrfragment.org">https://xrfragment.org</eref></t>
</abstract>
</front>
<middle>
<section anchor="quick-reference"><name>Quick reference</name>
<ol spacing="compact">
<li><eref target="#abstract">Abstract</eref></li>
<li><eref target="#index">Index</eref></li>
<li><eref target="#introduction">Introduction</eref></li>
<li><eref target="#what-is-xr-fragments">What is XR Fragments</eref></li>
<li><eref target="#hfl-hypermediatic-feedback-loop-for-xr-browsers">HFL (Hypermediatic Feedback Loop) for XR Browsers</eref></li>
<li><t><eref target="#conventions-and-definitions">Conventions and Definitions</eref></t>
<ol spacing="compact">
<li><eref target="#xr-fragment-url-grammar">XR Fragment URL Grammar</eref></li>
</ol></li>
<li><eref target="#spatial-referencing-3d">Spatial Referencing 3D</eref></li>
<li><t><eref target="#level0-files">Level0: Files</eref></t>
<ol spacing="compact">
<li><eref target="#via-href-metadata">via href metadata</eref></li>
<li><eref target="#via-chained-extension">via chained extension</eref></li>
<li><eref target="#via-subdocuments-xattr">via subdocuments/xattr</eref></li>
<li><eref target="#json-sidecar-file">JSON sidecar-file</eref></li>
</ol></li>
<li><t><eref target="#level1-uri">Level1: URI</eref></t>
<ol spacing="compact">
<li><eref target="#list-of-uri-fragments">List of URI Fragments</eref></li>
<li><eref target="#list-of-explicit-metadata">List of explicit metadata</eref></li>
</ol></li>
<li><t><eref target="#level2-href-links">Level2: href links</eref></t>
<ol spacing="compact">
<li><eref target="#interaction-behaviour">Interaction behaviour</eref></li>
<li><eref target="#xr-viewer-implementation">XR Viewer implementation</eref></li>
</ol></li>
<li><t><eref target="#level3-media-fragments">Level3: Media Fragments</eref></t>
<ol spacing="compact">
<li><eref target="#animation-s-timeline">Animation(s) timeline</eref></li>
<li><eref target="#specify-playback-loopmode">Specify playback loopmode</eref></li>
<li><eref target="#controlling-embedded-content">Controlling embedded content</eref></li>
</ol></li>
<li><t><eref target="#level4-prefix-operators">Level4: prefix operators</eref></t>
<ol spacing="compact">
<li><eref target="#object-teleports">Object teleports</eref></li>
<li><eref target="#object-multipliers">Object multipliers</eref></li>
<li><eref target="#de-selectors-and">De/selectors (+ and -)</eref></li>
<li><eref target="#sharing-object-or-file">Sharing object or file (#|)</eref></li>
<li><eref target="#xrf-uri-scheme">xrf:// URI scheme</eref></li>
</ol></li>
<li><eref target="#level5-uri-templates-rfc6570">Level5: URI Templates (RFC6570)</eref></li>
<li><t><eref target="#top-level-url-processing">Top-level URL processing</eref></t>
<ol spacing="compact">
<li><eref target="#ux">UX</eref></li>
</ol></li>
<li><t><eref target="#example-navigating-content-href-portals">Example: Navigating content href portals</eref></t>
<ol spacing="compact">
<li><eref target="#walking-surfaces">Walking surfaces</eref></li>
</ol></li>
<li><eref target="#example-virtual-world-rings">Example: Virtual world rings</eref></li>
<li><eref target="#additional-scene-metadata">Additional scene metadata</eref></li>
<li><t><eref target="#accessibility-interface">Accessibility interface</eref></t>
<ol spacing="compact">
<li><eref target="#two-button-navigation">Two-button navigation</eref></li>
<li><eref target="#overlap-with-fileformat-specific-extensions">Overlap with fileformat-specific extensions</eref></li>
</ol></li>
<li><eref target="#vendor-prefixes">Vendor Prefixes</eref></li>
<li><eref target="#security-considerations">Security Considerations</eref></li>
<li><eref target="#faq">FAQ</eref></li>
<li><eref target="#authors">Authors</eref></li>
<li><eref target="#iana-considerations">IANA Considerations</eref></li>
<li><eref target="#acknowledgments">Acknowledgments</eref></li>
<li><eref target="#appendix-definitions">Appendix: Definitions</eref></li>
</ol>
</section>
<section anchor="introduction"><name>Introduction</name>
<iref item="Introduction"/><t>How can we add more control to existing text and 3D scenes, without introducing new dataformats?<br />
Historically, there's many attempts to create the ultimate 3D fileformat.<br />
The lowest common denominator is: designers describing/tagging/naming things using <strong>plain text</strong>.<br />
XR Fragments exploits the fact that all 3D models already contain such metadata:</t>
<t><strong>XR Fragments allows deeplinking of 3D objects by mapping objectnames to URI fragments</strong></t>
<t>It solves:</t>
<ol spacing="compact">
<li>addressibility and <eref target="https://github.com/coderofsalvation/hypermediatic">hypermediatic</eref> navigation of 3D scenes/objects: <eref target="https://en.wikipedia.org/wiki/URI_fragment">URI Fragments</eref> using src/href spatial metadata</li>
<li>Interlinking text &amp; spatial objects by collapsing space into a Word Graph (XRWG) to show <eref target="#visible-links">visible links</eref></li>
<li>unlocking spatial potential of the (originally 2D) hashtag (which jumps to a chapter) for navigating XR documents</li>
<li>refraining from introducing scripting-engines for mundane tasks (and preventing its inevitable security-headaches)</li>
<li>the gap between text an 3d objects: object-names directly map to hashtags (=fragments), which allows 3D to text transcription.</li>
</ol>
<blockquote><t>NOTE: The chapters in this document are ordered from highlevel to lowlevel (technical) as much as possible</t>
</blockquote></section>
<section anchor="what-is-xr-fragments"><name>What is XR Fragments</name>
<iref item="What is XR Fragments"/><t>XR Fragments utilizes URLs:</t>
<ol spacing="compact">
<li>for 3D viewers/browser to manipulate the camera or objects (via URI fragments)</li>
<li>implicitly: by mapping 3D objectnames (of a 3D scene/file) to URI fragments (3D deeplinking)</li>
<li>explicitly: by scanning <tt>href</tt> metadata <strong>inside</strong> 3D scene-files to enable interactions</li>
<li>externally: progressively enhance a 3D (file) into an experience via <eref target="https://en.wikipedia.org/wiki/Sidecar_file">sidecarfiles</eref></li>
</ol>
</section>
<section anchor="hfl-hypermediatic-feedback-loop-for-xr-browsers"><name>HFL (Hypermediatic Feedback Loop) for XR Browsers</name>
<t><iref item="HFL (Hypermediatic Feedback Loop"/> for XR Browsers)</t>
<t><tt>href</tt> metadata traditionally implies <strong>click</strong> AND <strong>navigate</strong>, however XR Fragments adds stateless <strong>click</strong> (<tt>xrf://....</tt>) via the <tt>xrf://</tt> scheme, which does not change the top-level URL-adress (of the browser).
This allows for many extra interactions via URLs, which otherwise needs a scripting language.
These are called <strong>hashbus</strong>-only events/</t>
<blockquote><t>Being able to use the same URI Fragment DSL for navigation (<tt>href: #foo</tt>) as well as interactions (<tt>href: xrf://#foo</tt>) greatly simplifies implementation, increases HFL, and reduces need for scripting languages.</t>
</blockquote><t>This opens up the following benefits for traditional &amp; future webbrowsers:</t>
<ul spacing="compact">
<li><eref target="https://github.com/coderofsalvation/hypermediatic">hypermediatic</eref> loading/clicking 3D assets (gltf/fbx e.g.) natively (with or without using HTML).</li>
<li>potentially allowing 3D assets/nodes to publish XR Fragments to themselves/eachother using the <tt>xrf://</tt> hashbus (<tt>xrf://#person=walk</tt> to trigger <tt>walk</tt>-animation for object <tt>person</tt>)</li>
<li>potentially collapsing the 3D scene to an wordgraph (for essential navigation purposes) controllable thru a hash(tag)bus</li>
<li>completely bypassing the security-trap of loading external scripts (by loading 3D model-files, not HTML-javascriptable resources)</li>
</ul>
<t>XR Fragments itself are <eref target="https://github.com/coderofsalvation/hypermediatic">hypermediatic</eref> and HTML-agnostic, though pseudo-XR Fragment browsers <strong>can</strong> be implemented on top of HTML/Javascript.</t>
<table>
<thead>
<tr>
<th>principle</th>
<th>3D URL</th>
<th>HTML 2D URL</th>
</tr>
</thead>
<tbody>
<tr>
<td>the XRWG</td>
<td>wordgraph (collapses 3D scene to tags)</td>
<td>Ctrl-F (find)</td>
</tr>
<tr>
<td>the hashbus</td>
<td>hashtags alter camera/scene/object-projections</td>
<td>hashtags alter document positions</td>
</tr>
<tr>
<td>src metadata</td>
<td>renders content and offers sourceportation</td>
<td>renders content</td>
</tr>
<tr>
<td>href metadata</td>
<td>teleports to other XR document</td>
<td>jumps to other HTML document</td>
</tr>
<tr>
<td>href metadata</td>
<td>triggers predefined view</td>
<td>Media fragments</td>
</tr>
<tr>
<td>href metadata</td>
<td>triggers camera/scene/object/projections</td>
<td>n/a</td>
</tr>
<tr>
<td>href metadata</td>
<td>draws visible connection(s) for XRWG 'tag'</td>
<td>n/a</td>
</tr>
<tr>
<td>href metadata</td>
<td>filters certain (in)visible objects</td>
<td>n/a</td>
</tr>
<tr>
<td>href metadata</td>
<td>href=&quot;xrf://#-foo&amp;bar&quot;</td>
<td>href=&quot;javascript:hideFooAndShowBar()`</td>
</tr>
<tr>
<td></td>
<td>(this does not update topLevel URI)</td>
<td>(this is non-standard, non-hypermediatic)</td>
</tr>
</tbody>
</table><blockquote><t>An important aspect of HFL is that URI Fragments can be triggered without updating the top-level URI (default href-behaviour) thru their own 'bus' (<tt>xrf://#.....</tt>). This decoupling between navigation and interaction prevents non-standard things like (<tt>href</tt>:<tt>javascript:dosomething()</tt>).</t>
</blockquote></section>
<section anchor="conventions-and-definitions"><name>Conventions and Definitions</name>
<iref item="Conventions and Definitions"/><t>See appendix below in case certain terms are not clear.</t>
<section anchor="xr-fragment-url-grammar"><name>XR Fragment URL Grammar</name>
<iref item="XR Fragment URL Grammar"/><t>For typical HTTP-like browsers/applications:</t>
<artwork><![CDATA[reserved = gen-delims / sub-delims
gen-delims = "#" / "&"
sub-delims = "," / "="
]]>
</artwork>
<blockquote><t>Example: <tt>://foo.com/my3d.gltf#room1&amp;prio=-5&amp;t=0,100</tt></t>
</blockquote><table>
<thead>
<tr>
<th>Demo</th>
<th>Explanation</th>
</tr>
</thead>
<tbody>
<tr>
<td><tt>room1</tt></td>
<td>vector/coordinate argument e.g.</td>
</tr>
<tr>
<td><tt>room1&amp;cam1</tt></td>
<td>combinators</td>
</tr>
</tbody>
</table><blockquote><t>this is already implemented in all browsers</t>
</blockquote><t>Pseudo (non-native) browser-implementations (supporting XR Fragments using HTML+JS e.g.) can use the <tt>?</tt> search-operator to address outbound content.<br />
In other words, the URL updates to: <tt>https://me.com?https://me.com/other.glb</tt> when navigating to <tt>https://me.com/other.glb</tt> from inside a <tt>https://me.com</tt> WebXR experience e.g.<br />
That way, if the link gets shared, the XR Fragments implementation at <tt>https://me.com</tt> can load the latter (and still indicates which XR Fragments entrypoint-experience/client was used).</t>
</section>
</section>
<section anchor="spatial-referencing-3d"><name>Spatial Referencing 3D</name>
<iref item="Spatial Referencing 3D"/><t>3D files contain an hierarchy of objects.<br />
XR Fragments assumes the following objectname-to-URI-Fragment mapping, in order to deeplink 3D objects:</t>
<artwork><![CDATA[
my.io/scene.fbx
+─────────────────────────────+
│ sky │ src: http://my.io/scene.fbx#sky (includes building,mainobject,floor)
│ +─────────────────────────+ │
│ │ building │ │ src: http://my.io/scene.fbx#building (includes mainobject,floor)
│ │ +─────────────────────+ │ │
│ │ │ mainobject │ │ │ src: http://my.io/scene.fbx#mainobject (includes floor)
│ │ │ +─────────────────+ │ │ │
│ │ │ │ floor │ │ │ │ src: http://my.io/scene.fbx#floor (just floor object)
│ │ │ │ │ │ │ │
│ │ │ +─────────────────+ │ │ │
│ │ +─────────────────────+ │ │
│ +─────────────────────────+ │
+─────────────────────────────+
]]>
</artwork>
<blockquote><t>Every 3D fileformat supports named 3D object, and this name allows URLs (fragments) to reference them (and their children objects).</t>
</blockquote><t>Clever nested design of 3D scenes allow great ways for re-using content, and/or previewing scenes.<br />
For example, to render a portal with a preview-version of the scene, create an 3D object with:</t>
<ul spacing="compact">
<li>href: <tt>https://scene.fbx</tt></li>
</ul>
<blockquote><t>It also allows <strong>sourceportation</strong>, which basically means the enduser can teleport to the original XR Document of an <tt>src</tt> embedded object, and see a visible connection to the particular embedded object. Basically an embedded link becoming an outbound link by activating it.</t>
</blockquote></section>
<section anchor="level0-files"><name>Level0: Files</name>
<iref item="Level0: Files"/><iref item="Level0: Files"/><t>These are <strong>optional</strong> auto-loaded <eref target="">side-car files</eref> to enable hasslefree <eref target="#XR%20Movies">XR Movies</eref>.<br />
they can accomodate developers or applications who (for whatever reason) must not modify the 3D scene-file (a <tt>.glb</tt> e.g.).</t>
<section anchor="via-href-metadata"><name>via href metadata</name>
<artwork><![CDATA[scene.glb <--- 'href' extra [heuristic] detected inside!
scene.png (preview thumbnail)
scene.ogg (soundtrack to plays when global 3D animation starts)
scene.vtt (subtitles for accessibility or screenreaders)
scene.json (sidecar JSON-file with explicit metadata)
]]>
</artwork>
<t><strong>heuristics</strong>:</t>
<ul spacing="compact">
<li>if at least one <tt>href</tt> custom property/extra is found in a 3D scene</li>
<li>The viewer should poll for the above mentioned sidecar-file extensions (and present accordingly)</li>
</ul>
</section>
<section anchor="via-chained-extension"><name>via chained extension</name>
<artwork><![CDATA[scene.xrf.glb <--- '.xrf.' sidecar file heuristic detected!
scene.xrf.png (preview thumbnail)
scene.xrf.ogg (soundtrack to plays when global 3D animation starts)
scene.xrf.vtt (subtitles for accessibility or screenreaders)
scene.xrf.json (sidecar JSON-file with explicit metadata)
]]>
</artwork>
<blockquote><t>A fallback-mechanism to turn 3D files into <eref target="#XR%20Movies">XR Movies</eref> without editing them.</t>
</blockquote><t><strong>heuristics</strong>:</t>
<ul spacing="compact">
<li>the chained-extension heuristic <tt>.xrf.</tt> should be present in the filename (<tt>scene.xrf.glb</tt> e.g.)</li>
</ul>
</section>
<section anchor="via-subdocuments-xattr"><name>via subdocuments/xattr</name>
<t>More secure protocols (Nextgraph e.g.) don't allow for simply polling files.
In such case, subdocuments or extended attributes should be polled:</t>
<blockquote><t>NOTE: in the examples below we use the href-heuristic, but also the <tt>.xrf.</tt> chained-extension applies here.</t>
</blockquote>
<artwork><![CDATA[myspreadsheet.ods
└── explainer.glb <--- 'href' extra [heuristic] detected inside!
├── explainer.ogg (soundtrack to play when global 3D animation starts)
├── explainer.png (preview thumnbnail)
├── explainer.json (sidecar JSON-file with explicit metadata)
└── explainer.vtt (subtitles for accessibility or screenreaders)
]]>
</artwork>
<t>If only extended attributes (xattr) are available, the respective referenced file can be embedded:</t>
<artwork><![CDATA[$ setfattr -n explainer.ogg -v "soundtrack.ogg" explainer.glb
$ setfattr -n explainer.png -v "thumbnail.png" explainer.glb
$ setfattr -n explainer.vtt -v "subtitles.vtt" explainer.glb
]]>
</artwork>
<blockquote><t>NOTE: Linux's <tt>setfattr/getfattr</tt> is <tt>xattr</tt> on mac, and <tt>Set-Content/Get-content</tt> on Windows. See <eref target="https://www.lesbonscomptes.com/pxattr/index.html">pxattr</eref> for lowlevel access.</t>
</blockquote></section>
<section anchor="json-sidecar-file"><name>JSON sidecar-file</name>
<t>For developers, sidecar-file can allow for defining <strong>explicit</strong> XR Fragments links (&gt;level1), outside of the 3D file.<br />
This can be done via (objectname/metadata) key/value-pairs in a JSON <eref target="https://en.wikipedia.org/wiki/Sidecar_file">sidecar-file</eref>:</t>
<ul spacing="compact">
<li>experience.glb</li>
<li>experience.json <tt>&lt;----</tt></li>
</ul>
<sourcecode type="json"><![CDATA[{
"/":
"aria-description": "description of scene",
},
"button": {
"href": "#roomB"
}
}
]]>
</sourcecode>
<blockquote><t>This will make object <tt>button</tt> clickable, and teleport the user to object <tt>roomB</tt>.</t>
</blockquote><t>So after loading <tt>experience.glb</tt> the existence of <tt>experience.json</tt> is detected, to apply the explicit metadata.<br />
The sidecar will define (or <strong>override</strong> already existing) extras, which can be handy for multi-user platforms (offer 3D scene customization/personalization to users).</t>
<blockquote><t>In THREE.js-code this would boil down to:</t>
</blockquote>
<sourcecode type="javascript"><![CDATA[ scene.userData['aria-description'] = "description of scene"
scene.getObjectByName("button").userData.href = "#roomB"
// now the XR Fragments parser can process the XR Fragments userData 'extras' in the scene
]]>
</sourcecode>
</section>
</section>
<section anchor="level1-uri"><name>Level1: URI</name>
<iref item="Level1: URI"/><blockquote><t><strong>XR Fragments allows deeplinking of 3D objects by mapping objectnames to URI fragments</strong></t>
</blockquote><t>XR Fragments tries to seek to connect the world of text (semantical web / RDF), and the world of pixels.<br />
Instead of forcing authors to combine 3D/2D objects programmatically (publishing thru a game-editor e.g.), XR Fragments <strong>integrates all</strong> which allows a universal viewing experience.<br />
</t>
<artwork><![CDATA[ +───────────────────────────────────────────────────────────────────────────────────────────────+
│ │
│ U R N │
│ U R L | │
│ | |-----------------+--------| │
│ +--------------------------------------------------| │
│ | │
│ + https://foo.com/some/foo/scene.glb#someview <-- http URI (=URL and has URN) │
│ | │
│ + ipfs://cfe0987ec9r9098ecr/cats.fbx#someview <-- an IPFS URI (=URL and has URN) │
│ │
│ ec09f7e9cf8e7f09c8e7f98e79c09ef89e000efece8f7ecfe9fe <-- an interpeer URI │
│ │
│ │
│ |------------------------+-------------------------| │
│ | │
│ U R I │
│ │
+───────────────────────────────────────────────────────────────────────────────────────────────+
]]>
</artwork>
<t>Fact: our typical browser URL's are just <strong>a possible implementation</strong> of URI's (for untapped humancentric potential of URI's <eref target="https://interpeer.io">see interpeer.io</eref> or <eref target="https://nextgraph.org">NextGraph</eref> )</t>
<blockquote><t>XR Fragments does not look at XR (or the web) thru the lens of HTML or URLs.<br />
But approaches things from a higherlevel local-first 3D hypermedia browser-perspective.</t>
</blockquote><t>Below you can see how this translates back into good-old URLs:</t>
<artwork><![CDATA[ +───────────────────────────────────────────────────────────────────────────────────────────────+
│ │
│ the soul of any URL: ://macro /meso ?micro #nano │
│ │
│ 2D URL: ://library.com /document ?search #chapter │
│ xrf:// │
│ 4D URL: ://park.com /4Dscene.fbx ─> ?other.glb ─> #object ─> hashbus │
│ │ #filter │ │
│ │ #tag │ │
│ │ (hypermediatic) #material │ │
│ │ ( feedback ) #animation │ │
│ │ ( loop ) #texture │ │
│ │ #variable │ │
│ │ │ │
│ XRWG <─────────────────────<─────────────+ │
│ │ │ │
│ └─ objects ──────────────>─────────────+ │
│ │
│ │
+───────────────────────────────────────────────────────────────────────────────────────────────+
]]>
</artwork>
<blockquote><t>?-linked and #-linked navigation are JUST one possible way to implement XR Fragments: the essential goal is to allow a Hypermediatic FeedbackLoop (HFL) between external and internal 4D navigation.</t>
</blockquote>
<section anchor="list-of-uri-fragments"><name>List of URI Fragments</name>
<table>
<thead>
<tr>
<th>fragment</th>
<th>type</th>
<th>example</th>
<th>info</th>
</tr>
</thead>
<tbody>
<tr>
<td><tt>#......</tt></td>
<td>vector3</td>
<td><tt>#room1</tt> <tt>#room2</tt> <tt>#cam2</tt></td>
<td>positions/parents camera(rig) (or XR floor) to xyz-coord/object/camera and upvector</td>
</tr>
<tr>
<td><eref target="https://www.w3.org/TR/media-frags/">Media Fragments</eref></td>
<td><eref target="#media%20fragments%20and%20datatypes">media fragment</eref></td>
<td><tt>#t=0,2&amp;loop</tt></td>
<td>play (and loop) 3D animation from 0 seconds till 2 seconds</td>
</tr>
</tbody>
</table></section>
<section anchor="list-of-explicit-metadata"><name>List of *<em>explicit</em> metadata</name>
<t>These are the possible 'extras' for 3D nodes and sidecar-files</t>
<table>
<thead>
<tr>
<th>key</th>
<th>type</th>
<th>example (JSON)</th>
<th>function</th>
<th>existing compatibility</th>
</tr>
</thead>
<tbody>
<tr>
<td><tt>href</tt></td>
<td>string</td>
<td><tt>&quot;href&quot;: &quot;b.gltf&quot;</tt></td>
<td>XR teleport</td>
<td>custom property in 3D fileformats</td>
</tr>
</tbody>
</table></section>
</section>
<section anchor="level2-href-links"><name>Level2: href links</name>
<iref item="Level2: href links"/><t>Explicit href metadata ('extras') in a 3D object (of a 3D file), hint the viewer that the user ''can interact'' with that object :</t>
<t>| fragment | type | example value |
|<tt>href</tt>| string (uri or predefined view) | <tt>#pyramid</tt><br />
<tt>#lastvisit</tt><br />
<tt>xrf://#-someobject</tt><br />
<tt>://somefile.gltf#foo</tt><br />
|</t>
<section anchor="interaction-behaviour"><name>Interaction behaviour</name>
<t>When clicking an ''href''-value, the user(camera) is teleport to the referenced object.</t>
<t>The imported/teleported destination can be another object in the same scene-file, or a different file.</t>
</section>
<section anchor="xr-viewer-implementation"><name>XR Viewer implementation</name>
<t>| <strong>spec</strong> | <strong>action</strong> | <strong>feature</strong> |
|-|-|-|
| level0+1 | hover 3D file <eref target="#via-href-metadata">href</eref> | show the preview PNG thumbnail (if any). |
| level0+1 | launch 3D file <eref target="#via-href-metadata">href</eref> | replace the current scene with a new 3D file (<tt>href: other.glb</tt> e.g.) |
| level2 | click internal 3D file <eref target="#via-href-metadata">href</eref> (<tt>#roomB</tt> e.g.) | teleport the camera to the origin of object(name <tt>roomB</tt>). See [[teleport camera]].|
| level2 | click external 3D file <eref target="#via-href-metadata">href</eref> (<tt>foo.glb</tt> e.g.) | replace the current scene with a new 3D file (<tt>href: other.glb</tt> e.g.) |
| level2 | hover external 3D file <eref target="#via-href-metadata">href</eref> | show the preview PNG thumbnail (if any sidecar, see level0) |
| level2 | click <eref target="#via-href-metadata">href</eref> | hashbus: execute without changing the toplevel URL location (<tt>href: xrf://#someObjectName</tt> e.g.) |
| level3 | click <eref target="#via-href-metadata">href</eref> | set the global 3D animation timeline to its Media Fragment value (<tt>#t=2,3</tt> e.g.) |</t>
<blockquote><t>NOTE: hashbus links (<tt>xrf://#foo&amp;bar</tt>) don't change the toplevel URL, which makes it ideal for interactions (in contrast to typical <tt>#roomC</tt> navigation, which benefit back/forward browser-buttons), see &lt;a href=&quot;#hashbus&quot;&gt;hashbus&lt;/a&gt; for more info.</t>
</blockquote></section>
</section>
<section anchor="level3-media-fragments"><name>Level3: Media Fragments</name>
<iref item="Level3: Media Fragments"/><blockquote><t>these allow for XR Movies with a controllable timeline using <tt>href</tt> URI's with Media Fragments</t>
</blockquote><t>Just like with 2D media-files, W3C mediafragments (<tt>#t=1,2</tt>) can be used to control a timeline via the <eref target="##t">#t</eref> primitive.
XR Fragments Level3 makes the 3D timeline, as well as URL-referenced files <strong>controllable</strong> via Media Fragments like:</t>
<ul spacing="compact">
<li>level2 hrefs (<tt>href: #t=4</tt> e.g. to control 3D timeline)</li>
<li><t>level4: <tt>xrf:</tt> URI scheme:</t>
<ul spacing="compact">
<li><tt>href: xrf:foo.wav#t=0</tt> to play a wav</li>
<li><tt>href: xrf:news.glb?clone#t=0</tt> to instance and play another experience</li>
</ul></li>
</ul>
<section anchor="animation-s-timeline"><name>Animation(s) timeline</name>
<t>controls the animation(s) of the scene (or <tt>src</tt> resource which contains a timeline)</t>
<t>| fragment | type | functionality |
| &lt;b&gt;#t&lt;/b&gt;=start,stop | <eref target="default:`#t=0,0`">[vector2]</eref> | start,stop (in seconds |</t>
<t>| Example Value | Explanation |
| <tt>#t=1</tt> | play (3D) animations from 1 seconds till end (and stop) |
| <tt>#t=1,100</tt> | play (3D) animations from 1 till 100 seconds (and stop) |
| <tt>#t=0,0</tt> | stop (3D) animations at frame 0 |</t>
<blockquote><t>Use [[#s 🌱]] to control playback speed</t>
</blockquote></section>
<section anchor="specify-playback-loopmode"><name>Specify playback loopmode</name>
<t>This compensates a missing element from Media Fragments to enable/disable temporal looping. .</t>
<t>| fragment | type | functionality |
| &lt;b&gt;#loop&lt;/b&gt; | string | enables animation/video/audio loop |
| &lt;b&gt;#-loop&lt;/b&gt; | string | disables animation/video/audio loop |</t>
</section>
<section anchor="controlling-embedded-content"><name>Controlling embedded content</name>
<t>use [[URI Templates]] to control embedded media, for example a simple video-player:</t>
<artwork><![CDATA[ foo.usdz
│
├── ◻ loopbutton_enable
│ └ href: #loop <-- enable global loop
│
├── ◻ loopbutton_enable
│ └ href: #-loop <-- disable global loop
│
├── ◻ playbutton
│ └ href: #t=10&loop <-- play global 3D timeline (all anims) (looped)
│
└── ◻ playbutton_external
└ href: https://my.org/animation.glb#!&t=3,10 <-- import & play external anim
]]>
</artwork>
</section>
</section>
<section anchor="level4-prefix-operators"><name>Level4: prefix operators</name>
<iref item="Level4: prefix operators"/><t>Prefixing objectnames with the following simple operators allow for <strong>extremely powerful</strong> XR interactions:</t>
<ul spacing="compact">
<li>#!</li>
<li>#*</li>
<li>#+ or #-</li>
<li>#|</li>
<li>xrf: URI scheme</li>
</ul>
<blockquote><t><strong>Examples:</strong> <tt>#+menu</tt> to show a object, <tt>#-menu</tt> to hide a menu, <tt>#!menu</tt> to teleport a menu, <tt>#*block</tt> to clone a grabbable block, <tt>#|object</tt> to share an object</t>
</blockquote>
<section anchor="object-teleports"><name>Object teleports (!)</name>
<t>Prefixing an object with an exclamation-symbol, will teleport a (local or remote) referenced object from/to its original/usercamera location.<br />
</t>
<t>[img[objecteleport.png]]</t>
<t>Usecases:
* show/hide objects/buttons (menu e.g.) in front of user
* embed remote (object within) 3D file via remote URL
* instance an interactive object near the user regardless of location
* instance HUD or semi-transparent-textured-sphere (LUT) around the user</t>
<t>&lt;div class=&quot;border padding&quot; style=&quot;border:4px solid #888&quot;&gt;
&lt;span class=&quot;big hi1&quot;&gt;#!menu&lt;/span&gt;
&lt;/div&gt;
<br />
</t>
<t>Clicking the <eref target="#via-href-metadata">href</eref>-value above will:</t>
<ol spacing="compact">
<li><strong>reposition the referenced object</strong> (menu) to the usercamera's-coordinates.</li>
<li><strong>zoom</strong> in case of (non-empty) mesh-object: rescale to 1 m³, and position 1m in front of the camera</li>
<li>toggle behaviour: revert values if 1/2 were already applied</li>
<li><tt>#+</tt> is always implied (objects are always made visible)</li>
</ol>
<t>This tiny but powerful symbol allows incredible interactive possibilities, by carefully positioning re-usable objects outside of a scene (below the usercamera's floor e.g.).</t>
<ul spacing="compact">
<li>href: <tt>#whiteroom&amp;!explainer&amp;!exitmenu</tt></li>
</ul>
<blockquote><t>This will teleport the user to <tt>whiteroom</tt> and moves object <tt>explainer</tt> and <tt>exitmenu</tt> in front of the user.</t>
</blockquote>
<ul spacing="compact">
<li>href: `<eref target="https://my.org/foo.glb#!">https://my.org/foo.glb#!</eref></li>
</ul>
<t>Clicking the <eref target="#via-href-metadata">href</eref>-value above will:</t>
<ol spacing="compact">
<li>import <tt>foo.glb</tt> from <tt>my.org</tt>'s webserver</li>
<li>show it in front of the user (because <tt>#!</tt> indicates object teleport)</li>
</ol>
<ul spacing="compact">
<li>href: <tt>https://foo.glb#roomB&amp;!bar</tt></li>
</ul>
<t>Clicking the <eref target="#via-href-metadata">href</eref>-value above will:</t>
<ol spacing="compact">
<li>replace the current scene with <tt>foo.glb</tt></li>
<li>teleport the user to #roomB inside <tt>foo.glb</tt></li>
<li><strong>instance the referenced object</strong> (bar inside foo.glb) in front of the user.</li>
<li>it will update the top-Level URL (because <tt>xrf:</tt> was not used)</li>
<li>hide the <strong>instanced object</strong> when clicked again (toggle visibility)</li>
</ol>
<blockquote><t><strong>NOTE</strong>: level2 teleportation links, as well as instancing mitigates the 'broken embedded image'-issue of HTML: <strong>always</strong> attaching the href-values to <strong>a 3D (preview) object</strong> (that way broken links will not break the design).</t>
</blockquote><t><strong>Example:</strong> clicking a 3D button with title 'menu' and <eref target="#href">href</eref>-value <tt>xrf:menu.glb?instance#t=4,5</tt> would instance a 3D menu (<tt>menu.glb</tt>) in front of the user, and loop its animation between from 4-5 seconds (<tt>t=4,5</tt>)</t>
<blockquote><t><strong>NOTE</strong>: combining instance-operators allows dynamic construction of 3D scenes (<tt>#london&amp;!welcomeMenu&amp;!fadeBox</tt> e.g.)</t>
</blockquote></section>
<section anchor="object-multipliers"><name>Object multipliers (*)</name>
<t>The star-prefix will clone a (local or remote) referenced object to the usercamera's location, and make it grabbable.<br />
Usecases:
* object-picker (build stuff with objects)</t>
<blockquote><t><strong>NOTE</strong>: this is basically the <eref target="#%23%21">#! operator</eref> which infinitely <strong>clones</strong> the referenced object (instead of repositioning the object).</t>
</blockquote></section>
<section anchor="de-selectors-and"><name>De/selectors (+ and -)</name>
<ul spacing="compact">
<li>href: <tt>#-welcome</tt> (or <tt>#+welcome</tt>)</li>
</ul>
<t>Clicking href-value above will do:</t>
<ol spacing="compact">
<li>show/hide the target object (and children)</li>
</ol>
<ul spacing="compact">
<li>href: <tt>#https://my.org/foo.glb/#bar&amp;-welcome</tt></li>
</ul>
<blockquote><t><strong>NOTE:</strong> the latter shows that (de)selectors can also be with regular <eref target="#href">href</eref>-values</t>
</blockquote></section>
<section anchor="sharing-object-or-file"><name>Sharing object or file (#|)</name>
<t>The pipe-symbol (<tt>|</tt>) sends a (targeted) object to the OS.
Clicking the href-value below will:</t>
<ol spacing="compact">
<li>share the (targeted object in the) file to a another application</li>
</ol>
<blockquote><t>This URL can be fed straight into <eref target="https://developer.mozilla.org/en-US/docs/Web/API/Web_Share_API">Web Share API</eref> or <eref target="https://www.freedesktop.org/wiki/Software/xdg-utils/">xdg-open</eref></t>
</blockquote>
<ul spacing="compact">
<li>href: <tt>xrf://#|bar</tt></li>
</ul>
<blockquote><t><strong>NOTE</strong>: sharing is limited to (internal objects) via <tt>xrf:</tt> scheme-only</t>
</blockquote></section>
<section anchor="xrf-uri-scheme"><name>xrf:// URI scheme</name>
<t>Prefixing the <tt>xrf:</tt> to <eref target="#href">href</eref>-values <strong>will prevent</strong> <eref target="#📜%20level2:%20explicit%20links">level2</eref> <eref target="#href">href</eref>-values from changing the top-Level URL.</t>
<blockquote><t><strong>Usecase</strong>: for non-shareable URLs like <tt>href: xrf:#t=4,5</tt>, to display a stateful msg e.g.).</t>
</blockquote><t><strong>Reason:</strong> XR Fragments is inspired by HTML's <eref target="https://en.wikipedia.org/wiki/Hyperlink">href-attribute</eref>, which does various things:</t>
<ol spacing="compact">
<li>it updates the browser-location</li>
<li>it makes something clickable</li>
<li>it jumps to another document / elsewhere in the same document</li>
<li>and more</li>
</ol>
<t>The <tt>xrf:</tt> scheme will just do 2 &amp; 3 (so the URL-values will not leak into the top-level URL).</t>
<blockquote><t><strong>compliance with RFC 3986</strong>: unimplemented/unknown URI schemes (<tt>xrf:...</tt> e.g.) will not update the top-level URL</t>
</blockquote></section>
</section>
<section anchor="level5-uri-templates-rfc6570"><name>Level5: URI Templates (RFC6570)</name>
<t><iref item="Level5: URI Templates (RFC6570"/>)</t>
<t>XR Fragments adopts Level1 URI <strong>Fragment</strong> expansion to provide safe interactivity.<br />
This is non-normative, and the draft spec is available on request.</t>
</section>
<section anchor="top-level-url-processing"><name>Top-level URL processing</name>
<iref item="Top-level URL processing"/><blockquote><t>Example URL: <tt>://foo/world.gltf#room1&amp;t=10&amp;cam</tt></t>
</blockquote><t>The URL-processing-flow for hypermedia browsers goes like this:</t>
<ol spacing="compact">
<li>IF scene operators and/or animation operator (<tt>t</tt>) are present in the URL then (re)position the camera (to <tt>room1</tt>) and/or animation-range (<tt>10</tt>) accordingly.</li>
<li>IF no camera-position has been set in &lt;b&gt;step 1 or 2&lt;/b&gt; assume <tt>0,0,0</tt> as camera coordinate (XR: add user-height) (<eref target="https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/navigator.js#L31]]">example</eref>)</li>
<li>IF a camera-object exists with name <tt>cam</tt> assume that user(camera) position</li>
</ol>
<section anchor="ux"><name>UX</name>
<t>End-users should always have read/write access to:</t>
<ol spacing="compact">
<li>the current (toplevel) &lt;b&gt;URL&lt;/b&gt; (an URLbar etc)</li>
<li>URL-history (a &lt;b&gt;back/forward&lt;/b&gt; button e.g.)</li>
<li>Clicking/Touching an <tt>href</tt> navigates (and updates the URL) to another scene/file (and coordinate e.g. in case the URL contains XR Fragments).</li>
</ol>
</section>
</section>
<section anchor="example-navigating-content-href-portals"><name>Example: Navigating content href portals</name>
<iref item="Example: Navigating content href portals"/><t>navigation, portals &amp; mutations</t>
<table>
<thead>
<tr>
<th>fragment</th>
<th>type</th>
<th>example value</th>
</tr>
</thead>
<tbody>
<tr>
<td><tt>href</tt></td>
<td>string (uri or predefined view)</td>
<td><tt>#room1</tt><br />
<tt>#room1</tt><br />
<tt>://somefile.gltf#room1</tt><br />
</td>
</tr>
</tbody>
</table>
<ol>
<li><t>clicking an outbound ''external''- or ''file URI'' fully replaces the current scene and assumes <tt>room2</tt> by default (unless specified)</t>
</li>
<li><t>relocation/reorientation should happen locally for local URI's (<tt>#....</tt>)</t>
</li>
<li><t>navigation should not happen ''immediately'' when user is more than 5 meter away from the portal/object containing the href (to prevent accidental navigation e.g.)</t>
</li>
<li><t>URL navigation should always be reflected in the client URL-bar (in case of javascript: see [<eref target="https://github.com/coderofsalvation/xrfragment/blob/dev/src/3rd/js/three/navigator.js">here</eref> for an example navigator), and only update the URL-bar after the scene (default fragment <tt>#</tt>) has been loaded.</t>
</li>
<li><t>In immersive XR mode, the navigator back/forward-buttons should be always visible (using a wearable e.g., see [<eref target="https://github.com/coderofsalvation/xrfragment/blob/dev/example/aframe/sandbox/index.html#L26-L29">here</eref> for an example wearable)</t>
</li>
<li><t>make sure that the ''back-button'' of the ''browser-history'' always refers to the previous position (see [<eref target="https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/xrf/href.js#L97">here</eref>)</t>
</li>
<li><t>ignore previous rule in special cases, like clicking an <tt>href</tt> using camera-portal collision (the back-button could cause a teleport-loop if the previous position is too close)</t>
</li>
<li><t>href-events should bubble upward the node-tree (from children to ancestors, so that ancestors can also conain an href), however only 1 href can be executed at the same time.</t>
</li>
<li><t>the end-user navigator back/forward buttons should repeat a back/forward action until a <tt>#...</tt> primitive is found (the stateless xrf:// href-values should not be pushed to the url-history)</t>
</li>
</ol>
<t><eref target="https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/xrf/href.js">» example implementation</eref><br />
<eref target="https://github.com/coderofsalvation/xrfragment/blob/main/example/assets/href.gltf#L192">» example 3D asset</eref><br />
<eref target="https://github.com/coderofsalvation/xrfragment/issues/1">» discussion</eref><br />
</t>
<section anchor="walking-surfaces"><name>Walking surfaces</name>
<blockquote><t>By default position <tt>0,0,0</tt> of the 3D scene represents the walkable plane, however this is overridden when the following applies:</t>
</blockquote><t>XR Fragment-compatible viewers can infer this data based scanning the scene for:</t>
<ol spacing="compact">
<li>materialless (nameless &amp; textureless) mesh-objects (without <tt>href</tt> and &gt;0 faces)</li>
</ol>
<blockquote><t>optionally the viewer can offer thumbstick, mouse or joystick teleport-tools for non-roomscale VR/AR setups.</t>
</blockquote></section>
</section>
<section anchor="example-virtual-world-rings"><name>Example: Virtual world rings</name>
<iref item="Example: Virtual world rings"/><t>Consider 3D scenes linking to eachother using these <tt>href</tt> values, attached to 3D button-objects:</t>
<ul spacing="compact">
<li><tt>href: schoolA.edu/projects.gltf#math</tt></li>
<li><tt>href: schoolB.edu/projects.gltf#math</tt></li>
<li><tt>href: university.edu/projects.gltf#math</tt></li>
</ul>
<t>This would teleport users to the math-projects of those universities.<br />
Now consider adding a 'webring index'-button to each file, with this href-value:</t>
<ul spacing="compact">
<li>href: workgroup.edu/webrings.glb#!webringmenu</li>
</ul>
<t>This would allow displaying the (remote 3D file) webring menu with various href-buttons inside, all centrally curated by the workgroup.</t>
</section>
<section anchor="additional-scene-metadata"><name>Additional scene metadata</name>
<iref item="Additional scene metadata"/><t>XR Fragments does not aim to redefine the metadata-space or accessibility-space by introducing its own cataloging-metadata fields.
Instead, it encourages browsers to scan nodes for the following custom properties:</t>
<ul spacing="compact">
<li><eref target="https://spdx.dev/">SPDX</eref> license information</li>
<li><eref target="https://www.w3.org/WAI/standards-guidelines/aria/">ARIA</eref> attributes (<tt>aria-*: .....</tt>)</li>
<li><eref target="https://datapackage.org">datapackage.json</eref> findability, accessibility, interoperability, and reusability of data</li>
</ul>
<t>ARIA's <tt>aria-description</tt>-metadata is normative, to aid accessibility and scene transcripts</t>
<blockquote><t><strong>NOTE</strong>: please always start <tt>aria-description</tt> with a verb to aid transcripts.</t>
</blockquote><t>The following metadata are non-normative but encouraged, since they are popular and cheap to parse:</t>
<ul spacing="compact">
<li><eref target="https://json-ld.org">RDF/JSON-LD</eref> like <eref target="https://mvmd.org/standards/gltf/">this example</eref> or via glTF's <eref target="https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Khronos/KHR_xmp_json_ld">KHR_xmp_json_ld extension</eref></li>
<li><eref target="https://ogp.me">Open Graph</eref> attributes (<tt>og:*: .....</tt>)</li>
<li><eref target="https://www.dublincore.org/specifications/dublin-core/application-profile-guidelines/">Dublin-Core</eref> attributes(<tt>dc:*: .....</tt>)</li>
<li><eref target="https://bibtex.eu/fields">BibTex</eref> when known bibtex-keys exist with values enclosed in <tt>{</tt> and <tt>},</tt></li>
</ul>
<blockquote><t>Example: object 'tryceratops' with <tt>aria-description: is a huge dinosaurus standing on a #mountain</tt> generates transcript <tt>#tryceratops is a huge dinosaurus standing on a #mountain</tt>, where the hashtags are clickable XR Fragments (activating the visible-links in the XR browser).</t>
</blockquote><t>Individual nodes can be enriched with such metadata, but most importantly the scene node:</t>
<table>
<thead>
<tr>
<th>metadata key</th>
<th>example value</th>
</tr>
</thead>
<tbody>
<tr>
<td><tt>aria-description</tt>, <tt>og:description</tt>, <tt>dc:description</tt></td>
<td><tt>An immersive experience about Triceratops</tt> (*)</td>
</tr>
<tr>
<td><tt>SPDX</tt></td>
<td><tt>CC0-1.0</tt></td>
</tr>
<tr>
<td><tt>dc:creator</tt></td>
<td><tt>John Doe</tt></td>
</tr>
<tr>
<td><tt>dc:title</tt>, <tt>og:title</tt></td>
<td>'Triceratops` (*)</td>
</tr>
<tr>
<td><tt>og:site_name</tt></td>
<td><tt>https://xrfragment.org</tt></td>
</tr>
<tr>
<td><tt>dc.publisher</tt></td>
<td><tt>NLNET</tt></td>
</tr>
<tr>
<td><tt>dc.date</tt></td>
<td><tt>2024-01-01</tt></td>
</tr>
<tr>
<td><tt>dc.identifier</tt></td>
<td><tt>XRFRAGMENT-001</tt></td>
</tr>
<tr>
<td><tt>journal</tt> (bibTeX)</td>
<td><tt>{Future Of Text Vol 3},</tt></td>
</tr>
</tbody>
</table><blockquote><t>* = these are interchangable (only one needs to be defined)</t>
</blockquote><t>There's no silver bullet when it comes to metadata, so XR Fragment-implementations should support where the metadata is/goes.</t>
<blockquote><t>These attributes can be scanned and presented during an <tt>href</tt> or <tt>src</tt> eye/mouse-over.</t>
</blockquote></section>
<section anchor="accessibility-interface"><name>Accessibility interface</name>
<iref item="Accessibility interface"/><t>The addressibility of XR Fragments allows for unique 3D-to-text transcripts, as well as an textual interface to navigate 3D content.<br />
Spec:<br />
&lt;Br&gt;</t>
<ol spacing="compact">
<li>The enduser must be able to enable an accessibility-mode (which persists across application/webpage restarts)</li>
<li>Accessibility-mode must contain a text-input for the user to enter text</li>
<li>Accessibility-mode must contain a flexible textlog for the user to read (via screenreader, screen, or TTS e.g.)</li>
<li>the textlog contains <tt>aria-descriptions</tt>, and its narration (Screenreader e.g.) can be skipped (via 2-button navigation)</li>
<li>The <tt>back</tt> command should navigate back to the previous URL (alias for browser-backbutton)</li>
<li>The <tt>forward</tt> command should navigate back to the next URL (alias for browser-nextbutton)</li>
<li>A destination is a 3D node containing an <tt>href</tt> with a <tt>#...</tt> XR fragment (which matches a 3d object name)</li>
<li>The <tt>go</tt> command should list all possible destinations</li>
<li>The <tt>go left</tt> command should move the camera around 0.3 meters to the left</li>
<li>The <tt>go right</tt> command should move the camera around 0.3 meters to the right</li>
<li>The <tt>go forward</tt> command should move the camera 0.3 meters forward (direction of current rotation).</li>
<li>The <tt>rotate left</tt> command should rotate the camera 0.3 to the left</li>
<li>The <tt>rotate left</tt> command should rotate the camera 0.3 to the right</li>
<li>The (dynamic) <tt>go abc</tt> command should navigate to <tt>#scene2</tt> in case there's a 3D node with name <tt>abc</tt> and <tt>href</tt> value <tt>#scene2</tt></li>
<li>The <tt>look</tt> command should give an (contextual) 3D-to-text transcript, by scanning the <tt>aria-description</tt> values of the current <tt>#...</tt> (3D object) value (including its children)</li>
<li>The <tt>do</tt> command should list all possible <tt>href</tt> values which don't contain an <tt>#...</tt> XR Fragment</li>
<li>The (dynamic) <tt>do abc</tt> command should navigate/execute <tt>https://.../...</tt> in case a 3D node exist with name <tt>abc</tt> and <tt>href</tt> value <tt>https://.../...</tt></li>
</ol>
<section anchor="two-button-navigation"><name>Two-button navigation</name>
<t>For specific user-profiles, gyroscope/mouse/keyboard/audio/visuals will not be available.<br />
Therefore a 2-button navigation-interface is the bare minimum interface:</t>
<ol spacing="compact">
<li>objects with href metadata can be cycled via a key (tab on a keyboard)</li>
<li>objects with href metadata can be activated via a key (enter on a keyboard)</li>
<li>the TTS reads the href-value (and/or aria-description if available)</li>
</ol>
</section>
<section anchor="overlap-with-fileformat-specific-extensions"><name>Overlap with fileformat-specific extensions</name>
<t>Some 3D scene-fileformats have support for extensions.
What if the functionality of those overlap?
For example, GLTF has the <tt>OMI_LINK</tt> extension which might overlap with XR Fragment's <tt>href</tt>:</t>
<blockquote><t>Priority Order and Precedence, otherwise fallback applies</t>
</blockquote><t>1.<strong>Extensions Take Precedence</strong>: Since glTF-specific extensions are designed with the format’s
specific needs and optimizations in mind, they should take precedence over extras metadata
in cases where both contain overlapping functionality.
This approach aligns with the idea that extensions are more likely to be interpreted uniformly by glTF-compatible software.</t>
<ol spacing="compact" start="2">
<li><strong>Fallback Fall-through Mechanism</strong>:
If a glTF implementation does not support a particular extension, the (XRF) extras field can serve as a fallback. This way, metadata provided in extras can still be useful for applications that don't handle certain extensions.</li>
</ol>
<blockquote><t><strong>Example 1</strong> In case of the OMI_LINK glTF extension (<tt>href: https://nlnet.nl</tt>) and an XR Fragment (<tt>href: #otherroom</tt> or <tt>href: otherplanet.glb</tt>), it is clear that <tt>https://nlnet.nl</tt> should open in a browsertab, whereas the XR Fragment links should teleport the user. If the OMI_LINK contains an XR Fragment (<tt>#room1</tt> e.g.) a teleport should be performed only (and other [overlapping] metadata should be ignored).</t>
<t><strong>Example 2</strong> If an Extensions uses XR Fragments in URI's (<tt>href: #otherroom</tt> or <tt>href: xrf://-walls</tt> in OMI_LINK e.g.), then perform them according to XR Fragment spec (teleport user). But only once: ignore further overlapping metadata for that usecase.</t>
</blockquote></section>
</section>
<section anchor="vendor-prefixes"><name>Vendor Prefixes</name>
<iref item="Vendor Prefixes"/><t>Vendor-specific metadata in a 3D scenefiles, are similar to vendor-specific <eref target="https://en.wikipedia.org/wiki/CSS#Vendor_prefixes">CSS-prefixes</eref> (<tt>-moz-opacity: 0.2</tt> e.g.).
This allows popular 3D engines/frameworks, to initialize specific features when loading a scene/object, in a progressive enhanced way.</t>
<t>Vendor Prefixes allows embedding 3D engines/framework-specific features a 3D file via metadata:</t>
<table>
<thead>
<tr>
<th>what</th>
<th>XR metadata</th>
<th>Lowest common denominator</th>
</tr>
</thead>
<tbody>
<tr>
<td>CSS</td>
<td>vendor-agnostic</td>
<td>2D canvas + object referencing/styling</td>
</tr>
<tr>
<td>XR Fragments</td>
<td>vendor-agnostic</td>
<td>3D camera + object(file) load/embed/click/referencing</td>
</tr>
<tr>
<td>Vendor prefixs</td>
<td>vendor-<strong>specific</strong></td>
<td>Specialized Entity-Component implementation</td>
</tr>
</tbody>
</table><blockquote><t>Why? Because not all XR interactions can/should be solved/standardized by embedding XR Fragments into any 3D file.
The lowest common denominator between 3D engines is the 'entity'-part of their entity-component-system (ECS). The 'component'-part can be progressively enhanced via vendor prefixes.</t>
</blockquote><t>For example, the following metadata can be added to a .glb file, to make an object grabbable in AFRAME:</t>
<artwork><![CDATA[+────────────────────────────────────────────────────────────────────────────────────────────────────────+
│ http://y.io/z.glb | AFRAME app │
│-----------------------------------------------+--------------------------------------------------------│
│ | │
│ | after loading the glb, john can be placed into the │
│ +-[3D mesh]-+ | castle via hands, because the author added metadata to │
│ | / \ | | john via either: │
│ | / \ | | │
│ | / \ | | 1. Blender (custom property-box, no plugins needed) │
│ | |_____| | | │
│ +-----│-----+ | 2. javascript-code: │
│ │ | │
│ ├─ name: castle | for( var com in this.el.components ){ │
│ └─ tag: house baroque | this.el.object3D.userData[`-AFRAME-${com}`] = '' │
│ | } │
│ [3D mesh-+ | // save to z.glb in AFRAME inspector │
│ | ├─ name: john | │
│ | O ├─ age: 23 | │
│ | /|\ ├─ -aframe-grabbable: '' | > inits 'grabbable' component on object john │
│ | / \ ├─ -aframe-material.color: '#F0A' | > inits 'material' component on object john │
│ | ├─ -aframe-text.value: '{name}{age}'| > inits 'text' component (*) with value 'john' │
│ | ├─ -three-material.fog: false | > changes material settings in THREE.js app │
│ | ├─ -godot-Label3D.text: '{name}{age}'| > inits 'Label3D' component (*) in Godot │
│ +--------+ | │
│ | │
├─ -GODOT-version: '4.3' | > exporters/authors can report targeted version │
├─ -AFRAME-version: '1.6.0' | and (optionally) hint component-repo│
├─ -AFRAME-info: 'https://git.benetou.fr/comps' │
│ | │
+────────────────────────────────────────────────────────────────────────────────────────────────────────+
]]>
</artwork>
<ul spacing="compact">
<li>key/value syntax: -<tt>&lt;vendorname&gt;</tt>-<tt>&lt;component|version&gt;</tt>.<tt>&lt;key&gt;</tt> <tt>[string/boolean/float/int]</tt>-value</li>
</ul>
<t>String-templatevalues are evaluated as per <eref target="https://www.rfc-editor.org/rfc/rfc6570">URI Templates (RFC6570)</eref> Level 1.</t>
<blockquote><t>This 'separating of mechanism from policy' (unix rule) does <strong>somewhat</strong> break portability of an XR experience, but still prevents (E-waste of) handcoded virtual worlds. It allows for (XR experience) metadata to survive in future 3D engines and scene-fileformats.</t>
</blockquote></section>
<section anchor="security-considerations"><name>Security Considerations</name>
<iref item="Security Considerations"/><t>The only dynamic parts are <eref target="https://www.w3.org/TR/media-frags/">W3C Media Fragments</eref> and <eref target="https://www.rfc-editor.org/rfc/rfc6570">URI Templates (RFC6570)</eref>.<br />
The use of URI Templates is limited to pre-defined variables and Level0 fragments-expansion only, which makes it quite safe.<br />
n fact, it is much safer than relying on a scripting language (javascript) which can change URN too.</t>
</section>
<section anchor="faq"><name>FAQ</name>
<iref item="FAQ"/><t><strong>Q:</strong> Why is everything HTTP GET-based, what about POST/PUT/DELETE HATEOS<br />
<strong>A:</strong> Because it's out of scope: XR Fragment specifies a read-only way to surf XR documents. These things belong in the application layer (for example, an XR Hypermedia browser can decide to support POST/PUT/DELETE requests for embedded HTML thru <tt>src</tt> values)</t>
<t><strong>Q:</strong> Why isn't there support for scripting, URI Template Fragments are so limited compared to WASM &amp; javascript
<strong>A:</strong> This is out of scope as it unhyperifies hypermedia, and this is up to XR hypermedia browser-extensions.<br />
Historically scripting/Javascript seems to been able to turn webpages from hypermedia documents into its opposite (hyperscripted nonhypermedia documents).<br />
In order to prevent this backward-movement (hypermedia tends to liberate people from finnicky scripting) XR Fragment uses <eref target="https://www.w3.org/TR/media-frags/">W3C Media Fragments</eref> and <eref target="https://www.rfc-editor.org/rfc/rfc6570">URI Templates (RFC6570)</eref>, to prevent unhyperifying itself by hardcoupling to a particular markup or scripting language. <br />
XR Fragments supports filtering objects in a scene only, because in the history of the javascript-powered web, showing/hiding document-entities seems to be one of the most popular basic usecases.<br />
Doing advanced scripting &amp; networkrequests under the hood are obviously interesting endavours, but this is something which should not be hardcoupled with XR Fragments or hypermedia.<br />
This perhaps belongs more to browser extensions.<br />
Non-HTML Hypermedia browsers should make browser extensions the right place, to 'extend' experiences, in contrast to code/javascript inside hypermedia documents (this turned out as a hypermedia antipattern).</t>
</section>
<section anchor="authors"><name>authors</name>
<iref item="authors"/>
<ul spacing="compact">
<li>Leon van Kammen (@lvk@mastodon.online)</li>
<li>Jens Finkhäuser (@jens@social.finkhaeuser.de)</li>
</ul>
</section>
<section anchor="iana-considerations"><name>IANA Considerations</name>
<iref item="IANA Considerations"/><t>This document has no IANA actions.</t>
</section>
<section anchor="acknowledgments"><name>Acknowledgments</name>
<iref item="Acknowledgments"/>
<ul spacing="compact">
<li><eref target="https://nlnet.nl">NLNET</eref></li>
<li><eref target="https://futureoftext.org">Future of Text</eref></li>
<li><eref target="https://visual-meta.info">visual-meta.info</eref></li>
<li>Michiel Leenaars</li>
<li>Gerben van der Broeke</li>
<li>Mauve</li>
<li>Jens Finkhäuser</li>
<li>Marc Belmont</li>
<li>Tim Gerritsen</li>
<li>Frode Hegland</li>
<li>Brandel Zackernuk</li>
<li>Mark Anderson</li>
</ul>
</section>
<section anchor="appendix-definitions"><name>Appendix: Definitions</name>
<iref item="Appendix: Definitions"/><table>
<thead>
<tr>
<th>definition</th>
<th>explanation</th>
</tr>
</thead>
<tbody>
<tr>
<td>human</td>
<td>a sentient being who thinks fuzzy, absorbs, and shares thought (by plain text, not markuplanguage)</td>
</tr>
<tr>
<td>scene</td>
<td>a (local/remote) 3D scene or 3D file (index.gltf e.g.)</td>
</tr>
<tr>
<td>3D object</td>
<td>an object inside a scene characterized by vertex-, face- and customproperty data.</td>
</tr>
<tr>
<td>URI</td>
<td>some resource at something somewhere via someprotocol (<tt>http://me.com/foo.glb#foo</tt> or <tt>e76f8efec8efce98e6f</tt> <eref target="https://interpeer.io">see interpeer.io</eref>)</td>
</tr>
<tr>
<td>URL</td>
<td>something somewhere via someprotocol (<tt>http://me.com/foo.glb</tt>)</td>
</tr>
<tr>
<td>URN</td>
<td>something at some domain (<tt>me.com/foo.glb</tt>)</td>
</tr>
<tr>
<td>metadata</td>
<td>custom properties of text, 3D Scene or Object(nodes), relevant to machines and a human minority (academics/developers)</td>
</tr>
<tr>
<td>XR fragment</td>
<td>URI Fragment with spatial hints (which match the name of a 3D object-, camera-, animation-object)</td>
</tr>
<tr>
<td>the XRWG</td>
<td>wordgraph (collapses 3D scene to tags)</td>
</tr>
<tr>
<td>the hashbus</td>
<td>hashtags map to camera/scene-projections</td>
</tr>
<tr>
<td>spacetime hashtags</td>
<td>positions camera, triggers scene-preset/time</td>
</tr>
<tr>
<td>teleportation</td>
<td>repositioning the enduser to a different position (or 3D scene/file)</td>
</tr>
<tr>
<td>sourceportation</td>
<td>teleporting the enduser to the original XR Document of an <tt>src</tt> embedded object.</td>
</tr>
<tr>
<td>placeholder object</td>
<td>a 3D object which with src-metadata (which will be replaced by the src-data.)</td>
</tr>
<tr>
<td>src</td>
<td>(HTML-piggybacked) metadata of a 3D object which instances content</td>
</tr>
<tr>
<td>href</td>
<td>(HTML-piggybacked) metadata of a 3D object which links to content</td>
</tr>
<tr>
<td>filter</td>
<td>URI Fragment(s) which show/hide object(s) in a scene based on name/tag/property (<tt>#cube&amp;-price=&gt;3</tt>)</td>
</tr>
<tr>
<td>visual-meta</td>
<td><eref target="https://visual.meta.info">visual-meta</eref> data appended to text/books/papers which is indirectly visible/editable in XR.</td>
</tr>
<tr>
<td>requestless metadata</td>
<td>metadata which never spawns new requests (unlike RDF/HTML, which can cause framerate-dropping, hence not used a lot in games)</td>
</tr>
<tr>
<td>FPS</td>
<td>frames per second in spatial experiences (games,VR,AR e.g.), should be as high as possible</td>
</tr>
<tr>
<td>introspective</td>
<td>inward sensemaking (&quot;I feel this belongs to that&quot;)</td>
</tr>
<tr>
<td>extrospective</td>
<td>outward sensemaking (&quot;I'm fairly sure John is a person who lives in oklahoma&quot;)</td>
</tr>
<tr>
<td><tt>◻</tt></td>
<td>ascii representation of an 3D object/mesh</td>
</tr>
<tr>
<td>(un)obtrusive</td>
<td>obtrusive: wrapping human text/thought in XML/HTML/JSON obfuscates human text into a salad of machine-symbols and words</td>
</tr>
<tr>
<td>flat 3D object</td>
<td>a 3D object of which all verticies share a plane</td>
</tr>
<tr>
<td>BibTeX</td>
<td>simple tagging/citing/referencing standard for plaintext</td>
</tr>
<tr>
<td>BibTag</td>
<td>a BibTeX tag</td>
</tr>
<tr>
<td>(hashtag)bibs</td>
<td>an easy to speak/type/scan tagging SDL (<eref target="https://github.com/coderofsalvation/hashtagbibs">see here</eref> which expands to BibTex/JSON/XML</td>
</tr>
</tbody>
</table></section>
</middle>
</rfc>
Reference in a new issue View git blame Copy permalink