xrfragment/doc/RFC_XR_Fragments.txt

1513 lines
63 KiB
Plaintext
Raw Normal View History

2023-09-01 14:20:02 +02:00
Internet Engineering Task Force L.R. van Kammen
2023-09-21 13:05:30 +02:00
Internet-Draft 21 September 2023
2023-09-01 14:20:02 +02:00
Intended status: Informational
2023-09-04 21:20:59 +02:00
2023-09-01 14:20:02 +02:00
XR Fragments
draft-XRFRAGMENTS-leonvankammen-00
Abstract
2023-09-18 11:03:18 +02:00
This draft is a specification for 4D URLs & navigation, which links
together space, time & text together, for hypermedia browsers with-
or without a network-connection.
2023-09-01 14:20:02 +02:00
The specification promotes spatial addressibility, sharing,
2023-09-18 11:03:18 +02:00
navigation, query-ing and annotating interactive (text)objects across
2023-09-04 21:20:59 +02:00
for (XR) Browsers.
2023-09-01 14:20:02 +02:00
XR Fragments allows us to enrich existing dataformats, by recursive
2023-09-04 21:20:59 +02:00
use of existing proven technologies like URI Fragments
2023-09-06 15:43:29 +02:00
(https://en.wikipedia.org/wiki/URI_fragment) and BibTags notation.
2023-09-05 19:14:10 +02:00
Almost every idea in this document is demonstrated at
https://xrfragment.org (https://xrfragment.org)
2023-09-01 14:20:02 +02:00
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
2023-09-21 13:05:30 +02:00
This Internet-Draft will expire on 24 March 2024.
2023-09-01 14:20:02 +02:00
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 1]
2023-09-01 14:20:02 +02:00
2023-09-02 21:44:57 +02:00
Internet-Draft XR Fragments September 2023
2023-09-01 14:20:02 +02:00
2023-09-18 11:03:18 +02:00
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
2023-09-05 19:14:10 +02:00
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
2023-09-01 14:20:02 +02:00
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2023-09-05 19:14:10 +02:00
2. Core principle . . . . . . . . . . . . . . . . . . . . . . . 3
2023-09-21 13:05:30 +02:00
3. Conventions and Definitions . . . . . . . . . . . . . . . . . 5
3.1. XR Fragment URI Grammar . . . . . . . . . . . . . . . . . 5
4. List of URI Fragments . . . . . . . . . . . . . . . . . . . . 6
5. List of metadata for 3D nodes . . . . . . . . . . . . . . . . 6
6. Spatial Referencing 3D . . . . . . . . . . . . . . . . . . . 7
7. Navigating 3D . . . . . . . . . . . . . . . . . . . . . . . . 8
8. Top-level URL processing . . . . . . . . . . . . . . . . . . 9
9. Embedding XR content (src-instancing) . . . . . . . . . . . . 9
2023-09-21 13:30:14 +02:00
10. Navigating content (href portals) . . . . . . . . . . . . . . 12
2023-09-21 13:05:30 +02:00
10.1. UX spec . . . . . . . . . . . . . . . . . . . . . . . . 13
10.2. Scaling instanced content . . . . . . . . . . . . . . . 13
2023-09-21 13:30:14 +02:00
11. XR Fragment queries . . . . . . . . . . . . . . . . . . . . . 14
2023-09-21 13:05:30 +02:00
11.1. including/excluding . . . . . . . . . . . . . . . . . . 14
11.2. Query Parser . . . . . . . . . . . . . . . . . . . . . . 15
2023-09-21 13:30:14 +02:00
12. Visible links . . . . . . . . . . . . . . . . . . . . . . . . 16
2023-09-21 13:05:30 +02:00
13. Text in XR (tagging,linking to spatial objects) . . . . . . . 16
13.1. Default Data URI mimetype . . . . . . . . . . . . . . . 20
13.2. URL and Data URI . . . . . . . . . . . . . . . . . . . . 21
13.3. XR Text example parser . . . . . . . . . . . . . . . . . 22
14. Security Considerations . . . . . . . . . . . . . . . . . . . 24
2023-09-21 13:30:14 +02:00
15. FAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2023-09-21 13:05:30 +02:00
16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25
17. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 25
18. Appendix: Definitions . . . . . . . . . . . . . . . . . . . . 25
2023-09-01 14:20:02 +02:00
1. Introduction
How can we add more features to existing text & 3D scenes, without
2023-09-04 21:20:59 +02:00
introducing new dataformats?
Historically, there's many attempts to create the ultimate
markuplanguage or 3D fileformat.
2023-09-07 15:53:32 +02:00
Their lowest common denominator is: (co)authoring using plain text.
2023-09-05 19:14:10 +02:00
XR Fragments allows us to enrich/connect existing dataformats, by
2023-09-21 13:05:30 +02:00
introducing existing technologies/ideas:
2023-09-04 21:20:59 +02:00
2023-09-05 19:14:10 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 2]
2023-09-05 19:14:10 +02:00
Internet-Draft XR Fragments September 2023
2023-09-04 21:20:59 +02:00
2023-09-21 13:05:30 +02:00
1. addressibility and navigation of 3D scenes/objects: URI Fragments
(https://en.wikipedia.org/wiki/URI_fragment) + src/href spatial
metadata
2023-09-18 11:03:18 +02:00
2. Interlinking text/& 3D by collapsing space into a Word Graph
2023-09-21 13:05:30 +02:00
(XRWG) to show visible links (#visible-links) (and augmenting
text with bibs (https://github.com/coderofsalvation/tagbibs) /
BibTags (https://en.wikipedia.org/wiki/BibTeX) appendices (see
visual-meta (https://visual-meta.info) e.g.)
3. unlocking spatial potential of the (originally 2D) hashtag (which
jumps to a chapter) for navigating XR documents
2023-09-18 11:03:18 +02:00
2023-09-04 21:20:59 +02:00
| NOTE: The chapters in this document are ordered from highlevel to
| lowlevel (technical) as much as possible
2023-09-05 19:14:10 +02:00
2. Core principle
2023-09-04 21:20:59 +02:00
2023-09-05 19:14:10 +02:00
XR Fragments strives to serve (nontechnical/fuzzy) humans first, and
machine(implementations) later, by ensuring hasslefree text-vs-
thought feedback loops.
This also means that the repair-ability of machine-matters should be
human friendly too (not too complex).
2023-09-18 11:03:18 +02:00
XR Fragments tries to seek to connect the world of text (semantical
web / RDF), and the world of pixels.
Instead of combining them (in a game-editor e.g.), XR Fragments is
opting for a more integrated path *towards* them, by describing how
to make browsers *4D URL-ready*:
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 3]
Internet-Draft XR Fragments September 2023
2023-09-21 16:20:38 +02:00
+===========+=============================+====================+
| principle | XR 4D URL | HTML 2D URL |
+===========+=============================+====================+
| the XRWG | wordgraph (collapses 3D | Ctrl-F (find) |
| | scene to tags) | |
+-----------+-----------------------------+--------------------+
| the | hashtags map to camera/ | hashtags map to |
| hashbus | scene-projections | document positions |
+-----------+-----------------------------+--------------------+
| spacetime | positions camera, triggers | jumps/scrolls to |
| hashtags | scene-preset/time | chapter |
+-----------+-----------------------------+--------------------+
| src | renders content and offers | renders content |
| metadata | sourceportation | |
+-----------+-----------------------------+--------------------+
| href | teleports to other XR | jumps to other |
| metadata | document | HTML document |
+-----------+-----------------------------+--------------------+
| href | repositions camera or | jumps to camera |
| metadata | animation-range | |
+-----------+-----------------------------+--------------------+
| href | draws visible connection(s) | |
| metadata | for XRWG 'tag' | |
+-----------+-----------------------------+--------------------+
| href | triggers predefined view | Media fragments |
| metadata | | |
+-----------+-----------------------------+--------------------+
2023-09-21 13:05:30 +02:00
Table 1
2023-09-18 11:03:18 +02:00
| XR Fragments does not look at XR (or the web) thru the lens of
| HTML.
2023-09-21 16:20:38 +02:00
| But approaches things from a higherlevel feedbackloop/hypermedia
| browser-perspective:
2023-09-21 13:05:30 +02:00
2023-09-18 11:03:18 +02:00
2023-09-04 21:20:59 +02:00
2023-09-01 14:20:02 +02:00
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 4]
2023-09-18 11:03:18 +02:00
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
+──────────────────────────────────────────────────────────────────────────────────────────────+
│ │
│ the soul of any URL: ://macro /meso ?micro #nano │
│ │
│ 2D URL: ://library.com /document ?search #chapter │
│ │
│ 4D URL: ://park.com /4Dscene.fbx ──> ?misc ──> #view ───> hashbus │
│ │ #query │ │
│ │ #tag │ │
│ │ │ │
│ XRWG <─────────────────────<────────────+ │
│ │ │ │
│ ├─ objects ───────────────>────────────│ │
│ └─ text ───────────────>────────────+ │
│ │
│ │
+──────────────────────────────────────────────────────────────────────────────────────────────+
2023-09-18 11:03:18 +02:00
Traditional webbrowsers can become 4D document-ready by:
2023-09-21 16:20:38 +02:00
* loading 3D assets (gltf/fbx e.g.) natively (with or without using
HTML).
2023-09-18 11:03:18 +02:00
* allowing assets to publish hashtags to themselves (the scene)
using the hashbus (like hashtags controlling the scrollbar).
* collapsing the 3D scene to an wordgraph (for essential navigation
purposes) controllable thru a hash(tag)bus
2023-09-21 16:20:38 +02:00
XR Fragments itself are HTML-agnostic, though pseudo-XR Fragment
2023-09-07 15:53:32 +02:00
browsers *can* be implemented on top of HTML/Javascript.
2023-09-04 21:20:59 +02:00
2023-09-07 15:53:32 +02:00
3. Conventions and Definitions
2023-09-04 21:20:59 +02:00
2023-09-07 15:53:32 +02:00
See appendix below in case certain terms are not clear.
2023-09-04 21:20:59 +02:00
2023-09-18 11:03:18 +02:00
3.1. XR Fragment URI Grammar
2023-09-04 21:20:59 +02:00
2023-09-18 11:03:18 +02:00
reserved = gen-delims / sub-delims
gen-delims = "#" / "&"
sub-delims = "," / "="
| Example: ://foo.com/my3d.gltf#pos=1,0,0&prio=-5&t=0,100
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 5]
Internet-Draft XR Fragments September 2023
2023-09-18 11:03:18 +02:00
+=============================+=================================+
| Demo | Explanation |
+=============================+=================================+
| pos=1,2,3 | vector/coordinate argument e.g. |
+-----------------------------+---------------------------------+
| pos=1,2,3&rot=0,90,0&q=.foo | combinators |
+-----------------------------+---------------------------------+
2023-09-04 21:20:59 +02:00
2023-09-18 11:03:18 +02:00
Table 2
2023-09-05 19:14:10 +02:00
2023-09-18 11:03:18 +02:00
| this is already implemented in all browsers
4. List of URI Fragments
+==========+=========+==============+==============================+
| fragment | type | example | info |
+==========+=========+==============+==============================+
| #pos | vector3 | #pos=0.5,0,0 | positions camera (or XR |
| | | | floor) to xyz-coord 0.5,0,0, |
+----------+---------+--------------+------------------------------+
| #rot | vector3 | #rot=0,90,0 | rotates camera to xyz-coord |
| | | | 0.5,0,0 |
+----------+---------+--------------+------------------------------+
| #t | vector2 | #t=500,1000 | sets animation-loop range |
| | | | between frame 500 and 1000 |
+----------+---------+--------------+------------------------------+
| #...... | string | #.cubes | predefined views, XRWG |
| | | #cube | fragments and ID fragments |
+----------+---------+--------------+------------------------------+
Table 3
2023-09-06 15:13:36 +02:00
| xyz coordinates are similar to ones found in SVG Media Fragments
2023-09-04 21:20:59 +02:00
5. List of metadata for 3D nodes
2023-09-21 13:05:30 +02:00
+======+========+=============+===========+=========================+
| key | type | example | function | existing compatibility |
| | | (JSON) | | |
+======+========+=============+===========+=========================+
| name | string | "name": | identify/ | object supported in all |
| | | "cube" | tag | 3D fileformats & scenes |
+------+--------+-------------+-----------+-------------------------+
| tag | string | "tag": | tag | custom property in 3D |
| | | "cubes | object | fileformats |
| | | geo" | | |
+------+--------+-------------+-----------+-------------------------+
| href | string | "href": | XR | custom property in 3D |
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 6]
2023-09-18 11:03:18 +02:00
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
| | | "b.gltf" | teleport | fileformats |
+------+--------+-------------+-----------+-------------------------+
| src | string | "src": | XR embed | custom property in 3D |
| | | "#cube" | / | fileformats |
| | | | teleport | |
+------+--------+-------------+-----------+-------------------------+
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
Table 4
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
Supported popular compatible 3D fileformats: .gltf, .obj, .fbx,
.usdz, .json (THREE.js), .dae and so on.
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
| NOTE: XR Fragments are file-agnostic, which means that the
| metadata exist in programmatic 3D scene(nodes) too.
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
6. Spatial Referencing 3D
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
my.io/scene.fbx
+─────────────────────────────+
│ sky │ src: http://my.io/scene.fbx#sky (includes building,mainobject,floor)
│ +─────────────────────────+ │
│ │ building │ │ src: http://my.io/scene.fbx#building (includes mainobject,floor)
│ │ +─────────────────────+ │ │
│ │ │ mainobject │ │ │ src: http://my.io/scene.fbx#mainobject (includes floor)
│ │ │ +─────────────────+ │ │ │
│ │ │ │ floor │ │ │ │ src: http://my.io/scene.fbx#floor (just floor object)
│ │ │ │ │ │ │ │
│ │ │ +─────────────────+ │ │ │
│ │ +─────────────────────+ │ │
│ +─────────────────────────+ │
+─────────────────────────────+
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
Clever nested design of 3D scenes allow great ways for re-using
content, and/or previewing scenes.
For example, to render a portal with a preview-version of the scene,
create an 3D object with:
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
* href: https://scene.fbx
* src: https://otherworld.gltf#mainobject
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
| It also allows *sourceportation*, which basically means the
| enduser can teleport to the original XR Document of an src
| embedded object, and see a visible connection to the particular
| embedded object.
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 7]
2023-09-18 11:03:18 +02:00
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
7. Navigating 3D
2023-09-04 21:20:59 +02:00
2023-09-21 13:05:30 +02:00
+====================+=========+==================================+
| fragment | type | functionality |
+====================+=========+==================================+
| <b>#pos</b>=0,0,0 | vector3 | (re)position camera |
+--------------------+---------+----------------------------------+
| <b>#t</b>=0,100 | vector2 | (re)position looprange of scene- |
| | | animation or src-mediacontent |
+--------------------+---------+----------------------------------+
| <b>#rot</b>=0,90,0 | vector3 | rotate camera |
+--------------------+---------+----------------------------------+
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
Table 5
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
&#187; example implementation
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/pos.js)
&#187; discussion (https://github.com/coderofsalvation/xrfragment/
issues/5)
1. the Y-coordinate of pos identifies the floorposition. This means
that desktop-projections usually need to add 1.5m (average person
height) on top (which is done automatically by VR/AR headsets).
2. set the position of the camera accordingly to the vector3 values
of #pos
3. rot sets the rotation of the camera (only for non-VR/AR headsets)
4. t sets the animation-range of the current scene animation(s) or
src-mediacontent (video/audioframes e.g., use t=7,7 to 'STOP' at
certain frame)
5. in case an href does not mention any pos-coordinate, pos=0,0,0
will be assumed
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
Here's an ascii representation of a 3D scene-graph which contains 3D
objects &#9723; and their metadata:
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
+────────────────────────────────────────────────────────+
│ │
│ index.gltf │
│ │ │
│ ├── ◻ buttonA │
│ │ └ href: #pos=1,0,1&t=100,200 │
│ │ │
│ └── ◻ buttonB │
│ └ href: other.fbx │ <── file─agnostic (can be .gltf .obj etc)
│ │
+────────────────────────────────────────────────────────+
2023-09-18 11:03:18 +02:00
2023-09-07 14:06:50 +02:00
2023-09-07 15:53:32 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 8]
2023-09-07 15:53:32 +02:00
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
An XR Fragment-compatible browser viewing this scene, allows the end-
user to interact with the buttonA and buttonB.
In case of buttonA the end-user will be teleported to another
location and time in the *current loaded scene*, but buttonB will
*replace the current scene* with a new one, like other.fbx, and
assume pos=0,0,0.
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
8. Top-level URL processing
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
| Example URL: ://foo/world.gltf#cube&pos=0,0,0
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
The URL-processing-flow for hypermedia browsers goes like this:
2023-09-18 11:03:18 +02:00
2023-09-21 13:30:14 +02:00
1. IF a #cube matches a custom property-key (of an object) in the 3D
file/scene (#cube: #......) <b>THEN</b> execute that
predefined_view.
2. IF scene operators (pos) and/or animation operator (t) are
present in the URL then (re)position the camera and/or animation-
range accordingly.
3. IF no camera-position has been set in <b>step 1 or 2</b> update
the top-level URL with #pos=0,0,0 (example (https://github.com/co
derofsalvation/xrfragment/blob/main/src/3rd/js/three/
navigator.js#L31]]))
4. IF a #cube matches the name (of an object) in the 3D file/scene
then draw a line from the enduser('s heart) to that object (to
highlight it).
5. IF a #cube matches anything else in the XR Word Graph (XRWG) draw
wires to them (text or related objects).
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
9. Embedding XR content (src-instancing)
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
src is the 3D version of the <a target="_blank"
href="https://www.w3.org/html/wiki/Elements/iframe">iframe</a>.
It instances content (in objects) in the current scene/asset.
2023-09-18 11:03:18 +02:00
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 9]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
+========+==========+==============================================+
|fragment| type | example value |
+========+==========+==============================================+
|src | string | #cube |
| | (uri, | #sometag |
| | hashtag/ | #q=-ball_inside_cube<br>#q=-/sky |
| | query) | -rain<br>#q=-.language |
| | | .english<br>#q=price:>2 |
| | | price:<5`<br>https://linux.org/penguin.png |
| | | https://linux.world/distrowatch.gltf#t=1,100 |
| | | linuxapp://conference/nixworkshop/ |
| | | apply.gltf#q=flyer |
| | | androidapp://page1?tutorial#pos=0,0,1&t1,100 |
+--------+----------+----------------------------------------------+
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
Table 6
2023-09-07 15:53:32 +02:00
2023-09-21 13:05:30 +02:00
Here's an ascii representation of a 3D scene-graph with 3D objects
&#9723; which embeds remote & local 3D objects &#9723; with/out using
queries:
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
+────────────────────────────────────────────────────────+ +─────────────────────────+
│ │ │ │
│ index.gltf │ │ ocean.com/aquarium.fbx │
│ │ │ │ │ │
│ ├── ◻ canvas │ │ └── ◻ fishbowl │
│ │ └ src: painting.png │ │ ├─ ◻ bass │
│ │ │ │ └─ ◻ tuna │
│ ├── ◻ aquariumcube │ │ │
│ │ └ src: ://rescue.com/fish.gltf#bass%20tuna │ +─────────────────────────+
│ │ │
│ ├── ◻ bedroom │
│ │ └ src: #canvas │
│ │ │
│ └── ◻ livingroom │
│ └ src: #canvas │
│ │
+────────────────────────────────────────────────────────+
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
An XR Fragment-compatible browser viewing this scene, lazy-loads and
projects painting.png onto the (plane) object called canvas (which is
copy-instanced in the bed and livingroom).
Also, after lazy-loading ocean.com/aquarium.gltf, only the queried
objects bass and tuna will be instanced inside aquariumcube.
Resizing will be happen accordingly to its placeholder object
aquariumcube, see chapter Scaling.
2023-09-18 11:03:18 +02:00
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 10]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
| Instead of cherrypicking objects with #bass&tuna thru src, queries
| can be used to import the whole scene (and filter out certain
| objects). See next chapter below.
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
*Specification*:
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
1. local/remote content is instanced by the src (query) value (and
attaches it to the placeholder mesh containing the src property)
2. <b>local</b> src values (URL *starting* with #, like #cube&foo)
means *only* the mentioned objectnames will be copied to the
instanced scene (from the current scene) while preserving their
names (to support recursive selectors). (example code) (https://g
ithub.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/three/
xrf/src.js)
2023-09-18 11:03:18 +02:00
3. <b>local</b> src values indicating a query (#q=), means that all
included objects (from the current scene) will be copied to the
instanced scene (before applying the query) while preserving
their names (to support recursive selectors). (example code) (htt
ps://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/src.js)
4. the instanced scene (from a src value) should be <b>scaled
accordingly</b> to its placeholder object or <b>scaled
relatively</b> based on the scale-property (of a geometry-less
placeholder, an 'empty'-object in blender e.g.). For more info
see Chapter Scaling.
5. <b>external</b> src (file) values should be served with
appropriate mimetype (so the XR Fragment-compatible browser will
now how to render it). The bare minimum supported mimetypes are:
6. when the placeholder object is a 2D plane, but the mimetype is
3D, then render the spatial content on that plane via a stencil
buffer.
2023-09-21 13:05:30 +02:00
7. src-values are non-recursive: when linking to an external object
(src: foo.fbx#bar), then src-metadata on object bar should be
ignored.
8. clicking on external src-values always allow sourceportation:
teleporting to the origin URI to which the object belongs.
9. when only one object was cherrypicked (#cube e.g.), set its
2023-09-18 11:03:18 +02:00
position to 0,0,0
* model/gltf+json
* image/png
* image/jpg
* text/plain;charset=utf-8;bib=^@
2023-09-21 13:05:30 +02:00
&#187; example implementation
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/src.js)
&#187; example 3D asset
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 11]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
assets/src.gltf#L192)
&#187; discussion (https://github.com/coderofsalvation/xrfragment/
issues/4)
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
10. Navigating content (href portals)
2023-09-07 14:06:50 +02:00
2023-09-21 13:05:30 +02:00
navigation, portals & mutations
2023-09-07 14:06:50 +02:00
2023-09-18 11:03:18 +02:00
+==========+==================+============================+
| fragment | type | example value |
+==========+==================+============================+
| href | string (uri or | #pos=1,1,0 |
| | predefined view) | #pos=1,1,0&rot=90,0,0 |
| | | ://somefile.gltf#pos=1,1,0 |
+----------+------------------+----------------------------+
2023-09-21 13:05:30 +02:00
Table 7
2023-09-18 11:03:18 +02:00
1. clicking an ''external''- or ''file URI'' fully replaces the
current scene and assumes pos=0,0,0&rot=0,0,0 by default (unless
specified)
2. relocation/reorientation should happen locally for local URI's
(#pos=....)
3. navigation should not happen ''immediately'' when user is more
than 2 meter away from the portal/object containing the href (to
prevent accidental navigation e.g.)
4. URL navigation should always be reflected in the client (in case
of javascript: see [here
(https://github.com/coderofsalvation/xrfragment/blob/dev/src/3rd/
js/three/navigator.js) for an example navigator).
2023-09-21 13:05:30 +02:00
5. In XR mode, the navigator back/forward-buttons should be always
visible (using a wearable e.g., see [here
(https://github.com/coderofsalvation/xrfragment/blob/dev/example/
aframe/sandbox/index.html#L26-L29) for an example wearable)
6. in case of navigating to a new [[pos)ition, ''first'' navigate to
the ''current position'' so that the ''back-button'' of the
''browser-history'' always refers to the previous position (see
[here (https://github.com/coderofsalvation/xrfragment/blob/main/s
rc/3rd/js/three/xrf/href.js#L97))
7. portal-rendering: a 2:1 ratio texture-material indicates an
equirectangular projection
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 12]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
&#187; example implementation
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
three/xrf/href.js)
&#187; example 3D asset
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
assets/href.gltf#L192)
&#187; discussion (https://github.com/coderofsalvation/xrfragment/
issues/1)
10.1. UX spec
End-users should always have read/write access to:
1. the current (toplevel) <b>URL</b> (an URLbar etc)
2. URL-history (a <b>back/forward</b> button e.g.)
3. Clicking/Touching an href navigates (and updates the URL) to
another scene/file (and coordinate e.g. in case the URL contains
XR Fragments).
10.2. Scaling instanced content
Sometimes embedded properties (like src) instance new objects.
But what about their scale?
How does the scale of the object (with the embedded properties)
impact the scale of the referenced content?
| Rule of thumb: visible placeholder objects act as a '3D canvas'
| for the referenced scene (a plane acts like a 2D canvas for images
| e, a cube as a 3D canvas e.g.).
1. <b>IF</b> an embedded property (src e.g.) is set on an non-empty
placeholder object (geometry of >2 vertices):
* calculate the <b>bounding box</b> of the ''placeholder'' object
(maxsize=1.4 e.g.)
* hide the ''placeholder'' object (material e.g.)
* instance the src scene as a child of the existing object
* calculate the <b>bounding box</b> of the instanced scene, and
scale it accordingly (to 1.4 e.g.)
| REASON: non-empty placeholder object can act as a protective
| bounding-box (for remote content of which might grow over time
| e.g.)
2. ELSE multiply the scale-vector of the instanced scene with the
scale-vector (a common property of a 3D node) of the
<b>placeholder</b> object.
van Kammen Expires 24 March 2024 [Page 13]
Internet-Draft XR Fragments September 2023
2023-09-21 13:30:14 +02:00
| TODO: needs intermediate visuals to make things more obvious
11. XR Fragment queries
Include, exclude, hide/shows objects using space-separated strings:
2023-09-21 13:05:30 +02:00
+==================+==========================================+
| example | outcome |
+==================+==========================================+
| #q=-sky | show everything except object named sky |
+------------------+------------------------------------------+
| #q=-tag:language | hide everything with tag language, but |
| tag:english | show all tag english objects |
+------------------+------------------------------------------+
| #q=price:>2 | of all objects with property price, show |
| price:<5 | only objects with value between 2 and 5 |
+------------------+------------------------------------------+
Table 8
It's simple but powerful syntax which allows filtering the scene
using searchengine prompt-style feeling:
1. queries are a way to traverse a scene, and filter objects based
on their tag- or property-values.
2. words like german match tag-metadata of 3D objects like
"tag":"german"
3. words like german match (XR Text) objects with (Bib(s)TeX) tags
like #KarlHeinz@german or @german{KarlHeinz, ... e.g.
* see an (outdated) example video here
(https://coderofsalvation.github.io/xrfragment.media/queries.mp4)
11.1. including/excluding
2023-09-07 15:53:32 +02:00
2023-09-21 13:05:30 +02:00
+==========+=================================================+
| operator | info |
+==========+=================================================+
| - | removes/hides object(s) |
+----------+-------------------------------------------------+
| : | indicates an object-embedded custom property |
| | key/value |
+----------+-------------------------------------------------+
| > < | compare float or int number |
+----------+-------------------------------------------------+
| / | reference to root-scene. |
| | Useful in case of (preventing) showing/hiding |
| | objects in nested scenes (instanced by src) (*) |
2023-09-07 15:53:32 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 14]
2023-09-18 11:03:18 +02:00
Internet-Draft XR Fragments September 2023
2023-09-21 13:30:14 +02:00
+----------+-------------------------------------------------+
Table 9
2023-09-21 13:05:30 +02:00
| * = #q=-/cube hides object cube only in the root-scene (not nested
| cube objects)
| #q=-cube hides both object cube in the root-scene <b>AND</b>
| nested skybox objects |
2023-09-18 11:03:18 +02:00
&#187; example implementation
(https://github.com/coderofsalvation/xrfragment/blob/main/src/3rd/js/
2023-09-21 13:05:30 +02:00
three/xrf/q.js) &#187; example 3D asset
2023-09-18 11:03:18 +02:00
(https://github.com/coderofsalvation/xrfragment/blob/main/example/
2023-09-21 13:05:30 +02:00
assets/query.gltf#L192) &#187; discussion
(https://github.com/coderofsalvation/xrfragment/issues/3)
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
11.2. Query Parser
2023-09-07 15:53:32 +02:00
2023-09-21 13:05:30 +02:00
Here's how to write a query parser:
2023-09-07 15:53:32 +02:00
2023-09-21 13:05:30 +02:00
1. create an associative array/object to store query-arguments as
objects
2. detect object id's & properties foo:1 and foo (reference regex:
/^.*:[><=!]?/ )
3. detect excluders like -foo,-foo:1,-.foo,-/foo (reference regex:
/^-/ )
4. detect root selectors like /foo (reference regex: /^[-]?\// )
5. detect number values like foo:1 (reference regex: /^[0-9\.]+$/ )
6. for every query token split string on :
7. create an empty array rules
8. then strip key-operator: convert "-foo" into "foo"
9. add operator and value to rule-array
10. therefore we we set id to true or false (false=excluder -)
11. and we set root to true or false (true=/ root selector is
present)
12. we convert key '/foo' into 'foo'
13. finally we add the key/value to the store like store.foo =
{id:false,root:true} e.g.
2023-09-07 15:53:32 +02:00
2023-09-21 13:05:30 +02:00
| An example query-parser (which compiles to many languages) can be
| found here
| (https://github.com/coderofsalvation/xrfragment/blob/main/src/
| xrfragment/Query.hx)
2023-09-07 15:53:32 +02:00
2023-09-21 13:30:14 +02:00
2023-09-07 15:53:32 +02:00
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 15]
2023-09-05 19:14:10 +02:00
Internet-Draft XR Fragments September 2023
2023-09-21 13:30:14 +02:00
12. Visible links
When predefined views, XRWG fragments and ID fragments (#cube or
#mytag e.g.) are triggered by the enduser (via toplevel URL or
clicking href):
1. draw a wire from the enduser (preferabbly a bit below the camera,
heartposition) to object(s) matching that ID (objectname)
2023-09-21 13:05:30 +02:00
2. draw a wire from the enduser (preferabbly a bit below the camera,
heartposition) to object(s) matching that tag value
3. draw a wire from the enduser (preferabbly a bit below the camera,
heartposition) to object(s) containing that in their src or href
value
2023-09-21 13:30:14 +02:00
The obvious approach for this, is to consult the XRWG (example
(https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
src/3rd/js/XRWG.js)), which basically has all these things already
collected/organized for you during scene-load.
*UX*
4. do not update the wires when the enduser moves, leave them as is
5. offer a control near the back/forward button which allows the
user to (turn off) control the correlation-intensity of the XRWG
2023-09-21 13:05:30 +02:00
13. Text in XR (tagging,linking to spatial objects)
2023-09-09 11:30:03 +02:00
2023-09-18 11:03:18 +02:00
How does XR Fragments interlink text with objects?
| The XR Fragments does this by collapsing space into a *Word Graph*
2023-09-21 13:30:14 +02:00
| (the *XRWG* example
| (https://github.com/coderofsalvation/xrfragment/blob/feat/macros/
| src/3rd/js/XRWG.js)), augmented by Bib(s)Tex.
2023-09-18 11:03:18 +02:00
Instead of just throwing together all kinds media types into one
2023-09-21 13:05:30 +02:00
experience (games), what about their tagged/semantical relationships?
Perhaps the following question is related: why is HTML adopted less
in games outside the browser? Through the lens of constructive lazy
game-developers, ideally metadata must come *with* text, but not
*obfuscate* the text, or *spawning another request* to fetch it.
2023-09-18 11:03:18 +02:00
XR Fragments does this by detecting Bib(s)Tex, without introducing a
new language or fileformat
| Why Bib(s)Tex? Because its seems to be the lowest common
| denominator for an human-curated XRWG (extendable by
| speech/scanner/writing/typing e.g, see further motivation here
| (https://github.com/coderofsalvation/hashtagbibs#bibs--bibtex-
| combo-lowest-common-denominator-for-linking-data))
2023-09-09 11:30:03 +02:00
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 16]
Internet-Draft XR Fragments September 2023
2023-09-18 11:03:18 +02:00
Hence:
2023-09-09 11:30:03 +02:00
2023-09-21 13:05:30 +02:00
1. XR Fragments promotes (de)serializing a scene to the XRWG
2023-09-21 13:30:14 +02:00
(example (https://github.com/coderofsalvation/xrfragment/blob/fe
at/macros/src/3rd/js/XRWG.js))
2023-09-21 13:05:30 +02:00
2. XR Fragments primes the XRWG, by collecting words from the tag
and name-property of 3D objects.
3. XR Fragments primes the XRWG, by collecting words from
*optional* metadata *at the end of content* of text (see default
mimetype & Data URI)
4. Bib's (https://github.com/coderofsalvation/hashtagbibs) and
BibTex are first tag citizens for priming the XRWG with words
(from XR text)
5. Like Bibs, XR Fragments generalizes the BibTex author/title-
semantics (author{title}) into *this* points to *that*
(this{that})
6. The XRWG should be recalculated when textvalues (in src) change
7. HTML/RDF/JSON is still great, but is beyond the XRWG-scope (they
fit better in the application-layer)
8. Applications don't have to be able to access the XRWG
programmatically, as they can easily generate one themselves by
traversing the scene-nodes.
9. The XR Fragment focuses on fast and easy-to-generate end-user
controllable word graphs (instead of complex implementations
that try to defeat word ambiguity)
10. Tags are the scope for now (supporting https://github.com/WICG/
scroll-to-text-fragment (https://github.com/WICG/scroll-to-text-
fragment) will be considered)
Example:
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 17]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
http://y.io/z.fbx | Derived XRWG (expressed as BibTex)
2023-09-18 11:03:18 +02:00
----------------------------------------------------------------------------+--------------------------------------
| @house{castle,
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | url = {https://y.io/z.fbx#castle}
| Chapter one | | / \ | | }
| | | / \ | | @baroque{castle,
| John built houses in baroque style. | | / \ | | url = {https://y.io/z.fbx#castle}
| | | |_____| | | }
| #john@baroque | +-----│-----+ | @baroque{john}
| | │ |
| | ├─ name: castle |
| | └─ tag: house baroque |
+----------------------------------------+ |
[3D mesh ] |
| O ├─ name: john |
| /|\ | |
| / \ | |
+--------+ |
| the #john@baroque-bib associates both text John and objectname
| john, with tag baroque
Another example:
2023-09-21 13:05:30 +02:00
http://y.io/z.fbx | Derived XRWG (expressed as BibTex)
2023-09-18 11:03:18 +02:00
----------------------------------------------------------------------------+--------------------------------------
|
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | @house{castle,
| Chapter one | | / \ | | url = {https://y.io/z.fbx#castle}
| | | / \ | | }
| John built houses in baroque style. | | / \ | | @baroque{castle,
| | | |_____| | | url = {https://y.io/z.fbx#castle}
| #john@baroque | +-----│-----+ | }
| @baroque{john} | │ | @baroque{john}
| | ├─ name: castle |
| | └─ tag: house baroque |
+----------------------------------------+ | @house{baroque}
[3D mesh ] | @todo{baroque}
+-[remotestorage.io / localstorage]------+ | O + name: john |
| #baroque@todo@house | | /|\ | |
| ... | | / \ | |
+----------------------------------------+ +--------+ |
| both #john@baroque-bib and BibTex @baroque{john} result in the
| same XRWG, however on top of that 2 tages (house and todo) are now
| associated with text/objectname/tag 'baroque'.
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 18]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
As seen above, the XRWG can expand bibs
(https://github.com/coderofsalvation/hashtagbibs) (and the whole
scene) to BibTeX.
This allows hasslefree authoring and copy-paste of associations *for
and by humans*, but also makes these URLs possible:
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
+==================+======================================+
| URL example | Result |
+==================+======================================+
| https://my.com/ | draws lines between mesh john, 3D |
| foo.gltf#baroque | mesh castle, text John built(..) |
+------------------+--------------------------------------+
| https://my.com/ | draws lines between mesh john, and |
| foo.gltf#john | the text John built (..) |
+------------------+--------------------------------------+
| https://my.com/ | draws lines between mesh castle, and |
| foo.gltf#house | other objects with tag house or todo |
+------------------+--------------------------------------+
2023-09-18 11:03:18 +02:00
2023-09-21 13:05:30 +02:00
Table 10
2023-09-18 11:03:18 +02:00
| hashtagbibs (https://github.com/coderofsalvation/hashtagbibs)
| potentially allow the enduser to annotate text/objects by
| *speaking/typing/scanning associations*, which the XR Browser
| saves to remotestorage (or localStorage per toplevel URL). As
| well as, referencing BibTags per URI later on: https://y.io/
| z.fbx#@baroque@todo e.g.
The XRWG allows XR Browsers to show/hide relationships in realtime at
various levels:
* wordmatch *inside* src text
* wordmatch *inside* href text
* wordmatch object-names
* wordmatch object-tagnames
2023-09-09 11:30:03 +02:00
2023-09-21 13:05:30 +02:00
Spatial wires can be rendered between words/objects etc.
2023-09-18 11:03:18 +02:00
Some pointers for good UX (but not necessary to be XR Fragment
compatible):
9. The XR Browser needs to adjust tag-scope based on the endusers
2023-09-06 15:13:36 +02:00
needs/focus (infinite tagging only makes sense when environment
is scaled down significantly)
2023-09-18 11:03:18 +02:00
10. The XR Browser should always allow the human to view/edit the
2023-09-06 15:13:36 +02:00
metadata, by clicking 'toggle metadata' on the 'back'
(contextmenu e.g.) of any XR text, anywhere anytime.
2023-09-18 11:03:18 +02:00
11. respect multi-line BiBTeX metadata in text because of the core
principle (#core-principle)
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 19]
Internet-Draft XR Fragments September 2023
2023-09-18 11:03:18 +02:00
12. Default font (unless specified otherwise) is a modern monospace
font, for maximized tabular expressiveness (see the core
principle (#core-principle)).
13. anti-pattern: hardcoupling an XR Browser with a mandatory
*markup/scripting-language* which departs from onubtrusive plain
text (HTML/VRML/Javascript) (see the core principle (#core-
principle))
14. anti-pattern: limiting human introspection, by abandoning plain
text as first tag citizen.
| The simplicity of appending metadata (and leveling the metadata-
2023-09-09 11:30:03 +02:00
| playfield between humans and machines) is also demonstrated by
| visual-meta (https://visual-meta.info) in greater detail.
2023-09-05 19:14:10 +02:00
2023-09-18 11:03:18 +02:00
Fictional chat:
<John> Hey what about this: https://my.com/station.gltf#pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> I'm checking it right now
<Sarah> I don't see everything..where's our text from yesterday?
<John> Ah wait, that's tagged with tag 'draft' (and hidden)..hold on, try this:
<John> https://my.com/station.gltf#.draft&pos=0,0,1&rot=90,2,0&t=500,1000
<Sarah> how about we link the draft to the upcoming YELLO-event?
<John> ok I'm adding #draft@YELLO
<Sarah> Yesterday I also came up with other usefull assocations between other texts in the scene:
#event#YELLO
#2025@YELLO
<John> thanks, added.
<Sarah> Btw. I stumbled upon this spatial book which references station.gltf in some chapters:
<Sarah> https://thecommunity.org/forum/foo/mytrainstory.txt
<John> interesting, I'm importing mytrainstory.txt into station.gltf
<John> ah yes, chapter three points to trainterminal_2A in the scene, cool
2023-09-21 13:05:30 +02:00
13.1. Default Data URI mimetype
2023-09-04 21:20:59 +02:00
The src-values work as expected (respecting mime-types), however:
2023-09-02 21:44:57 +02:00
The XR Fragment specification bumps the traditional default browser-
mimetype
text/plain;charset=US-ASCII
2023-09-08 17:01:14 +02:00
to a hashtagbib(tex)-friendly one:
2023-09-04 21:20:59 +02:00
2023-09-06 15:13:36 +02:00
text/plain;charset=utf-8;bib=^@
2023-09-04 21:20:59 +02:00
2023-09-08 17:01:14 +02:00
This indicates that:
2023-09-04 21:20:59 +02:00
2023-09-08 17:01:14 +02:00
* utf-8 is supported by default
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 20]
Internet-Draft XR Fragments September 2023
2023-09-18 11:03:18 +02:00
* lines beginning with @ will not be rendered verbatim by default
(read more (https://github.com/coderofsalvation/
hashtagbibs#hashtagbib-mimetypes))
* the XRWG should expand bibs to BibTex occurring in text
(#contactjohn@todo@important e.g.)
By doing so, the XR Browser (applications-layer) can interpret
microformats (visual-meta (https://visual-meta.info) to connect text
further with its environment ( setup links between textual/spatial
objects automatically e.g.).
2023-09-02 21:44:57 +02:00
2023-09-08 17:01:14 +02:00
| for more info on this mimetype see bibs
| (https://github.com/coderofsalvation/hashtagbibs)
2023-09-02 21:44:57 +02:00
2023-09-08 17:01:14 +02:00
Advantages:
2023-09-18 11:03:18 +02:00
* auto-expanding of hashtagbibs
(https://github.com/coderofsalvation/hashtagbibs) associations
2023-09-08 17:01:14 +02:00
* out-of-the-box (de)multiplex human text and metadata in one go
(see the core principle (#core-principle))
2023-09-09 11:30:03 +02:00
* no network-overhead for metadata (see the core principle (#core-
principle))
* ensuring high FPS: HTML/RDF historically is too 'requesty'/'parsy'
for game studios
2023-09-08 17:01:14 +02:00
* rich send/receive/copy-paste everywhere by default, metadata being
retained (see the core principle (#core-principle))
* netto result: less webservices, therefore less servers, and
overall better FPS in XR
2023-09-04 21:20:59 +02:00
| This significantly expands expressiveness and portability of human
2023-09-06 15:13:36 +02:00
| tagged text, by *postponing machine-concerns to the end of the
| human text* in contrast to literal interweaving of content and
2023-09-04 21:20:59 +02:00
| markupsymbols (or extra network requests, webservices e.g.).
For all other purposes, regular mimetypes can be used (but are not
required by the spec).
2023-09-21 13:05:30 +02:00
13.2. URL and Data URI
2023-09-02 21:44:57 +02:00
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 21]
Internet-Draft XR Fragments September 2023
2023-09-02 21:44:57 +02:00
+--------------------------------------------------------------+ +------------------------+
| | | author.com/article.txt |
| index.gltf | +------------------------+
| │ | | |
| ├── ◻ article_canvas | | Hello friends. |
| │ └ src: ://author.com/article.txt | | |
2023-09-18 11:03:18 +02:00
| │ | | @book{greatgatsby |
2023-09-04 21:20:59 +02:00
| └── ◻ note_canvas | | ... |
2023-09-18 11:03:18 +02:00
| └ src:`data:welcome human\n@book{sunday...}` | | } |
2023-09-04 21:20:59 +02:00
| | +------------------------+
2023-09-02 21:44:57 +02:00
| |
+--------------------------------------------------------------+
2023-09-04 21:20:59 +02:00
The enduser will only see welcome human and Hello friends rendered
2023-09-18 11:03:18 +02:00
verbatim (see mimetype). The beauty is that text in Data URI
2023-09-09 11:30:03 +02:00
automatically promotes rich copy-paste (retaining metadata). In both
cases, the text gets rendered immediately (onto a plane geometry,
hence the name '_canvas'). The XR Fragment-compatible browser can
let the enduser access visual-meta(data)-fields after interacting
with the object (contextmenu e.g.).
2023-09-02 21:44:57 +02:00
2023-09-07 14:06:50 +02:00
| additional tagging using bibs
2023-09-08 17:01:14 +02:00
| (https://github.com/coderofsalvation/hashtagbibs): to tag spatial
2023-09-07 14:06:50 +02:00
| object note_canvas with 'todo', the enduser can type or speak
2023-09-18 11:03:18 +02:00
| #note_canvas@todo
2023-09-09 11:30:03 +02:00
2023-09-21 13:05:30 +02:00
13.3. XR Text example parser
2023-09-08 17:01:14 +02:00
2023-09-18 11:03:18 +02:00
To prime the XRWG with text from plain text src-values, here's an
example XR Text (de)multiplexer in javascript (which supports inline
bibs & bibtex):
2023-09-05 19:14:10 +02:00
2023-09-08 17:01:14 +02:00
xrtext = {
expandBibs: (text) => {
let bibs = { regex: /(#[a-zA-Z0-9_+@\-]+(#)?)/g, tags: {}}
text.replace( bibs.regex , (m,k,v) => {
tok = m.substr(1).split("@")
match = tok.shift()
if( tok.length ) tok.map( (t) => bibs.tags[t] = `@${t}{${match},\n}` )
else if( match.substr(-1) == '#' )
bibs.tags[match] = `@{${match.replace(/#/,'')}}`
else bibs.tags[match] = `@${match}{${match},\n}`
})
return text.replace( bibs.regex, '') + Object.values(bibs.tags).join('\n')
},
decode: (str) => {
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 22]
Internet-Draft XR Fragments September 2023
2023-09-08 17:01:14 +02:00
// bibtex: ↓@ ↓<tag|tag{phrase,|{ruler}> ↓property ↓end
let pat = [ /@/, /^\S+[,{}]/, /},/, /}/ ]
let tags = [], text='', i=0, prop=''
let lines = xrtext.expandBibs(str).replace(/\r?\n/g,'\n').split(/\n/)
for( let i = 0; i < lines.length && !String(lines[i]).match( /^@/ ); i++ )
text += lines[i]+'\n'
bibtex = lines.join('\n').substr( text.length )
bibtex.split( pat[0] ).map( (t) => {
try{
let v = {}
if( !(t = t.trim()) ) return
if( tag = t.match( pat[1] ) ) tag = tag[0]
if( tag.match( /^{.*}$/ ) ) return tags.push({ruler:tag})
2023-09-18 11:03:18 +02:00
if( tag.match( /}$/ ) ) return tags.push({k: tag.replace(/}$/,''), v: {}})
2023-09-08 17:01:14 +02:00
t = t.substr( tag.length )
t.split( pat[2] )
.map( kv => {
if( !(kv = kv.trim()) || kv == "}" ) return
v[ kv.match(/\s?(\S+)\s?=/)[1] ] = kv.substr( kv.indexOf("{")+1 )
})
tags.push( { k:tag, v } )
}catch(e){ console.error(e) }
})
return {text, tags}
2023-09-04 21:20:59 +02:00
},
2023-09-06 15:13:36 +02:00
encode: (text,tags) => {
let str = text+"\n"
for( let i in tags ){
let item = tags[i]
if( item.ruler ){
str += `@${item.ruler}\n`
continue;
}
str += `@${item.k}\n`
for( let j in item.v ) str += ` ${j} = {${item.v[j]}}\n`
str += `}\n`
}
return str
2023-09-04 21:20:59 +02:00
}
}
2023-09-07 14:06:50 +02:00
The above functions (de)multiplexe text/metadata, expands bibs,
2023-09-18 11:03:18 +02:00
(de)serialize bibtex and vice versa
2023-09-04 21:20:59 +02:00
| above can be used as a startingpoint for LLVM's to translate/
2023-09-06 15:13:36 +02:00
| steelman to a more formal form/language.
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 23]
Internet-Draft XR Fragments September 2023
2023-09-06 15:13:36 +02:00
str = `
hello world
2023-09-08 17:01:14 +02:00
here are some hashtagbibs followed by bibtex:
#world
#hello@greeting
#another-section#
2023-09-06 15:13:36 +02:00
@{some-section}
@flap{
asdf = {23423}
}`
var {tags,text} = xrtext.decode(str) // demultiplex text & bibtex
tags.find( (t) => t.k == 'flap{' ).v.asdf = 1 // edit tag
tags.push({ k:'bar{', v:{abc:123} }) // add tag
console.log( xrtext.encode(text,tags) ) // multiplex text & bibtex back together
2023-09-09 11:30:03 +02:00
This expands to the following (hidden by default) BibTex appendix:
2023-09-08 17:01:14 +02:00
hello world
here are some hashtagbibs followed by bibtex:
2023-09-07 14:06:50 +02:00
@{some-section}
@flap{
asdf = {1}
2023-09-06 15:13:36 +02:00
}
2023-09-08 17:01:14 +02:00
@world{world,
}
@greeting{hello,
}
@{another-section}
2023-09-07 14:06:50 +02:00
@bar{
abc = {123}
2023-09-06 15:13:36 +02:00
}
2023-09-02 21:44:57 +02:00
2023-09-08 17:01:14 +02:00
| when an XR browser updates the human text, a quick scan for
| nonmatching tags (@book{nonmatchingbook e.g.) should be performed
| and prompt the enduser for deleting them.
2023-09-21 13:05:30 +02:00
14. Security Considerations
2023-09-07 14:06:50 +02:00
Since XR Text contains metadata too, the user should be able to set
up tagging-rules, so the copy-paste feature can :
* filter out sensitive data when copy/pasting (XR text with
2023-09-18 11:03:18 +02:00
tag:secret e.g.)
2023-09-05 19:14:10 +02:00
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 24]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
15. FAQ
2023-09-06 15:13:36 +02:00
2023-09-21 13:05:30 +02:00
*Q:* Why is everything HTTP GET-based, what about POST/PUT/DELETE
HATEOS
*A:* Because it's out of scope: XR Fragment specifies a read-only way
to surf XR documents. These things belong in the application layer
(for example, an XR Hypermedia browser can decide to support
POST/PUT/DELETE requests for embedded HTML thru src values)
2023-09-01 14:20:02 +02:00
2023-09-21 13:05:30 +02:00
*Q:* Why isn't there support for scripting *A:* This is out of scope,
and up to the XR hypermedia browser. Javascript seems to been able
to turn webpages from hypermedia documents into its opposite
(hyperscripted nonhypermedia documents). In order to prevent this
backward-movement (hypermedia tends to liberate people from finnicky
scripting) XR Fragments should never unhyperify itself by
hardcoupling to a particular markup or scripting language. XR
Macro's (https://xrfragment.org/doc/RFC_XR_Macros.html) are an
example of something which is probably smarter and safer for
hypermedia browsers to implement, instead of going full-in with a
turing-complete scripting language (and suffer the security
consequences later).
16. IANA Considerations
2023-09-08 17:01:14 +02:00
2023-09-18 11:03:18 +02:00
This document has no IANA actions.
2023-09-08 17:01:14 +02:00
2023-09-21 13:05:30 +02:00
17. Acknowledgments
2023-09-08 17:01:14 +02:00
2023-09-09 11:30:03 +02:00
* NLNET (https://nlnet.nl)
* Future of Text (https://futureoftext.org)
2023-09-07 15:53:32 +02:00
* visual-meta.info (https://visual-meta.info)
2023-09-21 13:05:30 +02:00
18. Appendix: Definitions
+=================+===============================================+
| definition | explanation |
+=================+===============================================+
| human | a sentient being who thinks fuzzy, absorbs, |
| | and shares thought (by plain text, not |
| | markuplanguage) |
+-----------------+-----------------------------------------------+
| scene | a (local/remote) 3D scene or 3D file |
| | (index.gltf e.g.) |
+-----------------+-----------------------------------------------+
| 3D object | an object inside a scene characterized by |
| | vertex-, face- and customproperty data. |
+-----------------+-----------------------------------------------+
| metadata | custom properties of text, 3D Scene or |
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 25]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
| | Object(nodes), relevant to machines and a |
| | human minority (academics/developers) |
+-----------------+-----------------------------------------------+
| XR fragment | URI Fragment with spatial hints like |
| | #pos=0,0,0&t=1,100 e.g. |
+-----------------+-----------------------------------------------+
| the XRWG | wordgraph (collapses 3D scene to tags) |
+-----------------+-----------------------------------------------+
| the hashbus | hashtags map to camera/scene-projections |
+-----------------+-----------------------------------------------+
| spacetime | positions camera, triggers scene-preset/time |
| hashtags | |
+-----------------+-----------------------------------------------+
| teleportation | repositioning the enduser to a different |
| | position (or 3D scene/file) |
+-----------------+-----------------------------------------------+
| sourceportation | teleporting the enduser to the original XR |
| | Document of an src embedded object. |
+-----------------+-----------------------------------------------+
| placeholder | a 3D object which with src-metadata (which |
| object | will be replaced by the src-data.) |
+-----------------+-----------------------------------------------+
| src | (HTML-piggybacked) metadata of a 3D object |
| | which instances content |
+-----------------+-----------------------------------------------+
| href | (HTML-piggybacked) metadata of a 3D object |
| | which links to content |
+-----------------+-----------------------------------------------+
| query | an URI Fragment-operator which queries |
| | object(s) from a scene like #q=cube |
+-----------------+-----------------------------------------------+
| visual-meta | visual-meta (https://visual.meta.info) data |
| | appended to text/books/papers which is |
| | indirectly visible/editable in XR. |
+-----------------+-----------------------------------------------+
| requestless | metadata which never spawns new requests |
| metadata | (unlike RDF/HTML, which can cause framerate- |
| | dropping, hence not used a lot in games) |
+-----------------+-----------------------------------------------+
| FPS | frames per second in spatial experiences |
| | (games,VR,AR e.g.), should be as high as |
| | possible |
+-----------------+-----------------------------------------------+
| introspective | inward sensemaking ("I feel this belongs to |
| | that") |
+-----------------+-----------------------------------------------+
| extrospective | outward sensemaking ("I'm fairly sure John is |
| | a person who lives in oklahoma") |
2023-09-21 13:30:14 +02:00
van Kammen Expires 24 March 2024 [Page 26]
Internet-Draft XR Fragments September 2023
2023-09-21 13:05:30 +02:00
+-----------------+-----------------------------------------------+
| &#9723; | ascii representation of an 3D object/mesh |
+-----------------+-----------------------------------------------+
| (un)obtrusive | obtrusive: wrapping human text/thought in |
| | XML/HTML/JSON obfuscates human text into a |
| | salad of machine-symbols and words |
+-----------------+-----------------------------------------------+
| BibTeX | simple tagging/citing/referencing standard |
| | for plaintext |
+-----------------+-----------------------------------------------+
| BibTag | a BibTeX tag |
+-----------------+-----------------------------------------------+
| (hashtag)bibs | an easy to speak/type/scan tagging SDL (see |
| | here (https://github.com/coderofsalvation/ |
| | hashtagbibs) which expands to BibTex/JSON/XML |
+-----------------+-----------------------------------------------+
Table 11
2023-09-02 21:44:57 +02:00
2023-09-08 17:01:14 +02:00
2023-09-21 13:05:30 +02:00
van Kammen Expires 24 March 2024 [Page 27]