coderofsalvation/xrfragment-haxe
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

update documentation

Browse source
This commit is contained in:
Leon van Kammen 2023-09-11 15:20:43 +02:00
parent f699edc65b
commit 3e3724c950
1 changed files with 79 additions and 75 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

154
doc/RFC_XR_Fragments.md
View file

@ -109,7 +109,7 @@ Their lowest common denominator is: (co)authoring using plain text.<br>
XR Fragments allows us to enrich/connect existing dataformats, by recursive use of existing technologies:<br> XR Fragments allows us to enrich/connect existing dataformats, by recursive use of existing technologies:<br>
1. addressibility and navigation of 3D scenes/objects: [URI Fragments](https://en.wikipedia.org/wiki/URI_fragment) + src/href spatial metadata 1. addressibility and navigation of 3D scenes/objects: [URI Fragments](https://en.wikipedia.org/wiki/URI_fragment) + src/href spatial metadata
1. hasslefree tagging across text and spatial objects using [bibs](https://github.com/coderofsalvation/tagbibs) / [BibTags](https://en.wikipedia.org/wiki/BibTeX) appendices (see [visual-meta](https://visual-meta.info) e.g.) 1. Interlinking text/3D by deriving a Word Graph (XRWG) from the scene (and augmenting text with [bibs](https://github.com/coderofsalvation/tagbibs) / [BibTags](https://en.wikipedia.org/wiki/BibTeX) appendices (see [visual-meta](https://visual-meta.info) e.g.)
> NOTE: The chapters in this document are ordered from highlevel to lowlevel (technical) as much as possible > NOTE: The chapters in this document are ordered from highlevel to lowlevel (technical) as much as possible
@ -133,23 +133,23 @@ See appendix below in case certain terms are not clear.
# List of URI Fragments # List of URI Fragments
| fragment | type | example | info | | fragment | type | example | info |
|--------------|----------|-------------------|-------------------------------------------------------------------| |--------------|----------|-------------------|---------------------------------------------------------------------|
| `#pos` | vector3 | `#pos=0.5,0,0` | positions camera to xyz-coord 0.5,0,0 | | `#pos` | vector3 | `#pos=0.5,0,0` | positions camera to xyz-coord 0.5,0,0 |
| `#rot` | vector3 | `#rot=0,90,0` | rotates camera to xyz-coord 0.5,0,0 | | `#rot` | vector3 | `#rot=0,90,0` | rotates camera to xyz-coord 0.5,0,0 |
| `#t` | vector2 | `#t=500,1000` | sets animation-loop range between frame 500 and 1000 | | `#t` | vector2 | `#t=500,1000` | sets animation-loop range between frame 500 and 1000 |
| `#......` | string | `#.cubes` `#cube` | object(s) of interest (fragment to object name or class mapping) | | `#......` | string | `#.cubes` `#cube` | object(s) of interest (fragment-to-object-or-classname) |
> xyz coordinates are similar to ones found in SVG Media Fragments > xyz coordinates are similar to ones found in SVG Media Fragments
# List of metadata for 3D nodes # List of metadata for 3D nodes
| key | type | example (JSON) | info | | key | type | example (JSON) | info |
|--------------|----------|--------------------|--------------------------------------------------------| |--------------|----------|------------------------|--------------------------------------------------------|
| `name` | string | `"name": "cube"` | available in all 3D fileformats & scenes | | `name` | string | `"name": "cube"` | available in all 3D fileformats & scenes |
| `class` | string | `"class": "cubes"` | available through custom property in 3D fileformats | | `class` | string | `"class": "cubes geo"` | available through custom property in 3D fileformats |
| `href` | string | `"href": "b.gltf"` | available through custom property in 3D fileformats | | `href` | string | `"href": "b.gltf"` | available through custom property in 3D fileformats |
| `src` | string | `"src": "#q=cube"` | available through custom property in 3D fileformats | | `src` | string | `"src": "#q=cube"` | available through custom property in 3D fileformats |
Popular compatible 3D fileformats: `.gltf`, `.obj`, `.fbx`, `.usdz`, `.json` (THREE.js), `.dae` and so on. Popular compatible 3D fileformats: `.gltf`, `.obj`, `.fbx`, `.usdz`, `.json` (THREE.js), `.dae` and so on.
@ -210,12 +210,14 @@ Resizing will be happen accordingly to its placeholder object `aquariumcube`, se
Include, exclude, hide/shows objects using space-separated strings: Include, exclude, hide/shows objects using space-separated strings:
* `#q=cube` | example | outcome |
* `#q=cube -ball_inside_cube` |----------------------------------|-----------------------------------------------------------------------------|
* `#q=* -sky` | `#q=cube` | show only object named `cube` (if any) |
* `#q=-.language .english` | `#q=cube -ball_inside_cube` | show only object named `cube` but not child named `ball_inside_cube` |
* `#q=cube&rot=0,90,0` | `#q=* -sky` | show everything except object named `sky` |
* `#q=price:>2 price:<5` | `#q=-.language .english` | hide everything with class `language`, then show all class `english` objects|
| `#q=cube&rot=0,90,0` | show only object `cube` and rotate the view |
| `#q=price:>2 price:<5` | show only objects with custom property `price` with value between 2 and 5 |
It's simple but powerful syntax which allows <b>css</b>-like class/id-selectors with a searchengine prompt-style feeling: It's simple but powerful syntax which allows <b>css</b>-like class/id-selectors with a searchengine prompt-style feeling:
@ -288,37 +290,32 @@ sub-delims = "," / "="
# Text in XR (tagging,linking to spatial objects) # Text in XR (tagging,linking to spatial objects)
We still think and speak in simple text, not in HTML or RDF.<br> How does XR Fragments interlink text with objects?
The most advanced human will probably not shout `<h1>FIRE!</h1>` in case of emergency.<br>
Given the new dawn of (non-keyboard) XR interfaces, keeping text as is (not obscuring with markup) is preferred.<br>
Ideally metadata must come **with** text, but not **obfuscate** the text, or **spawning another request** to fetch it.<br>
``` > The XR Fragments does this by extracting a **Word Graph** (the **XRWG**) from the current scene, facilitated by Bib(s)Tex.
Spectrum of speak/scan/write/listen/keyboard-friendly 'tagging' notations:
(just # and @) (string only) (obuscated text) (type-aware text) The (`class`)names end up in the Word Graph (XRWG), but what about text (**inside** an article e.g.)? <br>
Ideally metadata must come **with** that text, but not **obfuscate** the text, or **spawning another request** to fetch it.<br>
<---- Bibs ---------- BibTeX ---------- XML / HTML --------- JSON / YAML / RDF --------> This is done by detecting Bib(s)Tex, without introducing a new language or fileformat<br>
``` > Why Bib(s)Tex? Because its seems to be the lowest common denominator for a human-curate-able XRWG (extendable by speech/scanner/writing/typing e.g, see [further motivation here](https://github.com/coderofsalvation/hashtagbibs#bibs--bibtex-combo-lowest-common-denominator-for-linking-data))
Hence: Hence:
1. XR Fragments promotes the importance of hasslefree plain text and string-based patternmatching 1. XR Fragments promotes (de)serializing a scene to the XRWG
2. XR Fragments allows <b id="tagging-text">hasslefree spatial tagging</b>, by detecting metadata **at the end of content** of text (see default mimetype & Data URI) 2. XR Fragments primes the XRWG, by collecting words from the `class` and name-property of 3D objects.
3. XR Fragments allows <b id="tagging-objects">hasslefree spatial tagging</b>, by treating 3D object name/class-pairs as BibTeX tags. 3. XR Fragments primes the XRWG, by collecting words from **optional** metadata **at the end of content** of text (see default mimetype & Data URI)
4. XR Fragments promotes hasslefree <a href="#textual-tag">textual tagging</a>, <a href="#spatial-tag">spatial tagging</a>, and <a href="#supra-tagging">supra tagging</a>, by mapping 3D/text object (class)names to (tag)text-occurences. 4. [Bib's](https://github.com/coderofsalvation/hashtagbibs) and BibTex are first class citizens for priming the XRWG with words (from XR text)
5. XR Fragments supports **requestless metadata** when found in plain text data (of `src` metadata), for adding/describing relationships spatially. 5. Like Bibs, XR Fragments generalizes the BibTex author/title-semantics (`author{title}`) into **this** points to **that** (`this{that}`)
6. **requestless metadata** should be string-only and typeless, and should be easy to edit/add by humans (using text). 6. The XRWG should be recalculated when textvalues (in `src`) change
7. Therefore, BibTeX and [Bib's](https://github.com/coderofsalvation/hashtagbibs) are first class citizens for XR text (HTML/RDF/JSON is great, but fits better in the application-layer) 7. HTML/RDF/JSON is still great, but is beyond the XRWG-scope (they fit better in the application-layer)
8. Opening tags for metadata (`#`, `@`, `{`, or `<`) should always start at the beginning of the line. 8. Applications don't have to be able to access the XRWG programmatically, as they can easily generate one themselves by traversing the scene-nodes.
9. The XR Fragment focuses on fast and easy-to-generate end-user controllable word graphs (instead of complex implementations that try to defeat word ambiguity)
This allows recursive connections between text itself, as well as 3D objects and vice versa.<br> Example:
Here's an example by expanding polyglot metadata to **BibTeX** associations:
``` ```
http://y.io/z.fbx | Derived BibTex / 'wires' & tags http://y.io/z.fbx | Derived XRWG (printed as BibTex)
----------------------------------------------------------------------------+-------------------------------------- ----------------------------------------------------------------------------+--------------------------------------
| @house{castle, | @house{castle,
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | url = {https://y.io/z.fbx#castle} +-[src: data:.....]----------------------+ +-[3D mesh]-+ | url = {https://y.io/z.fbx#castle}
@ -332,58 +329,65 @@ Here's an example by expanding polyglot metadata to **BibTeX** associations:
| | └─ class: house baroque | | | └─ class: house baroque |
+----------------------------------------+ | +----------------------------------------+ |
[3D mesh ] | [3D mesh ] |
+-[remotestorage.io / localstorage]------+ | O ├─ name: john | | O ├─ name: john |
| #contactjohn@todo@house | | /|\ | | | /|\ | |
| ... | | / \ | | | / \ | |
+----------------------------------------+ +--------+ | +--------+ |
``` ```
A (somewhat extreme) example of using polyglot (bib)tags: > the `#john@baroque`-bib associates both text `John` and objectname `john`, with class `baroque`
``` ```
http://y.io/z.fbx | Derived BibTex / 'wires' & tags
Another example:
```
http://y.io/z.fbx | Derived XRWG (printed as BibTex)
----------------------------------------------------------------------------+-------------------------------------- ----------------------------------------------------------------------------+--------------------------------------
| @baroque{john} |
+-[src: data:.....]----------------------+ +-[3D mesh]-+ | @house{castle, +-[src: data:.....]----------------------+ +-[3D mesh]-+ | @house{castle,
| Chapter one | | / \ | | url = {https://y.io/z.fbx#castle} | Chapter one | | / \ | | url = {https://y.io/z.fbx#castle}
| | | / \ | | } | | | / \ | | }
| John built houses in baroque style. | | / \ | | @baroque{castle, | John built houses in baroque style. | | / \ | | @baroque{castle,
| | | |_____| | | url = {https://y.io/z.fbx#castle} | | | |_____| | | url = {https://y.io/z.fbx#castle}
| #john@baroque | +-----│-----+ | } | #john@baroque | +-----│-----+ | }
| @house{baroque, info = {classic}, } | │ | @house{baroque, | @baroque{john} | │ | @baroque{john}
| { "tag":"baroque", "match":"john"} | ├─ name: castle | info = {classic} | | ├─ name: castle |
| <tag name="baroque" match="john"/> | └─ class: house baroque | } | | └─ class: house baroque |
+----------------------------------------+ | @house{contactjohn} +----------------------------------------+ | @house{baroque}
[3D mesh ] | @todo{contactjohn} [3D mesh ] | @todo{baroque}
+-[remotestorage.io / localstorage]------+ | O + name: john | +-[remotestorage.io / localstorage]------+ | O + name: john |
| #contactjohn@todo@house | | /|\ | | | #baroque@todo@house | | /|\ | |
| ... | | / \ | | | ... | | / \ | |
+----------------------------------------+ +--------+ | +----------------------------------------+ +--------+ |
``` ```
As seen above, we can extract tags/associations between text & 3D objects, by converting all scene metadata to (in this case) BibTeX, by expanding [hashtagbibs](https://github.com/coderofsalvation/hashtagbibs) and interpreting its polyglot tag-notation.<br> > both `#john@baroque`-bib and BibTex `@baroque{john}` result in the same XRWG, however on top of that 2 classes (`house` and `todo`) are now associated with text/objectname/class 'baroque'.
One huge advantage of polyglot tags is authoring and copy-paste **by humans**, which will be discussed later in this spec.<br>
> [hashtagbibs](https://github.com/coderofsalvation/hashtagbibs) also allows the enduser to annotate text/objects by **speaking/typing/scanning associations**, which the XR Browser saves to remotestorage (or localStorage per toplevel URL). As well as, referencing BibTags per URI later on: `https://y.io/z.fbx#@baroque@todo` e.g. As seen above, the XRWG can expand [bibs](https://github.com/coderofsalvation/hashtagbibs) (and the whole scene) to BibTeX.<br>
This allows hasslefree authoring and copy-paste of associations **for and by humans**, but also makes these URLs possible:
The Evaluated BiBTeX allows XR Browsers to show/hide relationships in realtime at various levels: | URL example | Result |
|---------------------------------------|---------------------------------------------------------------------------|
| `https://my.com/foo.gltf#.baroque` | highlights mesh `john`, 3D mesh `castle`, text `John built(..)` |
| `https://my.com/foo.gltf#john` | highlights mesh `john`, and the text `John built (..)` |
| `https://my.com/foo.gltf#house` | highlights mesh `castle`, and other objects with class `house` or `todo` |
| scope | tag-matching algo | > [hashtagbibs](https://github.com/coderofsalvation/hashtagbibs) potentially allow the enduser to annotate text/objects by **speaking/typing/scanning associations**, which the XR Browser saves to remotestorage (or localStorage per toplevel URL). As well as, referencing BibTags per URI later on: `https://y.io/z.fbx#@baroque@todo` e.g.
|---------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <b id="textual-tagging">textual</b> | text containing 'baroque' is now automatically tagged with 'house' (incl. plaintext `src` child nodes) |
| <b id="spatial-tagging">spatial</b> | spatial object(s) with name `baroque` or `"class":"house"` are now automatically tagged with 'house' (incl. child nodes) |
| <b id="supra-tagging">supra</b> | text- or spatial-object(s) (non-descendant nodes) elsewhere, (class)named 'baroque' or 'house', are automatically tagged with 'house' (current node to root nodes) |
| <b id="omni-tagging">omni</b> | text- or spatial-object(s) (non-descendant nodes) elsewhere, (class)named 'baroque' or 'house', are automatically tagged with 'house' (root node to all nodes) |
| <b id="infinite-tagging">infinite</b> | text- or spatial-object(s) (non-descendant nodes) elsewhere, (class)named 'baroque' or 'house', are automatically tagged with 'house' (root node to all nodes ) |
This allows the enduser to adjust different levels of associations (see [the core principle](#core-principle)): spatial wires can be rendered, words/objects can be highlighted/scaled etc.<br> The XRWG allows XR Browsers to show/hide relationships in realtime at various levels:
> NOTE: infinite matches both 'baroque' and 'house'-occurences in text, as well as spatial objects with `"class":"house"` or name "baroque". This multiplexing of id/category is deliberate, in order to support [the core principle](#core-principle). * wordmatch **inside** `src` text
* wordmatch **inside** `href` text
* wordmatch object-names
* wordmatch object-classnames
Spatial wires can be rendered, words/objects can be highlighted/scaled etc.<br>
Some pointers for good UX (but not necessary to be XR Fragment compatible):
9. The XR Browser needs to adjust tag-scope based on the endusers needs/focus (infinite tagging only makes sense when environment is scaled down significantly) 9. The XR Browser needs to adjust tag-scope based on the endusers needs/focus (infinite tagging only makes sense when environment is scaled down significantly)
10. The XR Browser should always allow the human to view/edit the metadata, by clicking 'toggle metadata' on the 'back' (contextmenu e.g.) of any XR text, anywhere anytime. 10. The XR Browser should always allow the human to view/edit the metadata, by clicking 'toggle metadata' on the 'back' (contextmenu e.g.) of any XR text, anywhere anytime.
11. When moving/copying/pasting metadata, always prefer converting to string-only microformats (BibTex/Bibs) 12. respect multi-line BiBTeX metadata in text because of [the core principle](#core-principle)
12. respect multi-line metadata because of [the core principle](#core-principle)
13. Default font (unless specified otherwise) is a modern monospace font, for maximized tabular expressiveness (see [the core principle](#core-principle)). 13. Default font (unless specified otherwise) is a modern monospace font, for maximized tabular expressiveness (see [the core principle](#core-principle)).
14. anti-pattern: hardcoupling an XR Browser with a mandatory **markup/scripting-language** which departs from onubtrusive plain text (HTML/VRML/Javascript) (see [the core principle](#core-principle)) 14. anti-pattern: hardcoupling an XR Browser with a mandatory **markup/scripting-language** which departs from onubtrusive plain text (HTML/VRML/Javascript) (see [the core principle](#core-principle))
15. anti-pattern: limiting human introspection, by abandoning plain text as first class citizen. 15. anti-pattern: limiting human introspection, by abandoning plain text as first class citizen.
@ -405,8 +409,8 @@ to a hashtagbib(tex)-friendly one:
This indicates that: This indicates that:
* utf-8 is supported by default * utf-8 is supported by default
* lines beginning with `@` will not be rendered verbatim by default (=Bibs/BibTex) * lines beginning with `@` will not be rendered verbatim by default ([read more](https://github.com/coderofsalvation/hashtagbibs#hashtagbib-mimetypes))
* bibs occurring in text (`#contactjohn@todo@important` e.g.) should expand to BibTeX * the XRWG should expand bibs to BibTex occurring in text (`#contactjohn@todo@important` e.g.)
By doing so, the XR Browser (applications-layer) can interpret microformats ([visual-meta](https://visual-meta.info) By doing so, the XR Browser (applications-layer) can interpret microformats ([visual-meta](https://visual-meta.info)
to connect text further with its environment ( setup links between textual/spatial objects automatically e.g.). to connect text further with its environment ( setup links between textual/spatial objects automatically e.g.).
@ -435,20 +439,20 @@ For all other purposes, regular mimetypes can be used (but are not required by t
| │ | | | | │ | | |
| ├── ◻ article_canvas | | Hello friends. | | ├── ◻ article_canvas | | Hello friends. |
| │ └ src: ://author.com/article.txt | | | | │ └ src: ://author.com/article.txt | | |
| │ | | { | | │ | | @book{greatgatsby |
| └── ◻ note_canvas | | ... | | └── ◻ note_canvas | | ... |
| └ src:`data:welcome human\n{...` | | } | | └ src:`data:welcome human\n@book{sunday...}` | | } |
| | +------------------------+ | | +------------------------+
| | | |
+--------------------------------------------------------------+ +--------------------------------------------------------------+
``` ```
The enduser will only see `welcome human` and `Hello friends` rendered spatially (see mimetype). The enduser will only see `welcome human` and `Hello friends` rendered verbatim (see mimetype).
The beauty is that text in Data URI automatically promotes rich copy-paste (retaining metadata). The beauty is that text in Data URI automatically promotes rich copy-paste (retaining metadata).
In both cases, the text gets rendered immediately (onto a plane geometry, hence the name '_canvas'). In both cases, the text gets rendered immediately (onto a plane geometry, hence the name '_canvas').
The XR Fragment-compatible browser can let the enduser access visual-meta(data)-fields after interacting with the object (contextmenu e.g.). The XR Fragment-compatible browser can let the enduser access visual-meta(data)-fields after interacting with the object (contextmenu e.g.).
> additional tagging using [bibs](https://github.com/coderofsalvation/hashtagbibs): to tag spatial object `note_canvas` with 'todo', the enduser can type or speak `@note_canvas@todo` > additional tagging using [bibs](https://github.com/coderofsalvation/hashtagbibs): to tag spatial object `note_canvas` with 'todo', the enduser can type or speak `#note_canvas@todo`
## XR Text example parser ## XR Text example parser