doc: how to bring stuff into XRSH scene

coderofsalvation commented

2025-01-15 17:15:54 +01:00

Owner

Some feedback from jonathan:

(..) I don't see an immediately obvious way to change the scene details?
I'd very much like to do a custom background and add in some other things to interact with the scene such as a
datasette instance

Perhaps we can include a chapter on this in the 'Getting Started'-section of https://xrsh.isvery.ninja

So far there's various ways of getting things into XRSH scene:

permanent: just editing index.html in this xrsh repo (regular AFRAME app, easiest for including background)
sessionbased: launch cm index.html to live-edit the scene
via livecoding: https://xrsh.isvery.ninja/#XR%20Livecoding
pasting/dragging zip into scene: https://xrsh.isvery.ninja/#XRSH%20zip%20packages
importing during boot via an url fragment (with wget): https://xrsh.isvery.ninja/#Unixy%20URLs
via the javascript interface: https://xrsh.isvery.ninja/#Unixy%20Javascript%20Interface

Maybe some code snippets too..like in the manual: https://forgejo.isvery.ninja/xrsh/xrsh-buildroot/src/branch/main/buildroot-v86/board/v86/rootfs_overlay/root/manual.md#calling-terminal-from-javascript

Some feedback from jonathan: > (..) I don't see an immediately obvious way to change the scene details? > I'd very much like to do a custom background and add in some other things to interact with the scene such as a > datasette instance Perhaps we can include a chapter on this in the 'Getting Started'-section of https://xrsh.isvery.ninja So far there's various ways of getting things into XRSH scene: * permanent: just editing `index.html` in this xrsh repo (regular AFRAME app, easiest for including background) * sessionbased: launch `cm index.html` to live-edit the scene * via livecoding: https://xrsh.isvery.ninja/#XR%20Livecoding * pasting/dragging zip into scene: https://xrsh.isvery.ninja/#XRSH%20zip%20packages * importing during boot via an url fragment (with `wget`): https://xrsh.isvery.ninja/#Unixy%20URLs * via the javascript interface: https://xrsh.isvery.ninja/#Unixy%20Javascript%20Interface Maybe some code snippets too..like in the manual: https://forgejo.isvery.ninja/xrsh/xrsh-buildroot/src/branch/main/buildroot-v86/board/v86/rootfs_overlay/root/manual.md#calling-terminal-from-javascript

coderofsalvation commented

2025-01-15 17:18:45 +01:00

Author

Owner

Based on quick conversation with Fabien:

(..) I "bet" (so it's just an hypothesis to validate or invalidate) that most people would try to use XRSH with... well whatever they have laying around. It "should" be keyboard but in reality it's probably :

* just hand tracking
* hand tracking and/or controllers
* hand tracker + BT kbd

roughly sorted by popularity. So... in that sense at least have a path to experiment or "at worst" documentation that does make it clear it's kbd only, would be good.

Now on the actual technical solution, I'd argue against super-hands because I'm not sure it's maintained anymore so ideally "just" AFrame would be better. AFAIR one only has to add controller and hands for them to work and swapping works (but not mixed, e.g. not one hand tracked + 1 controller tracked) so it usually means finding a button equivalent to pinching (and other gestures).

If it's event based, can be quite painless.

Based on quick conversation with Fabien: ``` (..) I "bet" (so it's just an hypothesis to validate or invalidate) that most people would try to use XRSH with... well whatever they have laying around. It "should" be keyboard but in reality it's probably : * just hand tracking * hand tracking and/or controllers * hand tracker + BT kbd roughly sorted by popularity. So... in that sense at least have a path to experiment or "at worst" documentation that does make it clear it's kbd only, would be good. Now on the actual technical solution, I'd argue against super-hands because I'm not sure it's maintained anymore so ideally "just" AFrame would be better. AFAIR one only has to add controller and hands for them to work and swapping works (but not mixed, e.g. not one hand tracked + 1 controller tracked) so it usually means finding a button equivalent to pinching (and other gestures). If it's event based, can be quite painless. ```

jonititan commented

2025-01-15 18:03:59 +01:00

Drag and drop off images works on PC. Drag and drop of files doesn't stop work on quest 3.
upload command didn't work on either

Haven't managed to build my own image with qemu or docker yet so I can have the files in there from start.

But then I couldn't get aframe to use an equirectangular image as sky? Whole scene is just black. Same file works on glitch.
A-Sky

Is there a way to access the error messages?
js console.log just returns "undefined"

Expected output

Drag and drop off images works on PC. Drag and drop of files doesn't stop work on quest 3. upload command didn't work on either Haven't managed to build my own image with qemu or docker yet so I can have the files in there from start. But then I couldn't get aframe to use an equirectangular image as sky? Whole scene is just black. Same file works on glitch. [A-Sky](https://aframe.io/docs/1.6.0/primitives/a-sky.html) Is there a way to access the error messages? js console.log just returns "undefined" [Expected output](https://special-abyssinian-deerstalker.glitch.me/)

PXL_20250115_160418912.jpg

3.3 MiB

coderofsalvation commented

2025-01-15 18:05:26 +01:00

Author

Owner

suggestions from AFRAME's discord regarding mixed grabbables: https://glitch.com/edit/#!/aframe-xr-starterkit?path=index.html%3A201%3A49

I'm not really convinced.
The obvious behaviour to me seems..just add grabbable and be done with it.

suggestions from AFRAME's discord regarding mixed grabbables: https://glitch.com/edit/#!/aframe-xr-starterkit?path=index.html%3A201%3A49 I'm not really convinced. The obvious behaviour to me seems..just add `grabbable` and be done with it.

coderofsalvation commented

2025-01-15 18:14:12 +01:00

Author

Owner

But then I couldn't get aframe to use an equirectangular image as sky? Whole scene is just black.

You're trying to load a file from your harddrive (src="/mnt/....) while you actually want to load it from your terminal filesystem (src="file://xrsh/mnt/....." see manual

Is there a way to access the error messages?

tail -f /var/log/messages /dev/browser/console see http://localhost:8080/#Polyglot%20debugging%20%26%20logging

If you press ctrl+a+1 you'll switch to the logging window in the terminal

js console.log just returns "undefined"

its because console.log does not return anything (try js 'return 123')

> But then I couldn't get aframe to use an equirectangular image as sky? Whole scene is just black. You're trying to load a file from your harddrive (`src="/mnt/....`) while you actually want to load it from your terminal filesystem (`src="file://xrsh/mnt/....."` see [manual](https://forgejo.isvery.ninja/xrsh/xrsh-buildroot/src/branch/main/buildroot-v86/board/v86/rootfs_overlay/root/manual.md#webrequests-to-the-filesystem) > Is there a way to access the error messages? `tail -f /var/log/messages /dev/browser/console` see http://localhost:8080/#Polyglot%20debugging%20%26%20logging If you press ctrl+a+1 you'll switch to the logging window in the terminal > js console.log just returns "undefined" its because `console.log` does not return anything (try `js 'return 123'`)

jonititan commented

2025-01-15 18:27:58 +01:00

Here's the error I got when I tried to use the upload command
I did it twice. Before I went into VR and after.

Here's the error I got when I tried to use the upload command I did it twice. Before I went into VR and after.

1a84b689fe41678133f4c14476df4dc9.png

271 KiB

jonititan commented

2025-01-15 18:52:38 +01:00

its because console.log does not return anything (try js 'return 123')
Ah I tried that and it works. I was just following your docs. Sorry I misunderstood. It's the 2nd column I was supposed to use.
I mainly use python and rust. Not a javascript person.

I got that file link working thanks to your comments but it still goes black and apparently its because the WebGLRenderer in three doesn't like the file. Maybe it was corrupted during upload.

<a-scene>
        <a-assets>
          <img id="sky" src="file://xrsh/mnt/clipboard/360_0175_Stitch_YHC.jpg" />
        </a-assets>
        <a-sky src="#sky"></a-sky>
      </a-scene>

/dev/browser: warn Three.WebGLRenderer: Texture marked for update but image is incomplete

> its because `console.log` does not return anything (try `js 'return 123'`) Ah I tried that and it works. I was just following your docs. Sorry I misunderstood. It's the 2nd column I was supposed to use. I mainly use python and rust. Not a javascript person. [](https://forgejo.isvery.ninja/xrsh/xrsh-buildroot/src/branch/main/buildroot-v86/board/v86/rootfs_overlay/root/manual.md) I got that file link working thanks to your comments but it still goes black and apparently its because the WebGLRenderer in three doesn't like the file. Maybe it was corrupted during upload. ``` <a-scene> <a-assets> <img id="sky" src="file://xrsh/mnt/clipboard/360_0175_Stitch_YHC.jpg" /> </a-assets> <a-sky src="#sky"></a-sky> </a-scene> ``` > /dev/browser: warn Three.WebGLRenderer: Texture marked for update but image is incomplete

jonititan commented

2025-01-16 13:32:56 +01:00

Yeah unfortunately even with the new version it has the image incomplete issue.
Same code works fine on glitch

Yeah unfortunately even with the new version it has the image incomplete issue. Same code works fine on glitch

PXL_20250116_111933658.jpg

2.9 MiB

coderofsalvation commented

2025-01-16 20:48:37 +01:00

Author

Owner

thanks for reporting this, I'll have to investigate this (I'm a bit busy with ironing out the bugs in the latest release)

coderofsalvation commented

2025-01-17 18:44:54 +01:00

Author

Owner

UPDATE: I've spent this afternoon debugging the issue.
What works in the latest version:

the upload cmd

However, the webrequest TO the filesystem (via <a-sky src="....">) was blocked by the browser-sandbox.
So I'm now in the process of refactoring things to make this possible:

<a-sky src="/mnt/clipboard/equirect.png"/>

That way the browser-sandbox won't get in the way.
However, I've noticed that your 35MB equirect is partially sent from the webworker..so there's still some shenanigans there.
For the time being it's better to just include them locally via a webserver.
So:

$ unzip xrsh.com index.html
$ ls 
xrsh.com equirect.png index.html
$ ./xrsh.com -D  .

and then in index.html add your AFRAME stuff as usual

you can also just fork the repository and edit the index.html there ofcourse, and serve it in a webdir e.g.

btw. don't use <a-assets> because that only works at page-load (with xrsh the AFRAME scene is loaded later on, so Im not sure whether that works at all). Just include urls inline (<a-sky src='/mnt/clipboard/....'> in the future..I should write that somewhere in the docs prolly.

UPDATE: I've spent this afternoon debugging the issue. What works in the latest version: * the upload cmd However, the webrequest TO the filesystem (via `<a-sky src="....">`) was blocked by the browser-sandbox. So I'm now in the process of refactoring things to make this possible: ``` <a-sky src="/mnt/clipboard/equirect.png"/> ``` That way the browser-sandbox won't get in the way. However, I've noticed that your 35MB equirect is partially sent from the webworker..so there's still some shenanigans there. For the time being it's better to just include them locally via a webserver. So: ``` $ unzip xrsh.com index.html $ ls xrsh.com equirect.png index.html $ ./xrsh.com -D . ``` and then in index.html add your AFRAME stuff as usual > you can also just fork the repository and edit the index.html there ofcourse, and serve it in a webdir e.g. btw. don't use `<a-assets>` because that only works at page-load (with xrsh the AFRAME scene is loaded later on, so Im not sure whether that works at all). Just include urls inline (`<a-sky src='/mnt/clipboard/....'>` in the future..I should write that somewhere in the docs prolly.

Rows
Columns

doc: how to bring stuff into XRSH scene #4