diff --git a/manyfold/README.md b/manyfold/README.md index 0ddbffb..f250aaa 100644 --- a/manyfold/README.md +++ b/manyfold/README.md @@ -17,6 +17,7 @@ $ docker load < $(nix-build nix/docker.nix) |-----------------------|--------------|------------------------| | `APPNAME` | `manyfold` | manyfold instance name | | `THEME` | `default` | bootstrap theme | +| `RUNTESTS` | `0` | set to `1` to run XRForge related [/test](test) scripts | | `NO_OVERLAYFS` | `` | disable the filesystem overlay mechanism | | `NO_DEFAULTDB` | `` | disable the default db (activates manyfold installer) | | `NO_DELETEBIGFILES` | `` | disable deleting big files which are older than 5 days and bigger than ($currentyear-2020) MB's | @@ -28,6 +29,10 @@ $ docker load < $(nix-build nix/docker.nix) | `FEDERATE_DRIVE_USER` | `` | specify HTTP AUTH credentials (`user` e.g.) for restricted sharing | | `FEDERATE_DRIVE_PW` | `` | specify HTTP AUTH credentials (`pass` e.g.) for restricted sharing | | `FEDERATE_DRIVE_CACHE`| `1m0s` | specify interval to re-check all models/directories | +| `FEDERATE_DRIVE_KEY` | `` | specify path to TLS PEM private key file (`-v ./key.pem:/key.pem -e FEDERATE_DRIVE_KEY=/key.pem` dockerflag e.g.) | +| `FEDERATE_DRIVE_CERT` | `` | specify path to TLS PEM public key certificate/CA/intermediate file (`-v ./cert.pem:/cert.pem -e FEDERATE_DRIVE_KEY=/cert.pem` dockerflag e.g.) | + +> NOTE: if you have nix installed, you can easily try out environment-flags by running: `docker load < $(nix-build nix/docker.nix) && manyfold/cli/manyfold run -e RUNTESTS=1` e.g. # Default database / admin login @@ -44,88 +49,46 @@ When found, it uses the files in there instead (`/manyfold/usr/src/app/public/40 # Federated drives -> Thanks to [rclone](https://rclone.org) network-drives automatically show up as manyfold libraries. +Besides ActivitPub, XRForge allows federating manyfold libraries too, allowing manyfold libraries to scale horizontally across instances: + +``` + + ┌────────────────────────┐ ┌────────────────────────┐ + │ │ │ │ + │ server instance A │ │ server instance B │ + │ │ │ │ + │ ┌──────────────────┐ │ rclone │ ┌─────────────────┐ │ + │ │ library │ │ │ │ library │ │ + │ │ ┼───┼──────────────┼─┤ │ │ + │ │ │ │ http-drive │ │ │ │ + │ │ │ │ │ │ │ │ + │ └──────────────────┘ │ │ └─────────────────┘ │ + │ │ │ │ + └────────────────────────┘ └────────────────────────┘ + + READ / WRITE READ-ONLY + +``` + +It does this by automatically mapping [rclone](https://rclone.org) network-drives as manyfold libraries. ![](https://i.imgur.com/4VMF3CQ.png) -To enable rclone to mount **readonly** network drives, the container must be run with FUSE-device support. +To enable rclone to mount **readonly** network drives (=remotes), the container must be run with FUSE-device support. The quickest way is: -* add `-v ./manyfold/root/.config:/root/.config --cap-add SYS_ADMIN --security-opt apparmor:unconfined --device /dev/fuse` to the docker cmd -* add network drives by running `docker exec -it rclone config` in a running container (see example below) -* profit! +1. create directory `./manyfold/root/.config` outside of the container +2. add `-v ./manyfold/root/.config:/root/.config --cap-add SYS_ADMIN --security-opt apparmor:unconfined --device /dev/fuse` to the docker cmd +3. now federate XRForge libraries by running `docker exec -it xrforge rclone config create myhttp http url=https://xrforgeinstanceB.com user=myuser pass=$(rclone obscure mypassword)` in a running container +4. profit! -Your drives will/should get automagically mounted **readonly** and added to the database automagically (by [manyfold.sh](cli/manyfold.sh) `rclone_automount`-cmd) during container boot. +**Default behaviour**: your drives will/should get automagically mounted **readonly** and added as a library automagically (by [manyfold.sh](cli/manyfold.sh) `rclone_automount`-cmd) during container boot. -> NOTE: by default all rclone remotes automagically show up as separate manyfold libraries, however use `RCLONE_REMOTE` this to specify a [combined](https://rclone.org/combine/) or [union](https://rclone.org/union/) rclone remote. - -* TIP1: use **alphanumeric** names for rclone remotes (manyfold libraries choke on dot- or other special-characters) -* TIP2: share the URL+port of the automatic RCLONE webserver (`FEDERATE_DRIVE_PATH`-envvar), to share experiences **readonly** +* TIP2: use env-var `RCLONE_REMOTE` to mount only one specific remote (in case of a [combined](https://rclone.org/combine/) or [union](https://rclone.org/union/) rclone remote e.g.). +* TIP2: use **alphanumeric** names for rclone remotes (manyfold libraries choke on dot- or other special-characters) By default environment-flag `FEDERATE_DRIVE_PATH` will share path `/mnt/models` as an open web directory. -This means it can be added as remote by other instances. -See the environment-flags for more options. - -
-

Example connect to other XRForge instance

-
-

-$ rclone config                                                                                                                                                                                                                                                           
-Current remotes:                                                                                                                                                                                                                                                                          
-                                                                                                                                                                                                                                                                                          
-Name                 Type                                                                                                                                                                                                                                                                 
-====                 ====                                                                                                                                                                                                                                                                 
-                                                                                                                                                                                                                                                                                          
-e) Edit existing remote                                                                                                                                                                                                                                                                   
-n) New remote                                                                                                                                                                                                                                                                             
-d) Delete remote                                                                                                                                                                                                                                                                          
-r) Rename remote                                                      
-c) Copy remote                                                        
-s) Set configuration password                                         
-q) Quit config                                                        
-e/n/d/r/c/s/q> n                                                      
-
-Enter name for new remote.                                            
-name> xrforge_instanceC                                                            
-
-Option Storage.                                                       
-Type of storage to configure.                                         
-Choose a number from below, or type in your own value.
-
-...
-22 / HTTP
-...
-
-Storage> 22
-
-Option url.
-URL of HTTP host to connect to.
-E.g. "https://example.com", or "https://user:pass@example.com" to use a username and password.
-Enter a value.
-url> http://url-to-another-xrforge-instance.com
-
-Option no_escape.
-Do not escape URL metacharacters in path names.
-Enter a boolean value (true or false). Press Enter for the default (false).
-no_escape> 
-
-Edit advanced config?
-y) Yes
-n) No (default)
-y/n> n
-                                   
-Configuration complete.      
-Options:             
-- type: http                      
-- url: http://localhost:8791                                          
-Keep this "test" remote?                                              
-y) Yes this is OK (default)        
-e) Edit this remote      
-d) Delete this remote    
-y/e/d> y
-
- -
+Make sure that the URL (and credentials if configure) of step 3 are setup properly, so it matches your reverse proxy/ or SSL configuration (via `FEDERATE_DRIVE_CERT` and `FEDERATE_DRIVE_KEY` flags) # Unixy event hooks diff --git a/manyfold/cli/manyfold.sh b/manyfold/cli/manyfold.sh index 09abbd4..e9107b5 100755 --- a/manyfold/cli/manyfold.sh +++ b/manyfold/cli/manyfold.sh @@ -156,6 +156,7 @@ rename_app(){ } start_syslog(){ + touch /var/log/messages syslogd -n & # start syslogd echocolor started syslog | logger tail -f /var/log/messages & diff --git a/manyfold/root/hook.d/boot/httpserver.sh b/manyfold/root/hook.d/boot/httpserver.sh index 30d11f5..21c9b20 100755 --- a/manyfold/root/hook.d/boot/httpserver.sh +++ b/manyfold/root/hook.d/boot/httpserver.sh @@ -9,7 +9,12 @@ test -n "$FEDERATE_DRIVE_USER" && test -m "$FEDERATE_DRIVE_PW" && { AUTH="--user $FEDERATE_DRIVE_USER --pass $FEDERATE_DRIVE_PW" } +test -n "$FEDERATE_DRIVE_CERT" && test -m "$FEDERATE_DRIVE_KEY" && { + SSL="--cert $FEDERATE_DRIVE_CERT --key $FEDERATE_DRIVE_KEY" +} + + set -x rclone serve http \ --poll-interval $FEDERATE_DRIVE_CACHE \ - --addr 0.0.0.0:$FEDERATE_DRIVE_PORT ${AUTH} $FEDERATE_DRIVE_PATH &> /var/log/rclone.log & + --addr 0.0.0.0:$FEDERATE_DRIVE_PORT ${AUTH} ${SSL} $FEDERATE_DRIVE_PATH &> /var/log/rclone.log &