coderofsalvation/xrforge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

milestone 4h: remove x day old files over X MB (temporary sharingservice for big files)

Browse source 
added 'cleanup_big_files' function to manyfold.sh.

Demo:

```
$ docker run xrforge
...
````

create a 10MB dummy_file.txt (and rewind date some years):

```
$ docker exec xrforge 'dd if=/dev/zero of=/mnt/models/dummy_file.txt bs=1M count=10'
10+0 records in
10+0 records out
10485760 bytes (10.0MB) copied, 0.005670 seconds, 1.7GB/s

$ touch -d "2021-01-01" /mnt/models/dummy_file.txt
$
```

Now when setting maxmb's to `5` and maxdays to `5`, we can see in the logs
that it gets deleted:

```
[/bin/infinite cleanup_big_files /mnt/models 5 5] Fri Aug  1 14:19:08 UTC 2025 executing
deleting: /mnt/models/dummy_file.txt
```

This behaviour is default, and can be disabled by docker environment-flag `NO_DISABLEBIGFILES`
This commit is contained in:
Leon van Kammen 2025-08-01 16:20:48 +02:00
parent 23c77dd67a
commit 3b6afc90ec
1 changed files with 40 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

41
manyfold/cli/manyfold.sh
View file

@ -38,7 +38,22 @@ overlayfs(){
test -d /manyfold || return 0; # nothing to override
echocolor "[$APPNAME]" "applying filesystem overlay"
cd /manyfold
rsync -ravuzi * /.
rsync -rvzi * /.
}
# cron-like function using sleep (./manifold.sh infinite 3600 zip -r /backup.zip /)
infinite(){
trap 'echocolor "/bin/infinite $*: process ended..infinite does not care.."; sleep 2s' INT
interval=$1
shift
loop(){
echocolor "[/bin/infinite $*]" "$(date) started"
while sleep ${interval}; do
echocolor "[/bin/infinite $*]" "$(date) executing"
"$@"
done;
}
loop "$@"
}
db(){
@ -111,6 +126,28 @@ rename_app(){
sed -i 's|powered_by_html:.*|powered_by_html: Powered by <a href="https://forgejo.isvery.ninja/coderofsalvation/xrforge">XR Forge</a>, <a href="https://manifold.app" target="_blank">Manyfold</a>, <a href="https://xrfragment.org">XR Fragments</a> and <a href="https://nixos.org" target="_blank">NIX</a>|g' /usr/src/app/config/locales/*.yml
}
cleanup_big_files(){
local DIR=$1
local MAX_DAYS=$2
local MAX_MB=$3
test -n "$MAX_DAYS" && test -n "$MAX_MB" && {
find "$DIR" -type f -mtime +"$MAX_DAYS" -size +$(( $MAX_MB * 1024 ))k | while read file; do
echo "deleting: $file"
rm "$file"
done
}
}
delete_big_files(){
#./manyfold infinite 86400 cleanup_big_files /mnt/models 5 5 & # 5 days for files over 5MB (5 in 2025, 6 in 2026 e.g. ?)
# check every day to delete files older than 5 days and >5mb (5mb in 2025, 6mb in 2026 e.g.)
test -z "$NO_DELETEBIGFILES" && {
intervalSec=86400 # = 1 day
maxMB=$(( $(date +%Y) - 2020 ))
$0 infinite 5 cleanup_big_files /mnt/models $intervalSec $maxMB &
}
}
# The new entrypoint of the docker
boot(){
echocolor "[$APPNAME]" "booting..."
@ -121,6 +158,8 @@ boot(){
set_modelpath
rclone_mount
set_upload_path
delete_big_files
exec "$@" # exec prevents error 's6-overlay-suexec: fatal: can only run as pid 1'
}