User Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
selfhost:fileserver [2024/01/18 10:56] – willyselfhost:fileserver [2025/03/19 15:09] (current) – willy
Line 1: Line 1:
-====== File Server ======+====== H) File Server ======
  
 +I will not discuss how to share your files on the home network using __legacy__ tools like [[https://it.wikipedia.org/wiki/Network_File_System|NFS]] or [[https://it.wikipedia.org/wiki/Samba_(software)|SAMBA]], there are plenty of tutorials online and, beside, it's kind out of the scope for self-hosting.
  
-Access to a **common area** and user-specific **private** areas.+I will focus on how to provide access via __web browser__ and via __WebDAV__, which is a web-based sharing protocol a bit like NFS or SAMBA, but aimed ad broader //in**ter**net// access, and not //in**tra**net// access.
  
-Access must be both from web page (HTTP/S) and from WebDAV.+The idea is to create share areas where your users will be able to store files. It is possible to extend this idea also to user-specific areas where each user can put private stuff not visible by other users, but this require a little bit extra complexity and might be addressed in the future.
  
-[[sailing:filebrowser]] is used for web based access from browser+You will be using your SSO authentication, there will be no need to create new users anywhere, and it will of course be protected by the Reverse Proxy for external access.
  
-[[sailing:NGINX]] is used as WebDAV server.+In the past i used a more complex solution leveraging more tools. That obsolete solution has been moved, for reference, [[selfhost:obsolete:fileserver-legacy|here]].
  
-===== Permissions and Users =====+===== Overall Architecture and Shares =====
  
-All users need to be in the **users** group.+This solution leverages the use of one tool called AList (installation & configuration instructions [[services:alist|here]]). AList is a pretty neat open source tool which is capable to provide, at the same time, both a browser-based solution and a WebDAV solution to access your files.
  
-The **common** share will be accessible by any user in the **users** group.+AList itself also support SSO integration, opening the way to provide also a public sharing approach, if needed since the SSO should not be enabled at reverse-proxy level.
  
-===== Shares Configuration =====+You can also define as many shared folders as you like, and even connect to remote services from the same UI.
  
-Files will be under **/home/common** for example. The shares will be configured in the **/etc/conf.d/shares** file: +I will assume that your shares are under **/data/shares**, but of course each share can be located anywhere you like. Let's also assume, as an example, that your share is called __/data/shares/common__ and is managed by the user __fileserver__ of the group __users__. The requirement for users and groups will be detailed later on.
-<file txt shares> +
-SHARES="common:3002 other:3003" +
-</file>+
  
-where "common" and "other" is the name of the folder under **/home** and 3002/3003 is the port number (which will be needed for NGINX reverse proxy access via browser).+Each share folder will have the following structure: 
 +  * /data/share/common/ 
 +  * /data/share/other_share/ 
 +  * /data/share/another_share/
  
 +Your AList installation will provide WebDAV and browser access from one single port hwich need to be reverse-proxied.
  
-===== Software Installation for Browser access =====+I choose to assign a dedicated subdomain, **drive.mydomain.com**, as file server and organize the shares like this: 
 +  * **https://drive.mydomain.com**: will show a main login page to access all the shares  
 +  * **https://drive.mydomain.com/common**: direct browser access URL for __common__ 
 +  * **https://drive.mydomain.com/webdav**: WebDAV access URL for all the shares 
 +  * **https://drive.mydomain.com/webdav/common**: WebDAV specific access URL for __common__ 
 +  * **https://drive.mydomain.com/dav**: WebDAV access URL for all the shares 
 +  * **https://drive.mydomain.com/dav/common**: WebDAV specific access URL for __common__
  
-[[https://filebrowser.org/|File Browser]] is a nice web-based file manager that you can use to access your file server via browser.+I think that /webdav is easier to remember than /dav, but AList by default shared WebDAV under /dav, NGINX will be used to map the /webdav path to /dav.
  
-I do not like the default installation method because it will install system-wide. I will show you how to install in a more customized way.+You can add any more folders as separate shares as you like. Due to how WebDAV works, it is mandatory to separate the browser accessible URLs from the WebDAV ones, like i did above.
  
-first you need to create a new user: 
-<code bash> 
- > useradd -d /data/daemons/filebrowser -m filebrowser -g users 
-</code> 
  
-the //filebrowser// user will have **users** as it's main group so that any files managed by it can be accessed and managed by users as well.+=== Permissions and Users ===
  
-You will need to create the following folders architecture in your filebrowser home folder: +(Note: you should run AList as the user **fileserver** and group **users**)
-  * bin: where the FileBrowser binary will be located +
-  * data/db: where the FileBrowser databases files will be stored +
-  * data/logs: where the various log files will be created+
  
-You need to set the //umask// for the user to **0002** so that any new files created by it will be writable by the users.+I assume you have already created the user **fileserver** when installing AList. 
  
-Then, as //filebrowser// user, get the software package and decompress it. The default install approach is based on a auto executable web link ([[https://raw.githubusercontent.com/filebrowser/get/master/get.sh|here]]) which i do not recommend to use directly. Instead go to [[https://github.com/filebrowser/filebrowser/releases/|here]] and download the proper package for your architecture. Then:+You need to set the //umask// for the fileserver user to **0002** so that any new files created by it will be writable by the users:
 <code bash> <code bash>
- > su - filebrowser +mkdir /data/shares 
- > echo "umask 0002" >> ~/.bashrc +mkdir /data/shares/common 
- > source ~/.bashrc +chown fileserver:users /data/shares
- > mkdir bin data data/logs data/db +
- > cd bin +
- > tar xvf ../linux-amd64-filebrowser.tar.gz+
 </code> </code>
  
-Now, you will need to start a copy of FileBrowser for each share you want to have, and it must be owned by the user that want file permissions on that share. 
  
-To achieve this, you will be using a special script called **fileserver.sh** which i will show you at the end, because it will contain also the WebDAV start stuff in it.+===== Fileserver access via Browser =====
  
-===== Software Installation for WebDAV access =====+Nothing extra needs to be done except install AList, and adding the new shares inside it's WEB configuration.
  
  
 +===== Fileserver access via WebDAV =====
  
 +__NOTE:__ using HTTP will cause a 301 redirect to HTTPS, and WebDAV clients will fail. So use HTTPS URL in webdav clients and not HTTP.
  
- +The only chnage you need to make is to add the following location to the NGINX configuration file you created during AList setup: 
- +<code> 
- +        location /webdav/ { 
- +               proxy_pass http://127.0.0.1:5244/dav/; 
- +               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 
- +               proxy_set_header X-Forwarded-Proto $scheme; 
- +               proxy_set_header Host $http_host; 
- +               proxy_set_header X-Real-IP $remote_addr; 
- +               proxy_set_header Range $http_range; 
- +               proxy_set_header If-Range $http_if_range; 
- +               proxy_redirect off; 
- +               client_max_body_size 20000m;
- +
-since it will be hidden behind the reverse proxy, you can disable FileBrowser internal authentication. +
- +
-You need to setup filebrowser to access your common archive, so create the folder **/data/archive/common** and own it to filebrowser:users: +
-<code bash> +
- > mkdir /data/archive +
- > mkdir /data/archive/common +
- > chown -R filebrowser:users /data/archive/common +
-</code> +
- +
-Now, a startup script: +
-<file bash filebrowser.sh> +
-#!/bin/bash +
-cd /data/daemons/filebrowser && +
-./filebrowser -r /depoisito/archive/common -p 3002 -b /archive/common/ 2>&1 > filebrowser.log +
-</file> +
- +
-And the usual autostart stuff: +
-<file bash 40-filebrowser.start> +
-#!/bin/bash +
-start-stop-daemon -b -m -p /var/run/filebrowser.pid -n filebrowser -u filebrowser /data/daemons/filebrowser/filebrowser.sh +
-</file> +
- +
-Make both files executable. +
- +
-Now, reverse proxy is simple, but this into **/etc/nginx/folders/filebrowser.conf**: +
-<file txt filebrowser.conf> +
- +
-  location /archive/common/ { +
-        client_max_body_size 512M; +
- +
-        proxy_pass http://127.0.0.1:3002; +
-        proxy_http_version 1.1; +
- +
-        proxy_set_header Connection $http_connection; +
-        proxy_set_header Connection 'upgrade'; +
-        proxy_cache_bypass $http_upgrade; +
-        proxy_set_header Host $host; +
-        proxy_set_header X-Real-IP $remote_addr; +
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; +
-        proxy_set_header X-Forwarded-Proto $scheme; +
-    } +
-</file> +
- +
-and put this file include inside the usual **/etc/nginx*/folders/main.conf**, and restart nginx. +
- +
- +
- +
-<file bash filebrowser.sh> +
-source /etc/conf.d/shares +
- +
-BASE_PATH=/deposito/daemons/filebrowser/data +
- +
-for i in $SHARES +
-do +
-        SHARE=$(echo $i | cut -d: -f1) +
-        PORT=$(echo $i | cut -d: -f2) +
-        OWNER=filebrowser +
- +
-        echo Starting FileBrowser for $OWNER on share $SHARE +
-        su - $OWNER -c "/deposito/daemons/filebrowser/bin/filebrowser config set --auth.method=noauth -d $BASE_PATH/db/filebrowser_$SHARE.db >/dev/null" +
-        su - $OWNER -c "/deposito/daemons/filebrowser/bin/filebrowser -r /deposito/$SHARE -p $PORT -b /archive/$SHARE -d $BASE_PATH/db/filebrowser_$SHARE.db -l $BASE_PATH/logs/filebrowser_$SHARE.log 2> $BASE_PATH/logs/filebrowser_${SHARE}_run.log"& +
- +
-        echo Starting WebDAV backend for $OWNER on share $SHARE +
-done +
-</file> +
- +
- +
- +
- +
-===== Background ===== +
- +
-From users point of view, the common area will be managed by user **filebrowser** which is designed to run as group **users** with an //umask// 550 so that any files uploaded via web browser will be accessible to the normal users. +
- +
-Of course, each user will need to be part of the **users** group as well. +
- +
-You will need a common "archive" folder under **/home/common** to store some needed stuff. +
- +
-This folder will need to contain: +
-  * **common** subfolder, where the common area files will be stored (created in the [[sailing:filebrowser]] instructions) +
-  * **temp/uploads** subfolder, required by WebDAV to upload files  +
-  * **temp/pids** subfolder, to store NGINX pids +
-  * **temp/tmp** subfolder, to store NGINX run files +
-  * **logs** subfolder, to store NGINX log files +
-  * **conf** subfolder, where you will store custom NGINX config files for the private areas (and common area too) +
- +
-Create the folders: +
-<code bash> +
- > mkdir /data/archive +
- > mkdir /data/archive/logs +
- > mkdir /data/archive/common +
- > mkdir /data/archive/temp +
- > mkdir /data/archive/temp/pids +
- > mkdir /data/archive/temp/tmp +
- > mkdir /data/archive/conf +
- > chown filebrowser:users -R /data/archive +
-</code> +
- +
-===== NGINX WebDAV approach ===== +
- +
-No need to use third party WebDAV server since NGINX has a pretty solid implementation of it already. Follow the [[sailing:nginx]] instructions to set NGINX up with WebDAV and PAM auth support. +
- +
-Now, there is a nasty catch here which stems from using NGINX as WebDAV server... You need to run NGINX as //filebrowser// user to ensure that the access trough WebDAV will not incur in access permissions errors. Running NGINX as standalone user requires the creation of a specific nginx.conf with some specifics in it. +
- +
-For consistency, this NGINX config file will be **/data/archive/conf/nginx_common.conf**: +
-<file txt /data/archive/conf/nginx_common.conf> +
-worker_processes 1; +
-pid /data/archive/temp/pids/nginx_common.pid; +
-error_log /data/archive/logs/common_error_log info; +
- +
-events { +
-        worker_connections 100; +
-        use epoll; +
-} +
- +
-http { +
-        include /etc/nginx/mime.types; +
-        default_type application/octet-stream; +
- +
-        # These folder MUST be redirected to avoid usage of system wide ones: +
-        client_body_temp_path  /data/archive/temp/tmp; +
-        proxy_temp_path  /data/archive/temp/tmp; +
-        fastcgi_temp_path  /data/archive/temp/tmp; +
-        uwsgi_temp_path  /data/archive/temp/tmp; +
-        scgi_temp_path  /data/archive/temp/tmp; +
-        disable_symlinks off; +
- +
-        keepalive_timeout 75 20; +
- +
-        server { +
-                server_name 127.0.0.1; +
- +
-                access_log /data/archive/logs/common_access_log; +
-                location / { +
-                        root /data/archive/common/; +
- +
-                        dav_methods PUT DELETE MKCOL COPY MOVE; +
-                        dav_ext_methods PROPFIND OPTIONS; +
-                        dav_access user:rw group:rw all:r; +
- +
-                        client_max_body_size 0; +
-                        create_full_put_path on; +
-                        client_body_temp_path /data/archive/uploads; +
-                } +
-                listen 10000;+
         }         }
-} 
-</file> 
- 
-This NGINX server will listen on 127.0.0.1:10000, and you will need to setup a reverse proxy from the main NGINX, by creating the following config file **/etc/nginx/folders/webdav.conf**: 
-<file txt webdav.conf> 
-location ~ ^/webdav/common { 
-        rewrite /webdav/common/(.*) /$1 break; 
-        proxy_pass http://127.0.0.1:10000; 
-} 
-</file> 
-and including it into the main NGINX server.  
- 
-Now, edit the **/data/daemons/filebrowser/filebrowser.sh** file and add the following line: 
-<code> 
-nginx -c /deposito/archive/conf/nginx_common.conf -e /deposito/archive/logs/common_error_log 
 </code> </code>
  
-like this: +which will remap /webdav to /dav 
-<file bash filebrowser.sh> +
-#!/bin/bash +
- +
-cd /data/daemons/filebrowser && +
-nginx -c /data/archive/conf/nginx_common.conf -e /data/archive/logs/common_error_log +
-./filebrowser -r /data/archive/common -p 3002 -b /archive/common 2>&1 > filebrowser.log +
-</file> +
- +
-and restart filebrwoser and the main NGINX. +
- +
-At this point, your common area will be ready and working both on WebDAV and directly via web browser. +
- +
-To access via browser: +
- +
-to access via WebDAV clients: +
- +
- +
- +