File Server

Access to a common area (future: also user-specific private areas).

Access must be both from web page (HTTP/S) and from WebDAV.

File Browser is used for web based access from browser.

Apache is used as WebDAV server.

NGINX is used as reverse proxy to provide secure access to both WebDAV clients and browsers.

All users need to be in the users group.

The common share will be accessible by any user in the users group.

Files will be under /home/common for example. The shares will be configured in the /etc/conf.d/shares file:

shares

SHARES="common:3002 other:3003"

where “common” and “other” is the name of the folder under /home and 3002/3003 is the port number (which will be needed for NGINX reverse proxy access via browser).

File Browser is a nice web-based file manager that you can use to access your file server via browser.

I do not like the default installation method because it will install system-wide. I will show you how to install in a more customized way.

first you need to create a new user:

 > useradd -d /data/daemons/filebrowser -m filebrowser -g users

the filebrowser user will have users as it's main group so that any files managed by it can be accessed and managed by users as well.

You will need to create the following folders architecture in your filebrowser home folder:

• bin: where the FileBrowser binary will be located
• data/db: where the FileBrowser databases files will be stored
• data/logs: where the various log files will be created

You need to set the umask for the user to 0002 so that any new files created by it will be writable by the users.

Then, as filebrowser user, get the software package and decompress it. The default install approach is based on a auto executable web link (here) which i do not recommend to use directly. Instead go to here and download the proper package for your architecture. Then:

 > su - filebrowser
 > echo "umask 0002" >> ~/.bashrc
 > source ~/.bashrc
 > mkdir bin data data/logs data/db
 > cd bin
 > tar xvf ../linux-amd64-filebrowser.tar.gz

Now, you will need to start a copy of FileBrowser for each share you want to have, and it must be owned by the user that want file permissions on that share. To achieve this, you will be using a special script called fileserver.sh which i will show you at the end, because it will contain also the WebDAV start stuff in it.

While there are a few WebDAV servers like Dave, they seems to be either unmaintained or overly complicated. Also NGINX can be a WebDAV server, but it seems to be buggy and not supporting LOCK stuff, so i decided to go with Apache web server, which also has a long standing WebDAV implementation.

The idea here is to run a dedicated copy of Apache as user filebrowser and group users so that it can access and manage the shared files. So first you need to emerge apache:

 > emerge apache

WebDAV is enabled by default in Gentoo Apache ebuild.

Running apache manually requires some effort, so, buckle up.

First of all, Apache needs some folders to operate, so you need to create:

• /data/daemons/filebrowser/data/conf: to store the apache config file
• /data/daemons/filebrowser/data/roots: which will map as WebDAV root (you will see why)
• /data/daemons/filebrowser/data/locks: which will be used for WebDAV lock databases
• /data/daemons/filebrowser/data/pids: which will be used to store apache PID files

 > su - filebrowser
 > mkdir /data/daemons/filebrowser/data/conf
 > mkdir /data/daemons/filebrowser/data/root

Then create the Apache config file for each share. You should create this config that will be used by each share /data/daemons/filebrowser/data/conf/apache_global.conf:

apache_global.conf

ServerRoot "/usr/lib64/apache2"
LoadModule actions_module modules/mod_actions.so
LoadModule alias_module modules/mod_alias.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_anon_module modules/mod_authn_anon.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authn_dbm_module modules/mod_authn_dbm.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_dbm_module modules/mod_authz_dbm.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule autoindex_module modules/mod_autoindex.so
<IfDefine CACHE>
LoadModule cache_module modules/mod_cache.so
</IfDefine>
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule dav_lock_module modules/mod_dav_lock.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule expires_module modules/mod_expires.so
LoadModule ext_filter_module modules/mod_ext_filter.so
<IfDefine CACHE>
LoadModule file_cache_module modules/mod_file_cache.so
</IfDefine>
LoadModule filter_module modules/mod_filter.so
LoadModule headers_module modules/mod_headers.so
<IfDefine HTTP2>
LoadModule http2_module modules/mod_http2.so
</IfDefine>
LoadModule include_module modules/mod_include.so
<IfDefine INFO>
LoadModule info_module modules/mod_info.so
</IfDefine>
LoadModule log_config_module modules/mod_log_config.so
TransferLog /deposito/archive/logs/common_transfer_log
LoadModule logio_module modules/mod_logio.so
LoadModule mime_module modules/mod_mime.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule setenvif_module modules/mod_setenvif.so
<IfDefine STATUS>
LoadModule status_module modules/mod_status.so
</IfDefine>
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule unixd_module modules/mod_unixd.so
<IfDefine USERDIR>
LoadModule userdir_module modules/mod_userdir.so
</IfDefine>
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
Include /etc/apache2/modules.d/*.conf

Then you can create one config file for each share. This is the file for the common share /data/daemons/filebrowser/data/conf/common.conf:

common.conf

Include /data/daemons/filebrowser/data/conf/apache_global.conf
 
User filebrowser
Group users
 
DavLockDB "/data/daemons/filebrowser/data/locks/common"
PidFile /data/daemons/filebrowser/data/pids/common.pid
ErrorLog /data/daemons/filebrowser/data/logs/common_error_log
TransferLog /data/daemons/filebrowser/data/logs/common_transfer_log
CustomLog /data/daemons/filebrowser/data/logs/common_access_log common
 
DocumentRoot /data/daemons/filebrowser/data/roots
 
ServerName 127.0.0.1
Listen 127.0.0.1:10001
 
<Directory /data/daemons/filebrowser/data/roots>
    DAV On
    AllowOverride All
    Options -Indexes +FollowSymlinks -ExecCGI -Includes
    Require all granted
</Directory>
 
SetEnv redirect-carefully
 
# vim: ts=4 filetype=apache

Now, the fun part is that you want to protect this behind the NGINX reverse proxy and it seems that WebDAV does not play well with URL redirection and similar funny things. In other words, the base url you will be using on the reverse proxy must match the url in the Apache. You cannot use rewrite directives or Alias stuff.

Since you will be exposing the browser-based access as https://your_server/archive/common and the WebDAV access as https://your_server/webdav/common it means that we need to sym-link your /home/common folder to /data/daemons/filebrowser/data/roots/webdav/common for it to work:

 > su - filebrowser
 > cd data/root
 > mkdir webdav
 > cd webdav
 > ln -s /home/common common

xxxxxx

Now, reverse proxy is simple, but this into /etc/nginx/folders/filebrowser.conf:

filebrowser.conf

  location /archive/common/ {
        client_max_body_size 512M;
 
        proxy_pass http://127.0.0.1:3002;
        proxy_http_version 1.1;
 
        proxy_set_header Connection $http_connection;
        proxy_set_header Connection 'upgrade';
        proxy_cache_bypass $http_upgrade;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

and put this file include inside the usual /etc/nginx*/folders/main.conf, and restart nginx.

filebrowser.sh

source /etc/conf.d/shares
 
BASE_PATH=/deposito/daemons/filebrowser/data
 
for i in $SHARES
do
        SHARE=$(echo $i | cut -d: -f1)
        PORT=$(echo $i | cut -d: -f2)
        OWNER=filebrowser
 
        echo Starting FileBrowser for $OWNER on share $SHARE
        su - $OWNER -c "/deposito/daemons/filebrowser/bin/filebrowser config set --auth.method=noauth -d $BASE_PATH/db/filebrowser_$SHARE.db >/dev/null"
        su - $OWNER -c "/deposito/daemons/filebrowser/bin/filebrowser -r /deposito/$SHARE -p $PORT -b /archive/$SHARE -d $BASE_PATH/db/filebrowser_$SHARE.db -l $BASE_PATH/logs/filebrowser_$SHARE.log 2> $BASE_PATH/logs/filebrowser_${SHARE}_run.log"&
 
        echo Starting WebDAV backend for $OWNER on share $SHARE
        su - $OWNER -c "apache2 -f /data/daemons/filebrowser/data/conf/$SHARE.conf"
done

And the usual autostart stuff:

40-filebrowser.start

#!/bin/bash
/data/daemons/filebrowser/fileserver.sh

Make both files executable.

From users point of view, the common area will be managed by user filebrowser which is designed to run as group users with an umask 550 so that any files uploaded via web browser will be accessible to the normal users.

Of course, each user will need to be part of the users group as well.

You will need a common “archive” folder under /home/common to store some needed stuff.

This folder will need to contain:

• common subfolder, where the common area files will be stored (created in the filebrowser instructions)
• temp/uploads subfolder, required by WebDAV to upload files
• temp/pids subfolder, to store NGINX pids
• temp/tmp subfolder, to store NGINX run files
• logs subfolder, to store NGINX log files
• conf subfolder, where you will store custom NGINX config files for the private areas (and common area too)

Create the folders:

 > mkdir /data/archive
 > mkdir /data/archive/logs
 > mkdir /data/archive/common
 > mkdir /data/archive/temp
 > mkdir /data/archive/temp/pids
 > mkdir /data/archive/temp/tmp
 > mkdir /data/archive/conf
 > chown filebrowser:users -R /data/archive

No need to use third party WebDAV server since NGINX has a pretty solid implementation of it already. Follow the nginx instructions to set NGINX up with WebDAV and PAM auth support.

Now, there is a nasty catch here which stems from using NGINX as WebDAV server… You need to run NGINX as filebrowser user to ensure that the access trough WebDAV will not incur in access permissions errors. Running NGINX as standalone user requires the creation of a specific nginx.conf with some specifics in it.

For consistency, this NGINX config file will be /data/archive/conf/nginx_common.conf:

/data/archive/conf/nginx_common.conf

worker_processes 1;
pid /data/archive/temp/pids/nginx_common.pid;
error_log /data/archive/logs/common_error_log info;
 
events {
        worker_connections 100;
        use epoll;
}
 
http {
        include /etc/nginx/mime.types;
        default_type application/octet-stream;
 
        # These folder MUST be redirected to avoid usage of system wide ones:
        client_body_temp_path  /data/archive/temp/tmp;
        proxy_temp_path  /data/archive/temp/tmp;
        fastcgi_temp_path  /data/archive/temp/tmp;
        uwsgi_temp_path  /data/archive/temp/tmp;
        scgi_temp_path  /data/archive/temp/tmp;
        disable_symlinks off;
 
        keepalive_timeout 75 20;
 
        server {
                server_name 127.0.0.1;
 
                access_log /data/archive/logs/common_access_log;
                location / {
                        root /data/archive/common/;
 
                        dav_methods PUT DELETE MKCOL COPY MOVE;
                        dav_ext_methods PROPFIND OPTIONS;
                        dav_access user:rw group:rw all:r;
 
                        client_max_body_size 0;
                        create_full_put_path on;
                        client_body_temp_path /data/archive/uploads;
                }
                listen 10000;
        }
}

This NGINX server will listen on 127.0.0.1:10000, and you will need to setup a reverse proxy from the main NGINX, by creating the following config file /etc/nginx/folders/webdav.conf:

webdav.conf

location ~ ^/webdav/common {
        rewrite /webdav/common/(.*) /$1 break;
        proxy_pass http://127.0.0.1:10000;
}

and including it into the main NGINX server.

Now, edit the /data/daemons/filebrowser/filebrowser.sh file and add the following line:

nginx -c /deposito/archive/conf/nginx_common.conf -e /deposito/archive/logs/common_error_log

like this:

filebrowser.sh

#!/bin/bash
 
cd /data/daemons/filebrowser &&
nginx -c /data/archive/conf/nginx_common.conf -e /data/archive/logs/common_error_log
./filebrowser -r /data/archive/common -p 3002 -b /archive/common 2>&1 > filebrowser.log

and restart filebrwoser and the main NGINX.

At this point, your common area will be ready and working both on WebDAV and directly via web browser.

To access via browser:

to access via WebDAV clients: