User Tools

This is an old revision of the document!

File Server

I will not discuss how to share your files on the home network using legacy tools like NFS or SAMBA, there are plenty of tutorials online and, beside, it's kind out of the scope for self-hosting.

I will focus on how to provide access via web browser and via WebDAV, which is a web-based sharing protocol a bit like NFS or SAMBA, but aimed ad broader internet access, and not intranet access.

The idea is to create share areas where your users will be able to store files. It is possible to extend this idea also to user-specific areas where each user can put private stuff not visible by other users, but this require a little bit extra complexity and might be addressed in the future.

You will be using your SSO authentication, there will be no need to create new users anywhere, and it will of course be protected by the Reverse Proxy for external access.

A future upgrade might prefer the use of sFtpGo instead of FileBrowser + Apache.

Overall Architecture and Shares

This solution leverages two tools:

  • FileBrowser, to provide browser access to your shares
  • Apache strictly as WebDAV server

The NGINX reverse proxy will integrate both with your preferred SSO authentication and add the HTTPS layer to ensure all access is properly encrypted.

I will assume, for the sake of example, that you want to share one folder called common. This folder is phisically located under /home/common on the server and it's owned by user:users. This share will be assigned a specific port, 3002, for FileBrowser access.

folder format here TBD

I choose to assign a dedicated subdomain, drive.mydomain.com, as file server and organize the shares like this:

You can add any more folders as separate shares as you like.

Permissions and Users

Each share will be accessible by different users, so this needs to be planned a bit. For user-specific shares, not much needs to be done except run FileBrowser for the specific share as the specific user. This is left as an exercise for you.

For common shares instead, it'ìs important to create one common user, which i will call fileserver user to run the associated services, and create the /home/common folder.

You need to assign that folder to the users group and the fileserver user: 

useradd -d /home/common -m fileserver -g users
chown fileserver:users  /home/common

You need to set the umask for the fileserver user to 0002 so that any new files created by it will be writable by the users. Also, create the .db folder, where the FileBrowser database will need to be located, and the .apache folder, there specific Apache configuration need to be located: 

su - fileserver
echo "umask 0002" >> ~/.bashrc
source ~/.bashrc
mkdir .db
mkdir .apache

All other users which need to access the common shares must be in the users group.

Fileserver access via Browser

I am currently using [services:filebrowser|FileBrowser]] because it's lightweight, don't get in the way, is flexible and simple to use. Check the linked page for the generic installation instructions, and here are the specific details for this case. I assume FileBrowser has been installed on your system already, following my guide.

You will need to run one instance of FileBrowser for each share, thus you will to allocate one specific port for each share. I will describe how to run it for the common share, so FileBrowser will run as the fileserver user that you created above.

So, create the specific /etc/conf.d/fileserver.common:

filebrowser.common
BASE_URL="/common"
DATABASE="/home/common/.db/filebrowser_common.db"
DESCRIPTION="Common web archive"
FOLDER="/home/common"
GROUP="users"
PORT=3002
USER="filebrowser"

Create the init.d symlink too, and start it. Of course, choose a free port. See my FileBrowser instructions page.

Fileserver access via WebDAV

NOTE: using HTTP will cause a 301 redirect to HTTPS, and WebDAV clients will fail. So use HTTPS URL in webdav clients.

Ok, this will not be very simple, but bear with me, it will work.

It will be more complex to add more WebDAV shares, specially if associated to different users, this is left as an exercise for you.

While there are a few WebDAV servers like Dave, they seems to be either unmaintained or overly complicated. Also NGINX can be a WebDAV server, but it seems to be buggy and not supporting LOCK stuff, so i decided to go with Apache web server, which also has a long standing WebDAV implementation.

The idea here is to run a dedicated copy of Apache as user fileserver and group users so that it can access and manage the shared files. So first you need to emerge apache: 

emerge apache`

WebDAV is enabled by default in Gentoo Apache ebuild, so there is no need to fix USE flags.

You will not be running Apache as system service, and this requires some effort, so, buckle up!

Prepare apache folders for each share

First of all, Apache needs some folders to operate as normal user, so you need to create:

  • /home/common/.apache/conf: to store the apache config file
  • /home/common/.apache/roots: which will map as WebDAV root (you will see why)
  • /home/common/.apache/locks: which will be used for WebDAV lock databases
  • /home/common/.apache/pids: which will be used to store apache PID files
su - fileserver
mkdir /home/common/.apache/conf
mkdir /home/common/.apache/root
mkdir /home/common/.apache/locks
mkdir /home/common/.apache/pids

Create apache configuration files for each share

Drop this file under /etc/init.d/webdav:

webdav
#!/sbin/openrc-run                                                                            
# Copyright 2024 Willy Garidol                                                                
# Distributed under the terms of the GNU General Public License v3            
                                                                                              
depend() {                                                                                    
        need localmount net
}                                                                                             
                                               
TIMEOUT=${TIMEOUT:-5}                                                                         
WD_LOG_PATH="/var/log/webdav"               
WD_SLOT="${SVCNAME#webdav.}"                                                                  
WD_USER=${USER:-${WD_SLOT}}
WD_GROUP=${GROUP:-${WD_SLOT}}                                                                 
                                                                                              
WD_ROOT="${FOLDER}/webdav/root"                                                               
WD_MOUNT="${WD_ROOT}/webdav/${SHARE}"                                                         
                                               
description=${DESCRIPTION:-WebDAV starter}
pidfile="/run/${RC_SVCNAME}.pid"
apache_args=(                                                                                 
-f /etc/webdav/common_config.conf
-c "User ${WD_USER}"
-c "Group ${WD_GROUP}"
-c "DavLockDB ${FOLDER}/webdav/locks"
-c "PidFile ${pidfile}"
-c "ErrorLog ${WD_LOG_PATH}/${WD_SLOT}/error.log"                          
-c "TransferLog ${WD_LOG_PATH}/${WD_SLOT}/transfer.log"
-c "CustomLog ${WD_LOG_PATH}/${WD_SLOT}/access.log common"
-c "DocumentRoot ${WD_ROOT}"
-c "ServerName 127.0.0.1"
-c "Listen 127.0.0.1:${PORT}"
-c "<Directory ${WD_ROOT}>"                 
-c " DAV On"                                                                                  
-c " AllowOverride All"                                                                       
-c " Options -Indexes +FollowSymlinks -ExecCGI -Includes"
-c " Require all granted"
-c "</Directory>"
-c "SetEnv redirect-carefully"
)
start_pre() {
        if [ "${WD_SLOT}" != "webdav" ]
        then
                local SHARE=$(basename ${FOLDER})
                ebegin "Checking log path..."
                test -e "${WD_LOG_PATH}" || mkdir "${WD_LOG_PATH}"
                test -e "${WD_LOG_PATH}/${WD_SLOT}" || {
                        ebegin "Creating log path '${WD_LOG_PATH}/${WD_SLOT}'"
                        mkdir "${WD_LOG_PATH}/${WD_SLOT}"
                } && chown -R ${WD_USER} "${WD_LOG_PATH}/${WD_SLOT}"

                ebegin "Checking root path '${WD_MOUNT}'"
                test -e ${WD_MOUNT} || {
                        ebegin "Creating root path '${WD_MOUNT}'"
                        mkdir -p ${WD_MOUNT}
                        chown ${WD_USER}:${WD_GROUP} ${WD_MOUNT}
                }
                ebegin "Checking root mount/bind '${WD_MOUNT}'"
                test -z "$(mount | grep ${WD_MOUNT})" && {
                        ebegin "Mounting/binding root path '${FOLDER}/data' -> '${WD_MOUNT}'"
                        mount -o bind ${FOLDER}/data ${WD_MOUNT}
                }
                eend 0
        else
                ebegin "Error: do not run this script, run a link to it!"
                eend 255
        fi
}

start() {
        start-stop-daemon -w ${TIMEOUT} --start --pidfile "${pidfile}" -- \
                /usr/bin/apache2 "${apache_args[@]}"
        eend $?
}

stop_post() {
        ebegin "Checking root mount/bind..."
        test -n "$(mount | grep ${WD_MOUNT})" && {
                ebegin "Unmounting/unbinding root path '${FOLDER}/data' -|-> '${WD_MOUNT}'"
                umount ${WD_MOUNT}
        }
        eend 0
}

-…. TBD TBD TBD ….

work in progress….

Each share will need it's full set of Apache configuration files, because you will need to run one specific apache server for each share.

Put all the following files (after editing it to your needs!) under /home/common/.apache/conf, of course adapt folder name for each share.

This is generic, and could be shared between all shares if you prefer:

apache_global.conf
ServerRoot "/usr/lib64/apache2"
LoadModule actions_module modules/mod_actions.so
LoadModule alias_module modules/mod_alias.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_anon_module modules/mod_authn_anon.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authn_dbm_module modules/mod_authn_dbm.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule authz_dbm_module modules/mod_authz_dbm.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule autoindex_module modules/mod_autoindex.so
<IfDefine CACHE>
LoadModule cache_module modules/mod_cache.so
</IfDefine>
LoadModule dav_module modules/mod_dav.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule dav_lock_module modules/mod_dav_lock.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule expires_module modules/mod_expires.so
LoadModule ext_filter_module modules/mod_ext_filter.so
<IfDefine CACHE>
LoadModule file_cache_module modules/mod_file_cache.so
</IfDefine>
LoadModule filter_module modules/mod_filter.so
LoadModule headers_module modules/mod_headers.so
<IfDefine HTTP2>
LoadModule http2_module modules/mod_http2.so
</IfDefine>
LoadModule include_module modules/mod_include.so
<IfDefine INFO>
LoadModule info_module modules/mod_info.so
</IfDefine>
LoadModule log_config_module modules/mod_log_config.so

# This is needed to avoid error on load due to default path being not accessible
TransferLog /home/common/.apache/logs/transfer_log

LoadModule logio_module modules/mod_logio.so
LoadModule mime_module modules/mod_mime.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule setenvif_module modules/mod_setenvif.so
<IfDefine STATUS>
LoadModule status_module modules/mod_status.so
</IfDefine>
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule unixd_module modules/mod_unixd.so
<IfDefine USERDIR>
LoadModule userdir_module modules/mod_userdir.so
</IfDefine>
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
Include /etc/apache2/modules.d/*.conf

This needs to be specific for each share:

common.conf
Include /home/common/.apache/conf/apache_global.conf

User fileserver
Group users

DavLockDB "/home/common/.apache/locks/common"
PidFile /home/common/.apache/pids/common.pid
ErrorLog /home/common/.apache/logs/error_log
TransferLog /home/common/.apache/logs/transfer_log
CustomLog /home/common/.apache/logs/access_log common

DocumentRoot /home/common/.apache/roots

ServerName 127.0.0.1
Listen 127.0.0.1:10001

<Directory /home/common/.apache/roots>
    DAV On
    AllowOverride All
    Options -Indexes +FollowSymlinks -ExecCGI -Includes
    Require all granted
</Directory>

SetEnv redirect-carefully

# vim: ts=4 filetype=apache

Please note the Listen directive: you want apache to be bound to 127.0.0.1 only and note the port too, this port will be needed for the reverse proxy. Each share will need it's own port.

Messing with the WebDAV root folder

Now, the fun part is that you want to protect this behind the NGINX reverse proxy (for HTTPS and authorization reasons) and it seems that WebDAV does not play well with URL redirection and similar funny things. In other words, the base url you will be using on the reverse proxy must match the url in the Apache. You cannot use rewrite directives or Alias stuff.

Since you will be exposing the browser-based access as https://drive.mydomain.com/common and the WebDAV access as https://drive.mydomain.com/webdav/common it means that we need to connect your /home/common folder to /home/common/.apache/roots/webdav/common for it to work. Since symbolic links cannot be used by WebDAV (could it be that simple?), the only viable option is mount -o bind which needs to be done by root.

so, create the paths first: 

 su - fileserver
cd /data/daemons/fileserver/data/root
mkdir webdav
cd webdav
mkdir common

Startup Apache for the share (and autostart)

To start Apache, create the following /etc/init.d/webdav script:

webdav
...

this startup script will take take of doing the mount -o bind which is mandatory for WebDAV to work. Make the script executable of course.

Link it as /etc/init.d/webdav.common and then create the associated configuration file /etc/conf.d/webdav.common: 

ln -s /etc/init.d/webdav /etc/init.d/webdav.common
webdav.common
...

Now add it to the default runlevel and start it: 

chmod +x /etc/init.d/webdav
rc-update add webdav.common default
/etc/init.d/webdav.common start

NOTE: the following needs to be rewritten, please ignore the rest of this parsagraph.

and mount the needed shares, you can create the /data/daemons/filebrowser/apache_start.sh script:

apache_start.sh
#!/bin/bash
# Bind/Mount the share for Apache
if [ "$(mount | grep /data/daemons/filebrowser/data/roots/webdav/common)" = "" ]
then
        mount -o bind /home/common /data/daemons/filebrowser/data/roots/webdav/common
fi
su - filebrowser -c "apache2 -f /data/daemons/filebrowser/data/conf/common.conf"

(ignore up to here)

Reverse Proxy and wrap-up

Everything is protected behind the NGINX Reverse Proxy, so you should create the following config file for the drive subdomain:

drive.conf
server {
        server_name drive.mydomain.com;
        listen 443 ssl; 
        listen 8443 ssl; 
        http2 on;

        access_log /var/log/nginx/drive.mydomain.com_access_log main;
        error_log /var/log/nginx/drive.mydomain.com_error_log info;

        # WebDAV requires basic auth, while normal auth can be used with FileBrowser
        include "com.mydomain/authelia_location.conf";
        include "com.mydomain/authelia_location-basic.conf";

        location / {
                include "com.mydomain/authelia_proxy.conf";
                include "com.mydomain/authelia_authrequest.conf";
                root /home/web/drive;
        }

        location = /common {
                 return 301 https://$host/common/;
        }

        location /common/ {
                include "com.mydomain/authelia_proxy.conf";
                include "com.mydomain/authelia_authrequest.conf";
                client_max_body_size 512M;
                proxy_pass http://127.0.0.1:3002;
                proxy_set_header Connection $http_connection;
                proxy_set_header Connection 'upgrade';
                proxy_cache_bypass $http_upgrade;
       }

       location /webdav/common {
                include "com.mydomain/authelia_proxy.conf";
                include "com.mydomain/authelia_authrequest-basic.conf";

                # https://mailman.nginx.org/pipermail/nginx/2007-January/000504.html - fix Destination: header
                # https://trac.nginx.org/nginx/ticket/348 - bug, workaround with named capture
                set $dest $http_destination;
                if ($http_destination ~ "^https://(?<myvar>(.+))") {
                        set $dest http://$myvar;
                }

                # Warning: adding / at the end of the proxy_pass will break WebDAV!
                proxy_pass http://127.0.0.1:10001;
                proxy_buffering off;
                gzip off;
                proxy_pass_request_headers on;
               proxy_set_header Destination       $dest;
        }
        client_max_body_size 100M;
}

This example also shows how i have integrated SSO Authentication with the filesever.

refer to the The Reverse Proxy concept page to activate this specific NGIX configuration. Of course you need to create the Let's Encrypt certificates and the subdomain in your DNS provider.

Main Directory Page

As you can spot from the above NGINX configuration, i have defined a common landing on https://drive.mydomain.com to provide a nice page to access the individual shares.

For this i am using my Simple Dashboard with the following site.json:

site.json
{         
    "title" : "My Drive Title",
    "header" : {                  
        "img" : "",               
        "text" : "My Drive" 
        },                                 
    "content" : [              
        {
        "foldable": false,                                                                                                                                                                   
        "title": "",         
        "content":          
            [ {
                "img" : "images/folder.png",
                "text" : "Common",         
                "link" : "/common/",      
                "style" : "box-inline",         
                "new_page" : true      
            } ]
        }
        ],
     "footer" : {
        "img" : "",
        "text" : "back home",
        "style" : "footer-light",
        "link" : "https://home.mydomain.com"
    }
}

Experimental stuff

Just some additional experiments i did, for future references.

Nephele-Serve

Replacing WebDAV with Nephele-Serve (which will support also CardDAV/CalDAV in the future)

https://www.npmjs.com/package/nephele-serve https://github.com/sciactive/nephele

NPM needs to be enabled for the fileserver user: 

NPM_PACKAGES="$HOME/.npm-packages" 
mkdir -p "$NPM_PACKAGES"  
echo "prefix = $NPM_PACKAGES" >> ~/.npmrc

And in ~/.bashrc: 

# NPM packages in homedir
NPM_PACKAGES="$HOME/.npm-packages"
# Tell our environment about user-installed node tools
PATH="$NPM_PACKAGES/bin:$PATH"
# Unset manpath so we can inherit from /etc/manpath via the `manpath` command
unset MANPATH # delete if you already modified MANPATH elsewhere in your configuration  
MANPATH="$NPM_PACKAGES/share/man:$(manpath)"
# Tell Node about these packages
NODE_PATH="$NPM_PACKAGES/lib/node_modules:$NODE_PATH"

Install: 

source ~/.bashrc  
npm install -g nephele-serve

Advantages: it's a simple server that supports pam_auth. In the future, it might also replace Radicale with a single service.

Disadvantages: do not support base_url, so unable to host under /webdav even with reverse proxy.

sFtpGO WebDAV / web browser

Interesting sFtpGo support both web-browser access and WebDAV from a single tool.

You need to start it once then edit sftpgo.json: 

"external_auth_hook": "/data/daemons/fileserver/login.sh",
"webdavd": {
    "bindings": [
      {
        "port": 10001, 
        "address": "127.0.0.1",
        "enable_https": false,
        "certificate_file": "",
        "certificate_key_file": "",
        "min_tls_version": 12,
        "client_auth_type": 0,
        "tls_cipher_suites": [],
        "prefix": "/webdav/common",
        "proxy_allowed": [],
        "client_ip_proxy_header": "",
        "client_ip_header_depth": 0,
        "disable_www_auth_header": false
      }
    ],

Advnatages: easier than Apache to setup, support base_url

Disadvantages: cannot use pam_auth and cannot disable authentication. So double-auth over reverse proxy that might be annoying.

This website uses technical cookies only. No information is shared with anybody or used in any way but provide the website in your browser.

More information