Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
| services:fileserver [2024/02/22 08:12] – created willy | services:fileserver [Unknown date] (current) – removed - external edit (Unknown date) 127.0.0.1 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== File Server ====== | ||
| - | In order to access your files from everywhere you need the following access vectors: | ||
| - | * From a web browser (to access anywhere) | ||
| - | * Via WebDAV (to access from apps and phone) | ||
| - | * Using NFS (to access from Linux) | ||
| - | * Using SMB (to access from Windows) | ||
| - | |||
| - | It is not possible to achieve all this using one single tool, so you will need to leverage different pieces together, and i will show yo how. | ||
| - | |||
| - | The idea is to create one share area where your users will be able to store files. It is possible to extend this idea also to user-specific areas where each user can put private stuff not visible by other users, but this require a little bit extra complexity and might be addressed in the future. | ||
| - | |||
| - | You will be using your home server authentication, | ||
| - | |||
| - | I will show you some DIY glue to manage everything together. | ||
| - | |||
| - | ===== Shares Configuration ===== | ||
| - | |||
| - | Let's assume you will need one common share, called with lots of imagination **common**, and the files will be under **/ | ||
| - | |||
| - | Create a text file under **/ | ||
| - | <file txt shares> | ||
| - | SHARES=" | ||
| - | </ | ||
| - | (as an example, i defined also a second share called //other//) | ||
| - | |||
| - | where " | ||
| - | |||
| - | === Permissions and Users === | ||
| - | |||
| - | All users which need to access the shares must be in the **users** group: the **common** share will be accessible by any user in the **users** group. | ||
| - | |||
| - | You will also need to add a specific **fileserver** user to run the associated services, then go ahead and create the **/ | ||
| - | <code bash> | ||
| - | useradd -d / | ||
| - | mkdir / | ||
| - | chown fileserver: | ||
| - | </ | ||
| - | |||
| - | ===== Fileserver access via Browser ===== | ||
| - | |||
| - | There are a few software out there, but i like [[https:// | ||
| - | |||
| - | // | ||
| - | - bin: where the FileBrowser binary will be located | ||
| - | - data/db: where the FileBrowser databases files will be stored | ||
| - | - data/logs: where the various log files will be created | ||
| - | |||
| - | You need to set the //umask// for the user to **0002** so that any new files created by it will be writable by the users. | ||
| - | |||
| - | Then, as // | ||
| - | <code bash> | ||
| - | su - fileserver | ||
| - | echo "umask 0002" >> ~/.bashrc | ||
| - | source ~/.bashrc | ||
| - | mkdir bin data data/logs data/db | ||
| - | cd bin | ||
| - | tar xvf ../ | ||
| - | </ | ||
| - | |||
| - | Now, you will need to start a copy of FileBrowser for each share you want to have, and it must be owned by the user that want file permissions on that share. To achieve this, you will be using a special script called **fileserver.sh** which i will show you at the end, because it will contain also the WebDAV start stuff in it. | ||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | ## Software Installation for WebDAV access | ||
| - | |||
| - | While there are a few WebDAV servers like [Dave](https:// | ||
| - | |||
| - | The idea here is to run a dedicated copy of Apache as user // | ||
| - | `> emerge apache` | ||
| - | |||
| - | WebDAV is enabled by default in Gentoo Apache ebuild. | ||
| - | |||
| - | Running apache manually requires some effort, so, buckle up. | ||
| - | |||
| - | First of all, Apache needs some folders to operate, so you need to create: | ||
| - | |||
| - | - / | ||
| - | - / | ||
| - | - / | ||
| - | - / | ||
| - | |||
| - | ``` | ||
| - | > su - filebrowser | ||
| - | > mkdir / | ||
| - | > mkdir / | ||
| - | ``` | ||
| - | |||
| - | Then create the Apache config file for each share. You should create this config that will be used by each share **/ | ||
| - | |||
| - | ``` | ||
| - | ServerRoot "/ | ||
| - | LoadModule actions_module modules/ | ||
| - | LoadModule alias_module modules/ | ||
| - | LoadModule auth_basic_module modules/ | ||
| - | LoadModule authn_anon_module modules/ | ||
| - | LoadModule authn_core_module modules/ | ||
| - | LoadModule authn_dbm_module modules/ | ||
| - | LoadModule authn_file_module modules/ | ||
| - | LoadModule authz_core_module modules/ | ||
| - | LoadModule authz_dbm_module modules/ | ||
| - | LoadModule authz_groupfile_module modules/ | ||
| - | LoadModule authz_host_module modules/ | ||
| - | LoadModule authz_owner_module modules/ | ||
| - | LoadModule authz_user_module modules/ | ||
| - | LoadModule autoindex_module modules/ | ||
| - | < | ||
| - | LoadModule cache_module modules/ | ||
| - | </ | ||
| - | LoadModule dav_module modules/ | ||
| - | LoadModule dav_fs_module modules/ | ||
| - | LoadModule dav_lock_module modules/ | ||
| - | LoadModule deflate_module modules/ | ||
| - | LoadModule dir_module modules/ | ||
| - | LoadModule env_module modules/ | ||
| - | LoadModule expires_module modules/ | ||
| - | LoadModule ext_filter_module modules/ | ||
| - | < | ||
| - | LoadModule file_cache_module modules/ | ||
| - | </ | ||
| - | LoadModule filter_module modules/ | ||
| - | LoadModule headers_module modules/ | ||
| - | < | ||
| - | LoadModule http2_module modules/ | ||
| - | </ | ||
| - | LoadModule include_module modules/ | ||
| - | < | ||
| - | LoadModule info_module modules/ | ||
| - | </ | ||
| - | LoadModule log_config_module modules/ | ||
| - | |||
| - | # This is needed to avoid error on load due to default path being not accessible | ||
| - | TransferLog / | ||
| - | |||
| - | LoadModule logio_module modules/ | ||
| - | LoadModule mime_module modules/ | ||
| - | LoadModule mime_magic_module modules/ | ||
| - | LoadModule negotiation_module modules/ | ||
| - | LoadModule rewrite_module modules/ | ||
| - | LoadModule setenvif_module modules/ | ||
| - | < | ||
| - | LoadModule status_module modules/ | ||
| - | </ | ||
| - | LoadModule unique_id_module modules/ | ||
| - | LoadModule unixd_module modules/ | ||
| - | < | ||
| - | LoadModule userdir_module modules/ | ||
| - | </ | ||
| - | LoadModule usertrack_module modules/ | ||
| - | LoadModule vhost_alias_module modules/ | ||
| - | Include / | ||
| - | ``` | ||
| - | |||
| - | Then you can create one config file for each share. This is the file for the common share **/ | ||
| - | |||
| - | ``` | ||
| - | Include / | ||
| - | |||
| - | User filebrowser | ||
| - | Group users | ||
| - | |||
| - | DavLockDB "/ | ||
| - | PidFile / | ||
| - | ErrorLog / | ||
| - | TransferLog / | ||
| - | CustomLog / | ||
| - | |||
| - | DocumentRoot / | ||
| - | |||
| - | ServerName 127.0.0.1 | ||
| - | Listen 127.0.0.1: | ||
| - | |||
| - | < | ||
| - | DAV On | ||
| - | AllowOverride All | ||
| - | Options -Indexes +FollowSymlinks -ExecCGI -Includes | ||
| - | Require all granted | ||
| - | </ | ||
| - | |||
| - | SetEnv redirect-carefully | ||
| - | |||
| - | # vim: ts=4 filetype=apache | ||
| - | ``` | ||
| - | |||
| - | Please note the Listen directive: you want apache to be bound to 127.0.0. only and note the port too, this port will be needed for the reverse proxy. Each share will need it's own port. | ||
| - | |||
| - | Now, the fun part is that you want to protect this behind the NGINX reverse proxy and it seems that WebDAV does **not** play well with URL redirection and similar funny things. In other words, the base url you will be using on the reverse proxy must match the url in the Apache. You cannot use rewrite directives or Alias stuff. | ||
| - | |||
| - | Since you will be exposing the browser-based access as **https:// | ||
| - | |||
| - | so, create the paths first: | ||
| - | |||
| - | ``` | ||
| - | > su - filebrowser | ||
| - | > cd data/root | ||
| - | > mkdir webdav | ||
| - | > cd webdav | ||
| - | > mkdir common | ||
| - | ``` | ||
| - | |||
| - | the startup script below will take take of doing the **mount -o bind** which is mandatory for WebDAV to work. | ||
| - | |||
| - | ## Reverse Proxy | ||
| - | |||
| - | You want to integrate all this into the SSL enabled reverse proxy, which is also using PAM authentication. | ||
| - | |||
| - | Now, reverse proxy is simple, but this into **/ | ||
| - | |||
| - | ``` | ||
| - | # Browser based access here | ||
| - | location / | ||
| - | client_max_body_size 512M; | ||
| - | |||
| - | proxy_pass http:// | ||
| - | proxy_http_version 1.1; | ||
| - | |||
| - | proxy_set_header Connection $http_connection; | ||
| - | proxy_set_header Connection ' | ||
| - | proxy_cache_bypass $http_upgrade; | ||
| - | proxy_set_header Host $host; | ||
| - | proxy_set_header X-Real-IP $remote_addr; | ||
| - | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
| - | proxy_set_header X-Forwarded-Proto $scheme; | ||
| - | } | ||
| - | # WebDAV access | ||
| - | location / | ||
| - | # https:// | ||
| - | # https:// | ||
| - | set $dest $http_destination; | ||
| - | if ($http_destination ~ " | ||
| - | set $dest http:// | ||
| - | } | ||
| - | |||
| - | proxy_pass http:// | ||
| - | proxy_redirect off; | ||
| - | proxy_buffering off; | ||
| - | gzip off; | ||
| - | proxy_pass_request_headers on; | ||
| - | proxy_set_header Destination | ||
| - | proxy_set_header | ||
| - | proxy_set_header | ||
| - | proxy_set_header | ||
| - | } | ||
| - | ``` | ||
| - | |||
| - | and put this file include inside the usual **/ | ||
| - | |||
| - | ## Wrapup and Autostart | ||
| - | |||
| - | Create the **/ | ||
| - | |||
| - | ``` | ||
| - | source / | ||
| - | |||
| - | BASE_PATH=/ | ||
| - | |||
| - | for i in $SHARES | ||
| - | do | ||
| - | SHARE=$(echo $i | cut -d: -f1) | ||
| - | PORT=$(echo $i | cut -d: -f2) | ||
| - | OWNER=filebrowser | ||
| - | |||
| - | echo Starting FileBrowser for $OWNER on share $SHARE | ||
| - | su - $OWNER -c "/ | ||
| - | su - $OWNER -c "/ | ||
| - | |||
| - | if [ " | ||
| - | then | ||
| - | echo Mounting WebDAV entry points for $SHARE | ||
| - | mount -o bind / | ||
| - | else | ||
| - | echo WebDAV entry point already mounted | ||
| - | fi | ||
| - | echo Starting WebDAV backend for $OWNER on share $SHARE | ||
| - | su - $OWNER -c " | ||
| - | done | ||
| - | ``` | ||
| - | |||
| - | And the usual autostart stuff **/ | ||
| - | |||
| - | ``` | ||
| - | #!/bin/bash | ||
| - | / | ||
| - | ``` | ||
| - | |||
| - | Make both files executable. | ||
| - | |||
| - | To access via browser: open https://< | ||
| - | |||
| - | to access via WebDAV clients: https://< | ||
| - | |||
| - | Please note that using HTTP here might cause a 301 redirect to HTTPS, and WebDAV clients will fail. So use HTTPS URL in webdav clients. | ||
| - | |||
| - | # EXPERIMENTAL STUFF - nephele WebDAV | ||
| - | |||
| - | Replacing WebDAV with Nephele-Serve (which will support also CardDAV/ | ||
| - | |||
| - | https:// | ||
| - | https:// | ||
| - | |||
| - | Occorre abilitare NPM per l' | ||
| - | |||
| - | ``` | ||
| - | NPM_PACKAGES=" | ||
| - | mkdir -p " | ||
| - | echo " | ||
| - | ``` | ||
| - | |||
| - | Nel file **~/ | ||
| - | |||
| - | ``` | ||
| - | # NPM packages in homedir | ||
| - | NPM_PACKAGES=" | ||
| - | # Tell our environment about user-installed node tools | ||
| - | PATH=" | ||
| - | # Unset manpath so we can inherit from / | ||
| - | unset MANPATH # delete if you already modified MANPATH elsewhere in your configuration | ||
| - | MANPATH=" | ||
| - | # Tell Node about these packages | ||
| - | NODE_PATH=" | ||
| - | ``` | ||
| - | |||
| - | Infine: | ||
| - | |||
| - | ``` | ||
| - | source ~/ | ||
| - | npm install -g nephele-serve | ||
| - | ``` | ||
| - | |||
| - | Vantaggio: server semplice, supporta pam_auth. Potrebbe sostituire ANCHE radicale (cal/card dav) in futuro. | ||
| - | Svantaggio: ma non supporta i base_url (come apache) | ||
| - | |||
| - | |||
| - | # EXPERIMENTAL STUFF - sFtpGO WebDAV / web browser | ||
| - | |||
| - | Interessante: | ||
| - | |||
| - | https:// | ||
| - | |||
| - | Prima lancialo una volta, poi modifica il file **sftpgo.json**: | ||
| - | |||
| - | ``` | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | { | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | } | ||
| - | ], | ||
| - | ``` | ||
| - | |||
| - | |||
| - | |||
| - | |||
| - | Vantaggio: server WebDAV più semplice da settare di Apache. Supporta il basE_url (prefix) | ||
| - | Svantaggio: impossibile disbailitare l' | ||