Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| services:photomanagement [2024/09/05 09:58] – [Tools & software] willy | services:photomanagement [Unknown date] (current) – removed - external edit (Unknown date) 127.0.0.1 | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Photos Management Server ====== | ||
| - | |||
| - | Photos (and videos!) are something very important in our everyday' | ||
| - | |||
| - | Photography has greatly changed over the years. For example i have thousands of photos in the last 20+ years with many different cameras and technologies: | ||
| - | |||
| - | Today the good old paper albums are just collecting dust in the bookshelves, | ||
| - | |||
| - | The challenge is to make the family album fun again, keep it private and make it last our grandchildren (maybe). | ||
| - | |||
| - | ====== The concepts ====== | ||
| - | |||
| - | the goal is to have the family photos neatly organized, sorted and easily accessible by family members, even (specially) the non tech-savvy ones. | ||
| - | |||
| - | There are three steps to the process: | ||
| - | * Collecting pictures and videos from the devices (cameras, phones, scans...) | ||
| - | * Organizing them into albums and collections, | ||
| - | * Display them all for easy fruition, possibly with Machine Learning for face detection and such | ||
| - | |||
| - | ===== Long term support & availability ===== | ||
| - | |||
| - | Tehcnology changes, fast. Applications are created new and dies out. The [[https:// | ||
| - | |||
| - | Using a database is **not** acceptable, since as much as logical it might be organized, you cannot extract and parse the tags& | ||
| - | |||
| - | The only viable option (CHANGE MY MIND) is having the photos organized in folders, where each folder is an album, and the folders themselves named in a clear, obvious, album title way. The tags and metadata should be either embedded in the photo files or stored in sidecar files along the photos themselves. | ||
| - | |||
| - | This is non-negotiable for me. If i didn't follow this rule since i started digitalizing my photos, today i would have nothing still readable and organized. and what about in 10 more years? | ||
| - | |||
| - | ===== Tools & software ===== | ||
| - | |||
| - | Let's be clear: there no tool that let you do that all-in-one, and i have tested almost all of them. | ||
| - | |||
| - | First of all, i discarded all **local** tools, as i want this done on the server and not on my local laptop. Ideally, every family member should be able to to it's part without //my// intervention. | ||
| - | (back in the past i used the great [[https:// | ||
| - | |||
| - | On the **web** tools the landscape is a little bit better than for the [[services: | ||
| - | |||
| - | I have tried (for each one, details in the associated page): | ||
| - | * [[services: | ||
| - | * [[services: | ||
| - | * [[services: | ||
| - | * [[services: | ||
| - | * [[services: | ||
| - | * [[services: | ||
| - | * [[services: | ||
| - | * [[services: | ||
| - | |||
| - | There are many more, which i didn't try because they where either **dead**, **dying** or non really open-source (as i stated many times, i try to steer from projects that are open-source only to generate a revenue). | ||
| - | |||
| - | ====== The Architecture & Workflow ====== | ||
| - | |||
| - | All the photos will be stored on the server RAID array (remember: this is **not** a backup, don't forget to [[selfhost: | ||
| - | * / | ||
| - | * / | ||
| - | * / | ||
| - | |||
| - | These folder shall be netowork-shared inside your home network in case you want to manage your photos via a local tool like Digikam, plus they need to be accessible via Filebrowser so that they can be managed via web too. | ||
| - | |||
| - | From a web point of view, you need a few sub-paths. I will assume all this will be accesible via **https:// | ||
| - | * https:// | ||
| - | * https:// | ||
| - | * https:// | ||
| - | * https:// | ||
| - | |||
| - | Here you can see a practical reason of **why** supporting sub-paths and not **only** sub-domains is important for web tools. Imagine having four different subdomains (and requiring separate login for each one every time...). | ||
| - | |||
| - | The **Staging** folder will need to have one fubfolder for each user, to keep uploaded photos not overlapping. The **Gallery** folder can be organized in any way you want ot organize your photo collection, and like so the **Public** one. | ||
| - | |||
| - | The workflow will be: | ||
| - | * Take photos and upload them to the server (automatically, | ||
| - | * Organize the photos on the server, via a web interface, into albums/ | ||
| - | * Have the folder published to a gallery app that ease browsing, tagging and such. | ||
| - | |||
| - | |||
| - | === Step 0: preparing users and folders === | ||
| - | |||
| - | For simple but secure management, you will need a dedicated user called **photos**, member of the **photos** group. For added accessiblity, | ||
| - | <code bash> | ||
| - | groupadd photos | ||
| - | useradd -d / | ||
| - | </ | ||
| - | |||
| - | Now you need to create the storage folders: | ||
| - | <code bash> | ||
| - | mkdir / | ||
| - | chown photos: | ||
| - | su - photos | ||
| - | echo "umask 0002" >> ~/.bashrc | ||
| - | source | ||
| - | mkdir / | ||
| - | mkdir / | ||
| - | mkdir / | ||
| - | </ | ||
| - | (the umask setting is to ensure photos are accessible by other photos users, YMMV) | ||
| - | |||
| - | Now make all that accessible via NFS by adding to your **/ | ||
| - | < | ||
| - | / | ||
| - | </ | ||
| - | and run: | ||
| - | <code bash> | ||
| - | exportfs -a | ||
| - | </ | ||
| - | |||
| - | Now make all that accessible via SAMBA (for windows machines) by adding to your **/ | ||
| - | < | ||
| - | [Photos] | ||
| - | | ||
| - | path = / | ||
| - | valid users = <your users that can access it> | ||
| - | | ||
| - | guest ok = yes | ||
| - | | ||
| - | | ||
| - | </ | ||
| - | and restart the samba service. | ||
| - | |||
| - | You might want to locally share only the **Gallery** sub-folder or separate the Gallery and the Staging sunfolders exports with different access criterias. | ||
| - | |||
| - | === Step 1: configure how to load stuff in the Staging Area === | ||
| - | |||
| - | At first i tried to use //Immich// for this job. The storage templates are somehow mandatory to this end, and it even work just fine for you. But i ended up with a more practical and effective approach since i am already using [[gentoo: | ||
| - | |||
| - | The advantage of Syncthing is that you can use it not only on mobile phones/ | ||
| - | |||
| - | In order to do so, you need to spin a second copy of Syncthing on a different port just for the photo users, since your main Syncthing copy is running with a different user. There are other ways to do this, but i think they are less secure. | ||
| - | |||
| - | The [[gentoo: | ||
| - | * Copy **/ | ||
| - | * Symlink **/ | ||
| - | * Edit **/ | ||
| - | * Run //once// " | ||
| - | * Start the service as usual | ||
| - | |||
| - | The configuration file should look something like: | ||
| - | <file - syncthing.photos> | ||
| - | SYNCTHING_USER=" | ||
| - | SYNCTHING_GROUP=" | ||
| - | SYNCTHING_HOMEDIR="/ | ||
| - | SYNCTHING_LOGFILE="/ | ||
| - | SYNCTHING_GUI_ADDRESS=" | ||
| - | </ | ||
| - | (note the //8385// port that must be different from the default //8384// not to clash with the main syncthing installation) | ||
| - | |||
| - | To add& | ||
| - | <code bash> | ||
| - | rc-update add syncthing.photos default | ||
| - | / | ||
| - | </ | ||
| - | |||
| - | As usual, install Syncthing on your phone too and link them together (see Syncthing documentation) and add one synced folder from your / | ||
| - | |||
| - | If you want to use Immich instead (or in addition to), check [[services: | ||
| - | |||
| - | |||
| - | === Step 2: setup how to organize the Staging Area files (and move to the Gallery) === | ||
| - | |||
| - | It is not a good idea to directly upload to the gallery because out mobile devices usually take lots of crap photos that you do not want to store. That's why i use a staging folder. | ||
| - | |||
| - | You can always do so by mounting the photo folders on a local PC and use some photo manangement tools (like Digikam), but the highlight here is that you can as well do that over the web, even while on vacation or from your phone/ | ||
| - | |||
| - | To do so, i use Filebrowser, | ||
| - | <file - photobrowser.sh> | ||
| - | #!/bin/bash | ||
| - | FILEBROWSER="/ | ||
| - | APP_PATH="/ | ||
| - | DATA_PATH="/ | ||
| - | BASE_URL="/ | ||
| - | PORT=3006 | ||
| - | " | ||
| - | " | ||
| - | </ | ||
| - | check your port, it needs to be unique of course. | ||
| - | |||
| - | As usual, make it start at boot with **/ | ||
| - | <file - 41-photobworser.start> | ||
| - | #!/bin/bash | ||
| - | start-stop-daemon -b -m -p / | ||
| - | </ | ||
| - | |||
| - | Make both executable! | ||
| - | |||
| - | You will wrap it up in the reverse proxy later on. | ||
| - | |||
| - | Now you can access the staging area, organize the photos, and move them to the gallery area. | ||
| - | |||
| - | === Step 3: setup the gallery (or galleries?) === | ||
| - | |||
| - | As i stated at the beginning there are many gallery tools out there. My picks are two at this time, and i cannot really choose between them, so i have installed both: | ||
| - | * Pigallery2 (install instructions [[services: | ||
| - | * HomeGallery (install instructions [[services: | ||
| - | |||
| - | Well, i have installed both so that you can choose which one you prefer each time you want to visit the gallery. | ||
| - | |||
| - | Go ahead, install them, then point them with the Reverse Proxy setup below. | ||
| - | |||
| - | === Step 4: backup === | ||
| - | |||
| - | Backing up your photos is a **must**. Nothing so far is a //backup//, so check [[selfhost: | ||
| - | |||
| - | ====== Let's put it all together ====== | ||
| - | |||
| - | For it all to come together you need both a small landing page, like a small dashboard, and a unifiyng Reverse Proxy configuration. | ||
| - | |||
| - | ===== Reverse Proxy ===== | ||
| - | |||
| - | Host all together with a similar config file: | ||
| - | <file - photos.config> | ||
| - | location /photos/ { | ||
| - | root / | ||
| - | rewrite ^/ | ||
| - | } | ||
| - | |||
| - | location / | ||
| - | proxy_pass http:// | ||
| - | proxy_http_version 1.1; | ||
| - | proxy_set_header Upgrade $http_upgrade; | ||
| - | proxy_set_header Connection ' | ||
| - | proxy_set_header Host $host; | ||
| - | proxy_cache_bypass $http_upgrade; | ||
| - | } | ||
| - | |||
| - | location / | ||
| - | proxy_pass http:// | ||
| - | proxy_http_version 1.1; | ||
| - | proxy_set_header Upgrade $http_upgrade; | ||
| - | proxy_set_header Connection ' | ||
| - | proxy_set_header Host $host; | ||
| - | proxy_cache_bypass $http_upgrade; | ||
| - | } | ||
| - | |||
| - | location / | ||
| - | client_max_body_size 512M; | ||
| - | proxy_pass http:// | ||
| - | proxy_http_version 1.1; | ||
| - | proxy_set_header Connection $http_connection; | ||
| - | proxy_set_header Connection ' | ||
| - | proxy_cache_bypass $http_upgrade; | ||
| - | proxy_set_header Host $host; | ||
| - | proxy_set_header X-Real-IP $remote_addr; | ||
| - | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
| - | proxy_set_header X-Forwarded-Proto $scheme; | ||
| - | # | ||
| - | } | ||
| - | | ||
| - | location / | ||
| - | proxy_pass http:// | ||
| - | proxy_read_timeout 120s; | ||
| - | access_log off; | ||
| - | } | ||
| - | </ | ||
| - | |||
| - | Make sure you match the ports you setup for the various services, and also check the paths and URLs since YMMV. | ||
| - | |||
| - | The **/ | ||
| - | |||
| - | ===== Dashboard ===== | ||
| - | |||
| - | Check [[services: | ||
| - | <file - site.json> | ||
| - | { | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | }, | ||
| - | " | ||
| - | [ { | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | [ { | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | },{ | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | } ] },{ | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | [ { | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | },{ | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | " | ||
| - | } ] } | ||
| - | ] | ||
| - | } | ||
| - | </ | ||
| - | |||
| - | Of course, adapt to your linkngs and download the dashboard pictures you want. | ||
| - | |||
| - | |||