Differences

This shows you the differences between two versions of the page.

--- sailing:storagesetup [2023/11/29 14:56] – willy
+++ sailing:storagesetup [2024/02/06 15:25] (current) – removed willy
@@ Line 1: / Line 1: @@
-Prev to: [[sailing:hwossetup|Hardware selection]]
-Next to: [[sailing:networksetup|Network setup]]
-----
-===== Storage Setup =====
-The idea is to store your entire media collection on a redundant software RAID-1 array located on a different drive from the one where the Operating System is installed. In this way it will be easier to migrate to a new server in the future and your data will be safe from a disk failure. Now, there are different solutions you can choose from. You could go RAID-5, RAID-0+1, and many more combinations. RAID-1 has a few advantages for me which are:
-  * Good balance with wasted disk space (50% usage)
-  * Fast enough on read (reads will be balanced on both disks)
-  * Solid enough to survive one disk fail (provided you monitor the RAID status and replace failed disks)
-If you feel like exploring different setups, go ahead.
-As i said earlier, i have been using Linux software RAID implementation for more than one decade and i have never been let down. It's solid, it's simple, it works and it's efficient. If you choose to use a commercial external RAID solution, skip the RAID part ahead. I will assume you have two external drives called** /dev/sdb** and **/dev/s**dc (**/dev/sda** i will assume it's the drive where Gentoo is installed). The size of the two disks is not important (i suggest the biggest ones you can afford) as well as the technology. SSDs are more silent, which is a plus for a home server, and consume less power, but they are still more expensive than traditional drives. Any way, it doesn't matter for the following.0
-I will add some speed considerations: you will be streaming your media over your home network, which more than often means WiFi. A good USB-3 SSD is more than capable to keep up any data transfer requirement for any streamed media today, even 4K, so there is not need to worry that external disks or USB-3 might be a bottleneck.
-Note: i will refer to //two// disks, but you can create more RAID arrays if you have //four// disks!
-===== Partitioning =====
-To create a software RAID, you need to first partition the two drives, for this job you can use the good old fdisk:
-<code bash>
- > su
- > fdisk /dev/sdb
-</code>
-Now, you need to initialize the disk (if it's a new disk), and in this case i suggest you initialize it as a GPT disk nowadays (fdisk command 'g'). If your disk is already initialized, keep it any way it is, but at this point delete all the partitions in it and please note that this will erase any data on the disk. If you do not want to delete your data, there are many ways to do so but are out of scope here, so please go ahead and use new disks or disks where you can delete all the data.
-(if you need to retain your data and you have only two disks, you can create the RAID only on one of the two, which will be deleted, mount with only one drive, copy the data over from the other disk, then format the other disk and hot-add it to the RAID-1. How to do this in details it's not difficult to figure out, but be careful not to lose your data in the process)
-Using //fdisk//, create one partition on each drive to fill it, that will be called **/dev/sdb1** and **/dev/sdc1**, these two partitions type needs to be of type //Linux RAID//. I assume the two drives are of the same size. If not, consider that the bigger one will have wasted space. In this case, create the partitions of identical sizes on both drives: the biggest drive will have free spare space that you can partition again as non-RAID partition.
-Save the changes and quit from fdisk, since the disks are not being used yet, you will not need to reboot the server.
-===== Creating the RAID array =====
-Then it's time to create the raid array:
- > mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
-===== Format and mount =====
-format it, and mount it as /data:
- > mkfs.ext4 /dev/md0
- > mkdir /data
- > mount /dev/md0 /data
-/data will be the entry point for your media collection. You will also store all the temporary files and executables for the various software stack that we will be using.
-Your raid needs to be automatically mounted at every boot, so you need to add a line like this to /etc/fstab:
-/dev/md0        /data     ext4            noatime         0 0
-(the noatime option will reduce USB traffic and wear-and-tear. You might want a different filesystem maybe, do your own research)
-===== Automate RAID at boot =====
-You also want to automate linux raid startup:
- > rc-update add mdraid boot
-and, maybe, you need to ensure the md0 device doesnt change name upon reboot (it happened to me sometimes), so put this line into your /etc/mdadm.conf:
-ARRAY /dev/md0 UUID=1758bcfa:67af3a42:d3df2d83:ecbb0728
-where the UUID can be read by the output of the command:
- > mdadm -detail /dev/md0
-Using the UUID ensures that even if your USB ports get shuffled around (and it happens, specially if you unplug/replug wrong) your arrays will still boot fine.
-===== Prepare the disk for the media collection =====
-You need to create the individual collections entry points like:
- > mkdir /data/Films
- > mkdir /data/Tv
- > mkdir /data/Music
- > mkdir /data/Books
- > mkdir /data/daemons
-The last one will not contain actual media, but it will be used to store the installation (or the cache folders) for the various softwares described in this page.
-You will need do change the ownership of the folders, but you can do it after you have installed the *Arr's, since that step will also create the associated users. What you can do now is at least change the group ownership to the **media** group:
-<code bash>
- > chgrp media -R /data/*
-</code>
-Ok, all done? Go to networking setup...
-----
-Next to: [[sailing:networksetup|Network setup]]
-Prev to: [[sailing:hwossetup|Hardware selection]]

Willy's Wiki

User Tools

Differences