Self-hosting is something that grows by the months, getting more complex and elaborated over time. You start small, with some recycled hardware and makeshift cables to host some experimental services, then you start self-hosting important stuff.
One day your home internet connection is down and so you find out about how reliability important… One day power goes out at home, and you find out that an unexpected server reboot might cause side issues… One day, maybe during a hot summer day, your USB network card fails on you, when you are on vacation…
The following are the very basics that i assume you will take into consideration, on which you should not even think to skip:
Still, bad things happens over time, and the above guidelines are just the bare minimum that is mandatory to start a self-hosting journey.
In addition to the above basics, i will illustrate two approaches. One, the simple one, will fit your bill but might prove limited over time. The advanced one, instead, resemble more how things are done at a professional level and offer better reliaiblity and scalability at a higher price range and complexity.
Home Automation doesn't really depend on the approach, more details here.
This is how i started, and while it has it's limitations and possible issues, it's a perfectly valid self.hosted approach.
The basic idea is to stay cheap, save on hardware using something you already have available and adapting to it's limitations. My choice has been an older, but powerfull, laptop which has quite a few advantages:
But it also have some drawbacks:
Most of the drawbacks can be overcome with modern USB devices. Today's USB-C bandwith, but even older USB 3.0 bandwitch, is more than enough to cater for your home needs. In fact, if you use WiFi, even the cheapest USB drive enclosure will be faster than that. If you have wired ethernet at home, 1Gbps will still be faster than your mechanical drives, and USB will not be a bottleneck. If you have all SSD's over USB and a perfectly wired 10Gbps ethernet cables and plugs, well, why cheap out on an older laptop then?
As a final word, this simpler approach works very well and might be all you need. The limitations come from the single point of failure that having only one piece of hardware implies: when it goes down, for hardware failure, software bugs or just maintenance, all of your home network will be offline. Your kids will be pestering you that their online-gaming is broken, their social media doesn't work, and your wife will be pissed off that the streamed movie doesn't work.
Let's focus. You need RAID storage, which means at very least two hard-drives (or SSDs) in RAID1. Better would be many devices on RAID6 or such, but letìs stick with two RAID1 mirrors for the simple approach.
Assuming your laptop cannot host two drives internally, you need to buy an external device. Laptops nowadays only come with various USB connectors, so i will not talk about E-SATA or other connection types, but only USB. Those are also the less expensive, so it's a win-win for the simple approach.
A few key concepts to follow:
As i said above, USB-3 and USB-C are more than capable to sustain disk troughputs and network loads, so USB will not be your bottleneck. but USB is quite unreliable, bot the protocol or bus itself, but the USB devices are usually low-quality. To ensure you have no issues over time, buy only reputable brands enclosures, do not save money on this item, and avoid RAID enclosures because i will be showing you how to leverage Linux built-in software raid, and JBOD enclosures have a better quality/price ratio.
Since laptops have limited USB ports, get only one big enclosure to fit all your drives, not many smaller ones. Prefer one which is externally powered, because they are more resillient, and buy a cheaper UPS power strip (anything in the lower end should fit) to keep the disks spinning then the occasional power outage will strike. In fact, having a laptpo which will survive a blackout while the disks don't, it's not a good nor safe idea. (remember: UPS batteries will need to be replaced every year or two, do it)
The second issue with using a laptop is that you will need at least two, better three, wired ethernet connections to use your server:
I suggest to avoid using WiFi because or reliability and bandwidth, so you need three ethernet NICs. If you are lucky, your laptop should have one, the others needs to be added via USB network cards. This is where things get a bit comples because USB network cards are quite unreliable. Luckly Linux nowadays support most of existent USB network cards, but in my experience they tend to fail quite easily. Some suggestions:
If you experience links going down, buy a different brand / model and hope for the best.
Your laptop will be your server and your router. Which means that all your services will run on it as well as all your routing tables, failover between ISPs and such.
Your server, or your laptop then, will need to be beefy enough to host any service you will need. In my experience, this is not a big issue. For low power laptops, a good video card might be useful for on-the-fly AV1 / x265 video decoding (if you want to host a media server like JellyFin).
Your laptop will be your server and your router. Which means that all your services will run on it as well as all your routing tables, failover between ISPs and such.
This means that if you mess up or need to reboot the laptop, your home will lose internet connection for a while. Also, if your laptop dies for any reason you will not only lose all your self-hosted services (until you restore a backup/replace hardware) but also everybody at home will be cut from internet.
Setting up routing with multi-ISPs (failsafe, or load sharing…) will be done manually with a few routing rules and settings (see Routing on the Home Server, Network Configuration for the Home Router and such pages).
While less glamour than using a fancy web GUI, it fits the same purpose and maybe it's also interesting to learn. This approach doesn't limit you to anything, actually might even be more fun than the advanced approach, but more error-prone e less resillient.
The advanced approach gives your more peace of mind, stronger resillience and more flexibility at the price of higher hardware cost, more knowledged involved, and more setup times. It's also more fun, but you might want to evolve from the simple to the advanced approach maybe to create a progressive investment in money and knowledge.
The basic idea is to split the simple approach by separating the two main pillars of the setup:
Using a dedicated firewall appliance like OpnSense is the best approach to unleash the full potential of your network, it let's you manage ISP failover, VLAN, DNS filtering and resolving, and much more using a nice web GUI interface on well-proven, state of the art, firewall dedicated software. Don't worry, it will not be any easier, in fact it will be much more complex to manage! But, at the same time, much more powerful and effective.
Splitting the home network management from the server providing the services ensures that you will never impact your networking even when doing maintenance / rebooting your home server. Moreover, using a semi-dedicated networking hardware helps with overall stability and resillience.
stay cheap, save on hardware using something you already have available and adapting to it's limitations. My choice has been an older, but powerfull, laptop which has quite a few advantages:
But it also have some drawbacks:
Most of the drawbacks can be overcome with modern USB devices. Today's USB-C bandwith, but even older USB 3.0 bandwitch, is more than enough to cater for your home needs. In fact, if you use WiFi, even the cheapest USB drive enclosure will be faster than that. If you have wired ethernet at home, 1Gbps will still be faster than your mechanical drives, and USB will not be a bottleneck. If you have all SSD's over USB and a perfectly wired 10Gbps ethernet cables and plugs, well, why cheap out on an older laptop then?
As a final word, this simpler approach works very well and might be all you need. The limitations come from the single point of failure that having only one piece of hardware implies: when it goes down, for hardware failure, software bugs or just maintenance, all of your home network will be offline. Your kids will be pestering you that their online-gaming is broken, their social media doesn't work, and your wife will be pissed off that the streamed movie doesn't work.
Let's focus. You need RAID storage, which means at very least two hard-drives (or SSDs) in RAID1. Better would be many devices on RAID6 or such, but letìs stick with two RAID1 mirrors for the simple approach.
Assuming your laptop cannot host two drives internally, you need to buy an external device. Laptops nowadays only come with various USB connectors, so i will not talk about E-SATA or other connection types, but only USB. Those are also the less expensive, so it's a win-win for the simple approach.
A few key concepts to follow:
As i said above, USB-3 and USB-C are more than capable to sustain disk troughputs and network loads, so USB will not be your bottleneck. but USB is quite unreliable, bot the protocol or bus itself, but the USB devices are usually low-quality. To ensure you have no issues over time, buy only reputable brands enclosures, do not save money on this item, and avoid RAID enclosures because i will be showing you how to leverage Linux built-in software raid, and JBOD enclosures have a better quality/price ratio.
Since laptops have limited USB ports, get only one big enclosure to fit all your drives, not many smaller ones. Prefer one which is externally powered, because they are more resillient, and buy a cheaper UPS power strip (anything in the lower end should fit) to keep the disks spinning then the occasional power outage will strike. In fact, having a laptpo which will survive a blackout while the disks don't, it's not a good nor safe idea. (remember: UPS batteries will need to be replaced every year or two, do it)
The second issue with using a laptop is that you will need at least two, better three, wired ethernet connections to use your server:
I suggest to avoid using WiFi because or reliability and bandwidth, so you need three ethernet NICs. If you are lucky, your laptop should have one, the others needs to be added via USB network cards. This is where things get a bit comples because USB network cards are quite unreliable. Luckly Linux nowadays support most of existent USB network cards, but in my experience they tend to fail quite easily. Some suggestions:
If you experience links going down, buy a different brand / model and hope for the best.
Your laptop will be your server and your router. Which means that all your services will run on it as well as all your routing tables, failover between ISPs and such.
Your server, or your laptop then, will need to be beefy enough to host any service you will need. In my experience, this is not a big issue. For low power laptops, a good video card might be useful for on-the-fly AV1 / x265 video decoding (if you want to host a media server like JellyFin).
Your laptop will be your server and your router. Which means that all your services will run on it as well as all your routing tables, failover between ISPs and such.
This means that if you mess up or need to reboot the laptop, your home will lose internet connection for a while. Also, if your laptop dies for any reason you will not only lose all your self-hosted services (until you restore a backup/replace hardware) but also everybody at home will be cut from internet.
Setting up routing with multi-ISPs (failsafe, or load sharing…) will be done manually with a few routing rules and settings (see Routing on the Home Server, Network Configuration for the Home Router and such pages).
While less glamour than using a fancy web GUI, it fits the same purpose and maybe it's also interesting to learn. This approach doesn't limit you to anything, actually might even be more fun than the advanced approach, but more error-prone e less resillient.