Linux has very powerful routing capabilities, and it would be a waste not to leverage them. If you also happen to have two ISPs, you can combine those routing capabilities to obtain an highly resilient and maybe even load-balanced home network.

Let's assume you have two upstream connections (for example, one could be a cell phone link, only for emergencies) it would be great to be able to:

• Switch between the two ISPs when one goes down
• Route access to specific servers trough ISP1 or ISP2
• Route specific services trough ISP1 or ISP2
• Load-balance your traffic

Having two ISPs is important for redundancy. When you start to rely on your home services for your everyday life you want them to be always accessible, so if ISP1 goes down switch to ISP2.

If your ISP1 is, for example, much faster but with a data-cap, while ISP2 is slower, but with unlimited data? It would be great to route all traffic trough ISP1, but some apps (like usenet or torrent) trough ISP2…

More over, you will want to set-up two SSH tunnels one trough ISP1 and one trough ISP2 so in any case you have remote access.

To achieve this you need to operate on two levels:

• At NAT level to set specific rules for packet filtering & modification inside the kernel
• At route level, because packets need to be properly routed outside

Having two ISPs means that when one goes down, it would be nice if everything would switch to the other one, and maybe even automatically. In this event, your home network would be quite resilient and you would keep being able to access back home even when away.

To achieve this, you need to monitor each ISP connectivity (with a script) to detect when one is down and automatically switch all your routing to the other one.

This will be incorporated in the script described in the last part of this page, because it's not that trivial

I will assume ISP1 is your default gateway, and you can have only one default route. What if you want a specific host to be reachable trough ISP2 instead of ISP1? For example because ISP1 blocks it, or because you want that specific traffic to be hidden from ISP1?

The basic idea is that if i want to reach external-server2 via ISP2, i need to add one route rule and one nft rule.

The nft rule will instruct the NAT to send any request from the internal network trough ISP2 interface and not the default gateway. Without this, 77.77.77.77 will not be reachable from the internal network.

The route rule will make sure that 77.77.77.77 is accessed trough ISP2 and not ISP1. This will work only for the home server, unless the nft rule is also applied this is because our NAT goes trough the ISP1, but home server route for 77.77.77.77 goes trough ISP2, making that IP address unreachable for devices on the home network.

nft add rule nat postrouting oifname "enp59s0u2u4c2" ip daddr 77.77.77.77 snat to 192.168.1.254
ip route add 77.77.77.77 via 192.168.1.254 dev enp59s0u2u4c2

The first line adds a new rule in the nat table, postrouting chain, that will use enp59s0u2u4c2 as outbound interface only for packets with 77.77.77.77 as destination, and will source-nat them to ISP gateway.

The second line ensures that home server knows that all packets for 77.77.77.77 must be routed trough ISP gateway and interface.

(note: the first line assumes you have already created the nat table and postrouting chain, and you already have SNAT in place for your home server as described here)

While it is possible to route individual apps via specific interfaces, this is a bit convoluted (implies using network namespaces, called also netns) and each single executable needs to be started by root, which is a bit complex to setup.

Luckly, there is a much better way. Since we will setup each service running under it's own user, it's simpler to apply custom routing tables to specific users.

The idea is to create a custom routing table which has the desired interface/gateway different from the default one, then create a rule to force packets created by a specific range of UIDs (user IDs) to go trough that one custom table.

The tricky part is the return packet. Since when return packets gets into your interface they will not have any user-id associated with them, they will be destroyed by the kernel that will assume a DDoS attach via IP spoofing. Luckly for you, this can be easily fixed by enabling loose mode (see here for more info).

This is probably not the best solution as it opens your home server to possible IP spoofing attacks, but i believe the risk is limited as those packets would need to pass trough your ISP gateway and internal networks first. Anyway when i will come up with a better solution (probably using conntrack and by marking outgoing packets) i will update this page.

Start with enabling loose mode on the target interface for ISP2:

echo 2 > /proc/sys/net/ipv4/conf/enp59s0u2u4c2/rp_filter

Then create the new routing table with the default gateway:

ip route add default via 192.168.1.254 dev enp59s0u2u4c2 table 100

this will create a new table, called 100 (names are not allowed, only numbers here), which has ISP2 as default gateway and interface. The table 100 will be automatically created.

To check what's in the table:

ip route show table 100

Then add the rule to route packets from user with UID 1000 to the table 100:

ip rule add uidrange 1000-1000 lookup 100

From this moment, user 1000 packets will all be routed trough ISP2 even if ISP1 is the default gateway for your home network!

Of course, you can add as many users to table 100, and you can create as many tables with different default gateways as you need.

Note: with only the default route in the table 100 your user might have issues reaching your internal network devices or the ISP1 route. In that case, you might want to add more routing rules to table 100.

Just a clarification: with destination routing you will affect both the home server and any host in the home network. With user routing, you affect only users on the home server and not users on the home network. Users routing is for services on the home server.

Everything i wrote up to here is cool, i think, and probably pretty useful too. But it's complex and requires lots of commands, and what if you want to change things because ISP1 or ISP2 go offline?

Why not create a nice script to automate all these steps for us, maybe with a simple enough configuration file to setup all the peculiar cases you want to tackle?

I have created a nice bash script that will perform all these things for you. The script can be downloaded from this github repository.

It consist in two files:

• One bash script
• One configuration file

You need to fill the configuration file and put it under /etc/conf.d/routes-setup and the bash script should be linked to /etc/router-setup.sh. A good approach might be to clone the above repository under your /opt folder and then link the script to /etc.

Here are the commands, as guidelines, for you:

cd /opt
git clone https://github.com/gardiol/routes-setup.git routes-setup
cd /etc
ln -s /opt/routes-setup/routes-setup.sh .

Upgrading is just a matter of going into /opt/routes-setup and run git pull.

You need to create a proper configuration file. You can take the example and copy it to /etc/conf.d/routes-setup and edit it according to your needs.

Now, create the usual startup script under /etc/local.d/01-routes.start:

01-routes.start

#!/bin/bash
/etc/routes-setup.sh

and make it executable (chmod +x …).

Now you can access internet safely from the home network.

To learn how to reach the internal server from the internet, head to the SSH tunnel description