Containers are a great tool that caters to some specific, and important, needs. But be aware that containers are not the solution to selfhosting-made-easy and, specifically, containers have been created to solve different issues than self-hosting!
Bear in mind, always, that containers while being an astounding piece of technology, are NOT meant and have NOT been created with self-hosting in mind. Since it's easy and simple to use them for self-hosting, at least let's see how to do that properly.
Containerization technology can roughly be divided into two big categories:
Over the course of time the use of Docker in the self-hosted world has increased so much that it's getting really troublesome in my opinion. In fact containers have some great positive points for self-hosters like:
I am realizing more and more the added value of using containers, so i have changed my mind over time. I still think there are negative points in relying on Docker for self-hosting:
In short, the spreading usage of Docker in the self-hosting world bring to Linux some bad practices and philosophy of the Windows world. Is this really necessary? Do we really want to become forced to be idiot-users who don't know any better? Is this really what means to be Linux users?
Question for your thoughts, now let's see how, at least, to use containers in a slightly better way.
We are already at the point where some services are provided only as containers, and this is what i think the the worse possible outcome of this using containers more and more. Where is the choice? Why would i be forced to use some semi-proprietary technology to deploy Open Source services?
I think that this is a trend that should be stopped. Please do provide containers and docker files, it's good and why not, but always also support bare-metal installations which means, please always provide binary distributions and installation instructions because without those, we will be nothing better than idiot-users and your service cannot be really called open.
Everybody uses Docker. You know why? Because it was the first kid on the block.
Docker, in today's tech lingo, means container, but it's not true. Docker was only the first one and it has a few drawbacks:
Luckily there are alternatives, which are better in my opinion.
Podman is a toolset to manage containers which follow the Open Containers Initiative standards and is fully compatible with Docker, but provides a few improvements:
Overall Podman is much more adherent to the Linux/Unix way of doing things.
Installing Podman is pretty easy since it's in Portage repository, but let's enable docker wrapper as well, so you can “forget” you run Podman and use the docker command instead:
echo "app-containers/podman wrapper" > /etc/portage/package.use/podman emerge app-containers/podman
and it will automatically act as “podman” You will need to unmerge docker-cli and docker-compose, if you have them installed.
To ensure that Podman will use Docker repository to pull images, add the following line to your /etc/containers/registries.conf:
unqualified-search-registries = ["docker.io" ]
You can, as with Docker, test your Podman installation with the command:
> podman run --rm hello-world
which i suggest to run as un-priviledged user to verify everything is working as non-root too.
Now, install podman-compose which is a bit more complex because at this time there is no official ebuild yet.
Follow my repo guide to create a custom repo (or use your already existing custom repo) to add the following ebuild:
# Copyright 2024 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
EAPI=8
DISTUTILS_USE_SETUPTOOLS=rdepend
PYTHON_COMPAT=( python3_{10..11} )
inherit distutils-r1
DESCRIPTION="Run docker-compose files without root with podman"
HOMEPAGE="https://pypi.org/project/podman-compose https://github.com/containers/podman-compose"
SRC_URI="https://files.pythonhosted.org/packages/65/a8/d77d2eaa85414d013047584d3aa10fac47edb328f5180ca54a13543af03a/podman-compose-1.0.6.tar.gz"
#SRC_URI="mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz"
LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~amd64"
DEPEND=""
RDEPEND="
${DEPEND}
dev-python/pyyaml[${PYTHON_USEDEP}]
dev-python/python-dotenv[${PYTHON_USEDEP}]
"
BDEPEND=""
as app-containers/podman-compose-1.0.6.ebuild and then emerge it (see https://bugs.gentoo.org/717748).
I hope that this ebuild will be merged to Gentoo official repo soon.
When running a container rootless, which is the main point behind Podman, you might end up having some issues with user IDs.
This article is a very good read on the topic.
In short: when running rootless, the user 0, or root, of the container, will be mapped to your user ID, and any additional user will be remapped according to the content of /etc/subuid and /etc/subgid files. This means that if the container defines additioanl users, they will not map to your user but to a different UID that is also sub-mapped to your user, but Linux itself will not recognize that and any files and folders created by the container will belog to this strange user.
The easiest way to fix it, is to force your container to run… as user 0! In fact, since we are using rootless podman, that only means as your own user nad not actually root.
Since you want to run containers as non-root, you need to create as root the networks. My approach is to create one subnetwork for each group of containers, and each group of containers will run a non-root user.
To create a Podman subnet you need to run the following command after each reboot, as root:
> podman network create my-container-net
Inside your docker-compose.yml you will add a new network called my-container-net, for eaxmple:
services:
.....
environment:
....
ports:
- xxx:yyy
volumes:
- my-volume:/root/.config/:ro
networks:
- my-container-net
....
networks:
my-container-net: {}
I strongly suggest that you edit your docker compose files and ensure each service has it's own independent network. I will give more details for each service on it's respective page.
Autostarting containers is pretty easy if you use SystemD, but even if you don't, and i don't, it's easy enough too.
#!/sbin/openrc-run
# Copyright 2024 Willy Garidol
# Distributed under the terms of the GNU General Public License v3
depend() {
need localmount net
}
UC_LOG_PATH=/var/log/user-containers
UC_SLOT="${SVCNAME#user-containers.}"
UC_USER=${USER:-${UC_SLOT}}
UC_COMPOSER_FILE=${COMPOSER_FILE:-docker-compose.yml}
UC_CHOWN_DIR=${CHOWN_DIR}
if [ "${UC_SLOT}" != "user-containers" ]
then
UC_HOME=${HOME:-$(su - ${UC_USER} -c "pwd")}
fi
extra_commands="update"
update() {
if [ "${UC_SLOT}" != "user-containers" ]
then
COMMAND="$(which podman)"
stop
ebegin "Running podman compose pull..."
su - ${UC_USER} -c "${COMMAND} compose -f ${UC_COMPOSER_FILE} pull"
start
else
ebegin "Error: do not run this script, run a link to it!"
eend 255
fi
}
description=${DESCRIPTION:-You forgot to describe your container}
pidfile="/run/${RC_SVCNAME}.pid"
start_pre() {
if [ "${UC_SLOT}" != "user-containers" ]
then
test -e "${UC_LOG_PATH}" || mkdir "${UC_LOG_PATH}"
test -e "${UC_LOG_PATH}/${UC_SLOT}" || {
mkdir "${UC_LOG_PATH}/${UC_SLOT}"
} && chown -R ${UC_USER} "${UC_LOG_PATH}/${UC_SLOT}"
if [ -n "${UC_CHOWN_DIR}" -a -e ${UC_CHOWN_DIR} ]
then
chown -R ${UC_USER} ${UC_CHOWN_DIR}
fi
else
ebegin "Error: do not run this script, run a link to it!"
eend 255
fi
}
start() {
ebegin "Starting container '${UC_SLOT}' for user '${UC_USER}' (${UC_HOME})"
COMMAND="$(which podman)"
ARGUMENTS=(compose -f ${UC_COMPOSER_FILE} up)
ebegin " ... ensuring nat table is loaded ..."
iptables -L -t nat &> /dev/null
ebegin " ... creating '${UC_SLOT}-net' ..."
podman network create ${UC_SLOT}-net &> /dev/null
su - "${UC_USER}" -c "$(which podman) compose down" &> /dev/null
start-stop-daemon -p ${pidfile} \
-1 "${UC_LOG_PATH}/${UC_SLOT}/${UC_SLOT}.out.log" \
-2 "${UC_LOG_PATH}/${UC_SLOT}/${UC_SLOT}.err.log" \
-u ${UC_USER} \
-d ${UC_HOME} \
-b -m \
--start "${COMMAND}" \
-- ${ARGUMENTS[@]}
eend $?
}
stop() {
ebegin " ... running 'podman compose down' ..."
su - "${UC_USER}" -c "$(which podman) compose -f ${UC_COMPOSER_FILE} down" &> /dev/null
start-stop-daemon -p ${pidfile} \
-u ${UC_USER} \
-d ${UC_HOME} \
--stop ${UC_SLOT}
eend $?
}
I assume you have aservice_name that runs with a podman compose file as user myuser.
Just link the above script to your new service file:
ln -s /etc/init.d/user-containers /etc/init.d/user-containers.service_name
and create simple configuration file with user name and description in it:
USER=myuser DESCRIPTION="my containerized service"
and add the service to your desired runlevel:
rc-update add user-containers.service_name default
That's it.
The above script also provide an “update” commnand that will update your containers automatically.
Well, just replace any docker command with podman and you are good to go. Including the usual:
> podman compose up
That's it.
Things to remember (differences from Docker):
If you already have some services (let's call it service) running with Docker, here is how to convert them.
Start by creating a non-privileged user (let's call it service to match the service name), let it point to where you stored the docker-compose.yml for the service (which should be /data/daemons/service already) and fix the permissions:
> useradd -d /data/daemons/service -m service > chown service:service -R /data/daemons/service
At this point you might want to edit the network part of the docker compose file according to what i wrote above.
Now, most probably all you need to do is the classic (but rewritten):
> su - service > podman compose -f docker-compose.yml up
and enjoy.
(remember to create the network, as root)
If you need to export your images from Docker to Podman (you don't if they are public images), as root, export all docker images relevant to your service (you can see them in the composer file), use docker image ls to list images and docker save -o … to save each one of them as a tar file:
> mkdir /data/daemons/service/docker-export > docker image ls > docker image save -o /data/daemons/service/docker-export/image_name.tar image-id > chown service:service /data/daemons/service/docker-export/image_name.tar
(repeat for each image for the service!)
If your container uses also volumes, copy them to your service user, as root:
> mkdir /data/daemons/service/volumes > cp -a /var/lib/docker/volumes/service-image /data/daemons/service/volumes > chown service:service -R /data/daemons/service/volumes
(repeat for each volume for the service!)
Now, as user server, import the images and create the volumes:
> su - service > podman load -i docker-export/image_name.tar
Volumes might be more interesting when migrating existing services, because volumes will contain your data, including databases and configurations.
The quickest way is to:
The last step is easy because you can find your Docker volumes under /var/lib/docker/volumes and your Podman volumes (with the same names!) under /data/daemons/service/.local/share/containers/storage/volumes, so all you need to do (as root) is perform a full copy, but don't forget to check owership of the files, since that might change from Docker to Podman:
> cp -a /var/lib/docker/volumes/my-volume /data/daemons///service///.local/share/containers/storage/volumes
- ping is restricted and cannot b performed from containers. If you need to enable it, type as root:
> sysctl -w "net.ipv4.ping_group_range=0 2000000"
This can be made permanent in /etc/sysctl.d.
- Gentoo on OpenRC will NOT kill your apps in background when logging-out from a user. You don't have anything to setup for Podman containers to properly run in background (point to OpenRC over SystemD).
- Running as simple user a container will not be allowed to bind to ports under 1024. Some ill-designed containers will insist on this. The only recurse (except don't use such broken images) is to allow ports for normal users with:
> sysctl -w "net.ipv4.ip_unprivileged_port_start=80"
This can be made permanent in /etc/sysctl.d.
- Podman containers are not restarted at boot. To achieve the same behaviour as Docker, my suggestion is to follow my lead on leveraging local.d with Custom User Services where i show you how i manage very easily to auto-start containers at boot and properly shut them down too, without writing a single line of script.
If you want to do things manually, just keen in mind that you need to ensure nat ip-table is loaded. While it is not used by podman, it seems to be necessary somehow to exist:
modprobe iptable_nat iptable -L -t nat