User Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
gentoo:containers [2024/04/02 13:14] – willygentoo:containers [2025/05/06 05:55] (current) – willy
Line 1: Line 1:
-====== Using Containers on Gentoo ======+====== F) Using Containers on Gentoo ======
  
-Containers are a great tool that caters to some specific, and important, needs. But be aware that //containers// are not **the** solution to selfhosting-made-easy and, specifically, **containers havebeen created to solve different issues than self-hosting!** +Containers are a great tool that caters to some specific, and important, needs. But be aware that //containers// are not **the** solution to selfhosting-made-easy and, specifically, **containers have been created to solve different issues than self-hosting!** (since people asked, yes, Docker was created to cater to developers with the aim of simplifing the development and test phase. Only after that it became also popular for deployment. Kubernetes, instead, has been created with production deployment in mind)
  
-Bear in mind, always, that **containers** while being an astounding piece of technology, are **NOT** meant and have **NOT** been created with self-hosting in mind. Since it's easy and simple to use them for self-hosting, at least let's see how to do that properly.+but since it's easy and simple to use containers for self-hosting, at least let's see how to do that properly.
  
 Containerization technology can roughly be divided into two big categories: Containerization technology can roughly be divided into two big categories:
Line 9: Line 9:
   * Clusters for deployment (Kubernetes and the like)   * Clusters for deployment (Kubernetes and the like)
  
-Over the course of time the use of Docker in the self-hosted world has increased so much that it's getting really troublesome in my opinion. In fact containers have some great positive points for self-hosters like:+Over the course of time the use of Docker in the self-hosted world has increased so much that today it's getting dangerously too much. The main risk is that services are provided **only** with a container deployment in mind and non containerized installation is simply not supported or not documented at all. 
 + 
 +Containers have some great positive points for self-hosters like:
   * Ease of deployment (docker compose up, and there you go)   * Ease of deployment (docker compose up, and there you go)
   * Ease of upgrade (docker pull, and there you go)   * Ease of upgrade (docker pull, and there you go)
Line 15: Line 17:
   * For developers, it's easier to provide a //docker-compose.yml// rather than proper installation instruction   * For developers, it's easier to provide a //docker-compose.yml// rather than proper installation instruction
  
-I am realizing more and more the added value of using containers, so i have changed my mind over time. I still think there are negative points in relying on Docker for self-hosting:+At the beginning i was against using containers for deployment, but i am realizing more and more the added value of using containers, so i have changed my mind over time.  
 + 
 +I still think there are negative points in relying on Docker for self-hosting:
   * All services running as //root// (against Linux policies)   * All services running as //root// (against Linux policies)
   * Lots of duplicated services (how many postgress databases have you running around?)   * Lots of duplicated services (how many postgress databases have you running around?)
Line 50: Line 54:
  
 [[https://podman.io|Podman]] is a toolset to manage containers which follow the [[https://opencontainers.org/|Open Containers Initiative]] standards and is fully compatible with Docker, but provides a few improvements: [[https://podman.io|Podman]] is a toolset to manage containers which follow the [[https://opencontainers.org/|Open Containers Initiative]] standards and is fully compatible with Docker, but provides a few improvements:
-  * Doesn't run as root +  * Doesn't need to run as root 
-  * Doesn't require a daemon+  * Doesn't require a daemon (reduced attack surface)
   * Integrates with OpenRC/SystemD without reinventing the wheel   * Integrates with OpenRC/SystemD without reinventing the wheel
   * It's fully Open Source   * It's fully Open Source
Line 57: Line 61:
   * It's not monolithic but it's actually a set of tools   * It's not monolithic but it's actually a set of tools
   * And it's also fully Docker-compatible (set alias docker=podman and you are dood to go, almost)   * And it's also fully Docker-compatible (set alias docker=podman and you are dood to go, almost)
 +  * Doesn't depend on a for-profit company to exist and be kept //free//.
  
 Overall **Podman** is much more adherent to the Linux/Unix way of doing things.  Overall **Podman** is much more adherent to the Linux/Unix way of doing things. 
Line 62: Line 67:
 Installing Podman is pretty easy since it's in Portage repository, but let's enable docker wrapper as well, so you can "forget" you run Podman and use the //docker// command instead: Installing Podman is pretty easy since it's in Portage repository, but let's enable docker wrapper as well, so you can "forget" you run Podman and use the //docker// command instead:
 <code bash> <code bash>
- > echo "app-containers/podman wrapper" > /etc/portage/package.use/podman +echo "app-containers/podman wrapper" > /etc/portage/package.use/podman 
- > emerge podman+emerge app-containers/podman
 </code> </code>
  
Line 79: Line 84:
 which i suggest to run as un-priviledged user to verify everything is working as non-root too. which i suggest to run as un-priviledged user to verify everything is working as non-root too.
  
-Now, install **podman-compose** which is a bit more complex because at this time there is no official ebuild yet.+Podman is fully compatible with **Docker Compose**, so you should just emerge it and start using compose: 
 +<code bash> 
 +emerge -vp docker-compose 
 +</code>
  
-Follow my [[gentoo:repositories|repo guide]] to create a custom repo (or use your already existing custom repo) to add the following ebuild: +Or you can choose to use **Podman Compose**, which is a compatible alternative, beware that it might be masked for your arch, in this case, just unmask it with your keyword, ex ~amd64: 
-<file - podman-compose-1.0.6.ebuild> +<code bash> 
-# Copyright 2024 Gentoo Authors +emerge -vp podman-compose 
-# Distributed under the terms of the GNU General Public License v2+</code>
  
-EAPI=8+of course, you need to pick one or the other. I am using podman-compose at this time.
  
-DISTUTILS_USE_SETUPTOOLS=rdepend 
-PYTHON_COMPAT=( python3_{10..11} ) 
  
-inherit distutils-r1+=== Podman rootless users ===
  
-DESCRIPTION="Run docker-compose files without root with podman" +When running a container rootless, which is the main point behind Podman, you might end up having some issues with user IDs.
-HOMEPAGE="https://pypi.org/project/podman-compose https://github.com/containers/podman-compose" +
-SRC_URI="https://files.pythonhosted.org/packages/65/a8/d77d2eaa85414d013047584d3aa10fac47edb328f5180ca54a13543af03a/podman-compose-1.0.6.tar.gz" +
-#SRC_URI="mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz"+
  
-LICENSE="GPL-2" +[[https://blog.christophersmart.com/2021/01/26/user-ids-and-rootless-containers-with-podman/|This]] article is a very good read on the topic.
-SLOT="0" +
-KEYWORDS="~amd64"+
  
-DEPEND="" +In short: when running rootless, the //user 0//, or root, of the container, will be mapped to your user ID, and any //additional// user will be remapped according to the content of **/etc/subuid** and **/etc/subgid** files. This means that if the container defines additioanl users, they will __not__ map to your user but to a different UID that is also sub-mapped to your user, but Linux itself will not recognize that and any files and folders created by the container will belog to this strange user.
-RDEPEND=" +
-        ${DEPEND} +
-        dev-python/pyyaml[${PYTHON_USEDEP}] +
-        dev-python/python-dotenv[${PYTHON_USEDEP}] +
-" +
-BDEPEND="" +
-</file>+
  
-as //app-containers/podman-compose-1.0.6.ebuild// and then emerge it (see [[https://bugs.gentoo.org/717748]]).+The easiest way to fix it, is to force your container to run... **as user 0**! In fact, since we are using rootless podman, that only means **as your own user** nad not actually root.
  
-I hope that this ebuild will be merged to Gentoo official repo soon. 
  
 ==== Podman networks ==== ==== Podman networks ====
Line 121: Line 114:
 To create a Podman subnet you need to run the following command after each reboot, as root: To create a Podman subnet you need to run the following command after each reboot, as root:
 <code bash> <code bash>
- >   podman network create my-container-net+podman network create my-container-net
 </code> </code>
  
Line 141: Line 134:
 networks: networks:
   my-container-net: {}   my-container-net: {}
-  </code>+</code>
      
-  I strongly suggest that you edit your docker compose files and ensure each service has it's own independent network. I will give more details for each service on it's respective page.+I strongly suggest that you edit your docker compose files and ensure each service has it's own independent network. I will give more details for each service on it's respective page. 
 + 
 + 
 +==== Podman containers autostart ==== 
 + 
 +Autostarting containers is pretty easy if you use SystemD, but even if you don't, and i don't, it's easy enough too. 
 + 
 +<file - /etc/init.d/user-containers> 
 +#!/sbin/openrc-run 
 +# Copyright 2024 Willy Garidol 
 +# Distributed under the terms of the GNU General Public License v3 
 + 
 +depend() { 
 +        need localmount net 
 +} 
 + 
 +UC_LOG_PATH=/var/log/user-containers 
 +UC_SLOT="${SVCNAME#user-containers.}" 
 +UC_USER=${USER:-${UC_SLOT}} 
 +UC_COMPOSER_FILE=${COMPOSER_FILE:-docker-compose.yml} 
 +UC_CHOWN_DIR=${CHOWN_DIR} 
 +if [ "${UC_SLOT}" != "user-containers" ] 
 +then 
 +        UC_HOME=${HOME:-$(su - ${UC_USER} -c "pwd")} 
 +fi 
 + 
 +extra_commands="update" 
 +update() { 
 +        if [ "${UC_SLOT}" != "user-containers" ] 
 +        then 
 +                COMMAND="$(which podman)" 
 +                stop 
 +                ebegin "Running podman compose pull..." 
 +                su - ${UC_USER} -c "${COMMAND} compose -f ${UC_COMPOSER_FILE} pull" 
 +                start 
 +        else 
 +                ebegin "Error: do not run this script, run a link to it!" 
 +                eend 255 
 +        fi 
 +} 
 + 
 +description=${DESCRIPTION:-You forgot to describe your container} 
 +pidfile="/run/${RC_SVCNAME}.pid" 
 + 
 +start_pre() { 
 +        if [ "${UC_SLOT}" != "user-containers" ] 
 +        then 
 +                test -e "${UC_LOG_PATH}" || mkdir "${UC_LOG_PATH}" 
 +                test -e "${UC_LOG_PATH}/${UC_SLOT}" || { 
 +                        mkdir "${UC_LOG_PATH}/${UC_SLOT}" 
 +                } && chown -R ${UC_USER} "${UC_LOG_PATH}/${UC_SLOT}" 
 +                if [ -n "${UC_CHOWN_DIR}" -a -e ${UC_CHOWN_DIR} ] 
 +                then 
 +                        chown -R ${UC_USER} ${UC_CHOWN_DIR} 
 +                fi 
 +        else 
 +                ebegin "Error: do not run this script, run a link to it!" 
 +                eend 255 
 +        fi 
 +} 
 + 
 + 
 +start() { 
 +        ebegin "Starting container '${UC_SLOT}' for user '${UC_USER}' (${UC_HOME})" 
 +        COMMAND="$(which podman)" 
 +        ARGUMENTS=(compose -f ${UC_COMPOSER_FILE} up) 
 +        ebegin " ... ensuring nat table is loaded ..." 
 +        iptables -L -t nat &> /dev/null 
 +        ebegin " ... creating '${UC_SLOT}-net' ..." 
 +        podman network create ${UC_SLOT}-net &> /dev/null 
 +        su - "${UC_USER}" -c "$(which podman) compose down" &> /dev/null 
 +        start-stop-daemon -p ${pidfile} \ 
 +                          -1 "${UC_LOG_PATH}/${UC_SLOT}/${UC_SLOT}.out.log" \ 
 +                          -2 "${UC_LOG_PATH}/${UC_SLOT}/${UC_SLOT}.err.log" \ 
 +                          -u ${UC_USER} \ 
 +                          -d ${UC_HOME} \ 
 +                          -b -m \ 
 +                          --start  "${COMMAND}" \ 
 +                          -- ${ARGUMENTS[@]} 
 +        eend $? 
 +} 
 + 
 +stop() { 
 +        ebegin " ... running 'podman compose down' ..." 
 +        su - "${UC_USER}" -c "$(which podman) compose -f ${UC_COMPOSER_FILE} down" &> /dev/null 
 +        start-stop-daemon -p ${pidfile} \ 
 +                          -u ${UC_USER} \ 
 +                          -d ${UC_HOME} \ 
 +                          --stop ${UC_SLOT} 
 +        eend $? 
 +} 
 +</file> 
 + 
 +I assume you have a**service_name** that runs with a podman compose file as user **myuser**. 
 + 
 +Just link the above script to your new service file: 
 +<code bash> 
 +ln -s /etc/init.d/user-containers /etc/init.d/user-containers.service_name 
 +</code> 
 + 
 +and create  simple configuration file with user name and description in it: 
 +<file - /etc/conf.d/user-containers.service_name> 
 +USER=myuser 
 +DESCRIPTION="my containerized service" 
 +</file> 
 + 
 +and add the service to your desired runlevel: 
 +<code bash> 
 +rc-update add user-containers.service_name default 
 +</code> 
 + 
 +That's it.  
 + 
 +The above script also provide an "update" commnand that will update your containers automatically. 
  
 ==== Using Podman for new containers ==== ==== Using Podman for new containers ====
Line 149: Line 256:
 Well, just replace any **docker** command with **podman** and you are good to go. Including the usual: Well, just replace any **docker** command with **podman** and you are good to go. Including the usual:
 <code bash> <code bash>
- > podman compose up+podman compose up
 </code> </code>
  
Line 158: Line 265:
   * Podman containers are not restarted at boot: i will give you instructions for those services as needed.   * Podman containers are not restarted at boot: i will give you instructions for those services as needed.
   * Networks needs to be explicitly declared and created as root.   * Networks needs to be explicitly declared and created as root.
 +
 +
  
 ==== Migration from Docker to Podman ==== ==== Migration from Docker to Podman ====
Line 165: Line 274:
 Start by creating a non-privileged user (let's call it //service// to match the service name), let it point to where you stored the docker-compose.yml for the service (which should be **/data/daemons/service** already) and fix the permissions: Start by creating a non-privileged user (let's call it //service// to match the service name), let it point to where you stored the docker-compose.yml for the service (which should be **/data/daemons/service** already) and fix the permissions:
 <code bash> <code bash>
- > useradd -d /data/daemons/service -m service +useradd -d /data/daemons/service -m service 
- > chown service:service -R /data/daemons/service+chown service:service -R /data/daemons/service
 </code> </code>
  
Line 173: Line 282:
 Now, most probably all you need to do is the classic (but rewritten): Now, most probably all you need to do is the classic (but rewritten):
 <code bash> <code bash>
- > su - service +su - service 
- > podman compose -f docker-compose.yml up+podman compose -f docker-compose.yml up
 </code> </code>
  
Line 185: Line 294:
 If you need to export your images from Docker to Podman (you don't if they are public images), as root, export all docker images relevant to your service (you can see them in the composer file), use //docker image ls// to list images and //docker save -o ...// to save each one of them as a tar file: If you need to export your images from Docker to Podman (you don't if they are public images), as root, export all docker images relevant to your service (you can see them in the composer file), use //docker image ls// to list images and //docker save -o ...// to save each one of them as a tar file:
 <code bash> <code bash>
- > mkdir /data/daemons/service/docker-export +mkdir /data/daemons/service/docker-export 
- > docker image ls +docker image ls 
- > docker image save -o /data/daemons/service/docker-export/image_name.tar image-id +docker image save -o /data/daemons/service/docker-export/image_name.tar image-id 
- > chown service:service /data/daemons/service/docker-export/image_name.tar+chown service:service /data/daemons/service/docker-export/image_name.tar
 </code> </code>
 (repeat for each image for the service!) (repeat for each image for the service!)
Line 194: Line 303:
 If your container uses also volumes, copy them to your //service// user, as root: If your container uses also volumes, copy them to your //service// user, as root:
 <code bash> <code bash>
- > mkdir /data/daemons/service/volumes +mkdir /data/daemons/service/volumes 
- > cp -a /var/lib/docker/volumes/service-image /data/daemons/service/volumes +cp -a /var/lib/docker/volumes/service-image /data/daemons/service/volumes 
- > chown service:service -R /data/daemons/service/volumes+chown service:service -R /data/daemons/service/volumes
 </code> </code>
 (repeat for each volume for the service!) (repeat for each volume for the service!)
Line 202: Line 311:
 Now, as user //server//, import the images and create the volumes: Now, as user //server//, import the images and create the volumes:
 <code bash> <code bash>
- > su - service +su - service 
- > podman load -i docker-export/image_name.tar+podman load -i docker-export/image_name.tar
 </code> </code>
  
Line 219: Line 328:
  
 <code bash> <code bash>
- > cp -a /var/lib/docker/volumes/my-volume /data/daemons///service///.local/share/containers/storage/volumes +cp -a /var/lib/docker/volumes/my-volume /data/daemons///service///.local/share/containers/storage/volumes 
 </code> </code>
  
Line 229: Line 338:
 - ping is restricted and cannot b performed from containers. If you need to enable it, type as root: - ping is restricted and cannot b performed from containers. If you need to enable it, type as root:
 <code bash> <code bash>
- > sysctl -w "net.ipv4.ping_group_range=0 2000000" +sysctl -w "net.ipv4.ping_group_range=0 2000000" 
 </code> </code>
 This can be made permanent in **/etc/sysctl.d**. This can be made permanent in **/etc/sysctl.d**.
Line 237: Line 346:
 - Running as simple user a container will not be allowed to bind to ports under 1024. Some ill-designed containers will insist on this. The only recurse (except don't use such broken images) is to allow ports for normal users with: - Running as simple user a container will not be allowed to bind to ports under 1024. Some ill-designed containers will insist on this. The only recurse (except don't use such broken images) is to allow ports for normal users with:
 <code bash> <code bash>
- > sysctl -w "net.ipv4.ip_unprivileged_port_start=80"+sysctl -w "net.ipv4.ip_unprivileged_port_start=80"
 </code> </code>
 This can be made permanent in **/etc/sysctl.d**. This can be made permanent in **/etc/sysctl.d**.

This website uses technical cookies only. No information is shared with anybody or used in any way but provide the website in your browser.

More information