User Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
gentoo:containers [2024/09/04 13:13] – [Podman containers autostart] willygentoo:containers [2025/05/06 05:55] (current) – willy
Line 1: Line 1:
-====== Using Containers on Gentoo ======+====== F) Using Containers on Gentoo ======
  
-Containers are a great tool that caters to some specific, and important, needs. But be aware that //containers// are not **the** solution to selfhosting-made-easy and, specifically, **containers have been created to solve different issues than self-hosting!** +Containers are a great tool that caters to some specific, and important, needs. But be aware that //containers// are not **the** solution to selfhosting-made-easy and, specifically, **containers have been created to solve different issues than self-hosting!** (since people asked, yes, Docker was created to cater to developers with the aim of simplifing the development and test phase. Only after that it became also popular for deployment. Kubernetes, instead, has been created with production deployment in mind)
  
-Bear in mind, always, that **containers** while being an astounding piece of technology, are **NOT** meant and have **NOT** been created with self-hosting in mind. Since it's easy and simple to use them for self-hosting, at least let's see how to do that properly.+but since it's easy and simple to use containers for self-hosting, at least let's see how to do that properly.
  
 Containerization technology can roughly be divided into two big categories: Containerization technology can roughly be divided into two big categories:
Line 9: Line 9:
   * Clusters for deployment (Kubernetes and the like)   * Clusters for deployment (Kubernetes and the like)
  
-Over the course of time the use of Docker in the self-hosted world has increased so much that it's getting really troublesome in my opinion. In fact containers have some great positive points for self-hosters like:+Over the course of time the use of Docker in the self-hosted world has increased so much that today it's getting dangerously too much. The main risk is that services are provided **only** with a container deployment in mind and non containerized installation is simply not supported or not documented at all. 
 + 
 +Containers have some great positive points for self-hosters like:
   * Ease of deployment (docker compose up, and there you go)   * Ease of deployment (docker compose up, and there you go)
   * Ease of upgrade (docker pull, and there you go)   * Ease of upgrade (docker pull, and there you go)
Line 15: Line 17:
   * For developers, it's easier to provide a //docker-compose.yml// rather than proper installation instruction   * For developers, it's easier to provide a //docker-compose.yml// rather than proper installation instruction
  
-I am realizing more and more the added value of using containers, so i have changed my mind over time. I still think there are negative points in relying on Docker for self-hosting:+At the beginning i was against using containers for deployment, but i am realizing more and more the added value of using containers, so i have changed my mind over time.  
 + 
 +I still think there are negative points in relying on Docker for self-hosting:
   * All services running as //root// (against Linux policies)   * All services running as //root// (against Linux policies)
   * Lots of duplicated services (how many postgress databases have you running around?)   * Lots of duplicated services (how many postgress databases have you running around?)
Line 50: Line 54:
  
 [[https://podman.io|Podman]] is a toolset to manage containers which follow the [[https://opencontainers.org/|Open Containers Initiative]] standards and is fully compatible with Docker, but provides a few improvements: [[https://podman.io|Podman]] is a toolset to manage containers which follow the [[https://opencontainers.org/|Open Containers Initiative]] standards and is fully compatible with Docker, but provides a few improvements:
-  * Doesn't run as root +  * Doesn't need to run as root 
-  * Doesn't require a daemon+  * Doesn't require a daemon (reduced attack surface)
   * Integrates with OpenRC/SystemD without reinventing the wheel   * Integrates with OpenRC/SystemD without reinventing the wheel
   * It's fully Open Source   * It's fully Open Source
Line 57: Line 61:
   * It's not monolithic but it's actually a set of tools   * It's not monolithic but it's actually a set of tools
   * And it's also fully Docker-compatible (set alias docker=podman and you are dood to go, almost)   * And it's also fully Docker-compatible (set alias docker=podman and you are dood to go, almost)
 +  * Doesn't depend on a for-profit company to exist and be kept //free//.
  
 Overall **Podman** is much more adherent to the Linux/Unix way of doing things.  Overall **Podman** is much more adherent to the Linux/Unix way of doing things. 
Line 79: Line 84:
 which i suggest to run as un-priviledged user to verify everything is working as non-root too. which i suggest to run as un-priviledged user to verify everything is working as non-root too.
  
-Now, install **podman-compose** which is a bit more complex because at this time there is no official ebuild yet.+Podman is fully compatible with **Docker Compose**, so you should just emerge it and start using compose: 
 +<code bash> 
 +emerge -vp docker-compose 
 +</code>
  
-Follow my [[gentoo:repositories|repo guide]] to create a custom repo (or use your already existing custom repo) to add the following ebuild: +Or you can choose to use **Podman Compose**, which is a compatible alternative, beware that it might be masked for your arch, in this case, just unmask it with your keyword, ex ~amd64: 
-<file - podman-compose-1.0.6.ebuild> +<code bash> 
-# Copyright 2024 Gentoo Authors +emerge -vp podman-compose 
-# Distributed under the terms of the GNU General Public License v2+</code>
  
-EAPI=8+of course, you need to pick one or the other. I am using podman-compose at this time.
  
-DISTUTILS_USE_SETUPTOOLS=rdepend 
-PYTHON_COMPAT=( python3_{10..11} ) 
  
-inherit distutils-r1+=== Podman rootless users ===
  
-DESCRIPTION="Run docker-compose files without root with podman" +When running a container rootless, which is the main point behind Podman, you might end up having some issues with user IDs.
-HOMEPAGE="https://pypi.org/project/podman-compose https://github.com/containers/podman-compose" +
-SRC_URI="https://files.pythonhosted.org/packages/65/a8/d77d2eaa85414d013047584d3aa10fac47edb328f5180ca54a13543af03a/podman-compose-1.0.6.tar.gz" +
-#SRC_URI="mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz"+
  
-LICENSE="GPL-2" +[[https://blog.christophersmart.com/2021/01/26/user-ids-and-rootless-containers-with-podman/|This]] article is a very good read on the topic.
-SLOT="0" +
-KEYWORDS="~amd64"+
  
-DEPEND="" +In short: when running rootless, the //user 0//, or root, of the container, will be mapped to your user ID, and any //additional// user will be remapped according to the content of **/etc/subuid** and **/etc/subgid** files. This means that if the container defines additioanl users, they will __not__ map to your user but to a different UID that is also sub-mapped to your user, but Linux itself will not recognize that and any files and folders created by the container will belog to this strange user.
-RDEPEND=" +
-        ${DEPEND} +
-        dev-python/pyyaml[${PYTHON_USEDEP}] +
-        dev-python/python-dotenv[${PYTHON_USEDEP}] +
-" +
-BDEPEND="" +
-</file>+
  
-as //app-containers/podman-compose-1.0.6.ebuild// and then emerge it (see [[https://bugs.gentoo.org/717748]]).+The easiest way to fix it, is to force your container to run... **as user 0**! In fact, since we are using rootless podman, that only means **as your own user** nad not actually root.
  
-I hope that this ebuild will be merged to Gentoo official repo soon. 
  
 ==== Podman networks ==== ==== Podman networks ====
Line 121: Line 114:
 To create a Podman subnet you need to run the following command after each reboot, as root: To create a Podman subnet you need to run the following command after each reboot, as root:
 <code bash> <code bash>
- >   podman network create my-container-net+podman network create my-container-net
 </code> </code>
  
Line 141: Line 134:
 networks: networks:
   my-container-net: {}   my-container-net: {}
-  </code>+</code>
      
-  I strongly suggest that you edit your docker compose files and ensure each service has it's own independent network. I will give more details for each service on it's respective page.+I strongly suggest that you edit your docker compose files and ensure each service has it's own independent network. I will give more details for each service on it's respective page.
  
  
Line 162: Line 155:
 UC_SLOT="${SVCNAME#user-containers.}" UC_SLOT="${SVCNAME#user-containers.}"
 UC_USER=${USER:-${UC_SLOT}} UC_USER=${USER:-${UC_SLOT}}
 +UC_COMPOSER_FILE=${COMPOSER_FILE:-docker-compose.yml}
 +UC_CHOWN_DIR=${CHOWN_DIR}
 if [ "${UC_SLOT}" != "user-containers" ] if [ "${UC_SLOT}" != "user-containers" ]
 then then
         UC_HOME=${HOME:-$(su - ${UC_USER} -c "pwd")}         UC_HOME=${HOME:-$(su - ${UC_USER} -c "pwd")}
 fi fi
 +
 +extra_commands="update"
 +update() {
 +        if [ "${UC_SLOT}" != "user-containers" ]
 +        then
 +                COMMAND="$(which podman)"
 +                stop
 +                ebegin "Running podman compose pull..."
 +                su - ${UC_USER} -c "${COMMAND} compose -f ${UC_COMPOSER_FILE} pull"
 +                start
 +        else
 +                ebegin "Error: do not run this script, run a link to it!"
 +                eend 255
 +        fi
 +}
  
 description=${DESCRIPTION:-You forgot to describe your container} description=${DESCRIPTION:-You forgot to describe your container}
Line 171: Line 181:
  
 start_pre() { start_pre() {
-        if [ "${WD_SLOT}" != "user-containers" ] +        if [ "${UC_SLOT}" != "user-containers" ] 
-             test -e "${UC_LOG_PATH}" || mkdir "${UC_LOG_PATH}" +        then 
-             test -e "${UC_LOG_PATH}/${UC_SLOT}" || { +                test -e "${UC_LOG_PATH}" || mkdir "${UC_LOG_PATH}" 
-                     mkdir "${UC_LOG_PATH}/${UC_SLOT}" +                test -e "${UC_LOG_PATH}/${UC_SLOT}" || { 
-             } && chown -R ${UC_USER} "${UC_LOG_PATH}/${UC_SLOT}"+                        mkdir "${UC_LOG_PATH}/${UC_SLOT}" 
 +                } && chown -R ${UC_USER} "${UC_LOG_PATH}/${UC_SLOT}" 
 +                if [ -n "${UC_CHOWN_DIR}" -a -e ${UC_CHOWN_DIR} ] 
 +                then 
 +                        chown -R ${UC_USER} ${UC_CHOWN_DIR} 
 +                fi
         else         else
                 ebegin "Error: do not run this script, run a link to it!"                 ebegin "Error: do not run this script, run a link to it!"
Line 181: Line 196:
         fi         fi
 } }
 +
  
 start() { start() {
         ebegin "Starting container '${UC_SLOT}' for user '${UC_USER}' (${UC_HOME})"         ebegin "Starting container '${UC_SLOT}' for user '${UC_USER}' (${UC_HOME})"
         COMMAND="$(which podman)"         COMMAND="$(which podman)"
-        ARGUMENTS=(compose up)+        ARGUMENTS=(compose -f ${UC_COMPOSER_FILE} up)
         ebegin " ... ensuring nat table is loaded ..."         ebegin " ... ensuring nat table is loaded ..."
         iptables -L -t nat &> /dev/null         iptables -L -t nat &> /dev/null
         ebegin " ... creating '${UC_SLOT}-net' ..."         ebegin " ... creating '${UC_SLOT}-net' ..."
         podman network create ${UC_SLOT}-net &> /dev/null         podman network create ${UC_SLOT}-net &> /dev/null
-        ebegin " ... running a 'podman compose down' just in case ..." 
         su - "${UC_USER}" -c "$(which podman) compose down" &> /dev/null         su - "${UC_USER}" -c "$(which podman) compose down" &> /dev/null
         start-stop-daemon -p ${pidfile} \         start-stop-daemon -p ${pidfile} \
Line 205: Line 220:
 stop() { stop() {
         ebegin " ... running 'podman compose down' ..."         ebegin " ... running 'podman compose down' ..."
-        su - "${UC_USER}" -c "$(which podman) compose down" &> /dev/null+        su - "${UC_USER}" -c "$(which podman) compose -f ${UC_COMPOSER_FILE} down" &> /dev/null
         start-stop-daemon -p ${pidfile} \         start-stop-daemon -p ${pidfile} \
                           -u ${UC_USER} \                           -u ${UC_USER} \
Line 232: Line 247:
 </code> </code>
  
-That's it.+That's it.  
 + 
 +The above script also provide an "update" commnand that will update your containers automatically. 
  
 ==== Using Podman for new containers ==== ==== Using Podman for new containers ====
Line 238: Line 256:
 Well, just replace any **docker** command with **podman** and you are good to go. Including the usual: Well, just replace any **docker** command with **podman** and you are good to go. Including the usual:
 <code bash> <code bash>
- > podman compose up+podman compose up
 </code> </code>
  
Line 256: Line 274:
 Start by creating a non-privileged user (let's call it //service// to match the service name), let it point to where you stored the docker-compose.yml for the service (which should be **/data/daemons/service** already) and fix the permissions: Start by creating a non-privileged user (let's call it //service// to match the service name), let it point to where you stored the docker-compose.yml for the service (which should be **/data/daemons/service** already) and fix the permissions:
 <code bash> <code bash>
- > useradd -d /data/daemons/service -m service +useradd -d /data/daemons/service -m service 
- > chown service:service -R /data/daemons/service+chown service:service -R /data/daemons/service
 </code> </code>
  
Line 264: Line 282:
 Now, most probably all you need to do is the classic (but rewritten): Now, most probably all you need to do is the classic (but rewritten):
 <code bash> <code bash>
- > su - service +su - service 
- > podman compose -f docker-compose.yml up+podman compose -f docker-compose.yml up
 </code> </code>
  
Line 276: Line 294:
 If you need to export your images from Docker to Podman (you don't if they are public images), as root, export all docker images relevant to your service (you can see them in the composer file), use //docker image ls// to list images and //docker save -o ...// to save each one of them as a tar file: If you need to export your images from Docker to Podman (you don't if they are public images), as root, export all docker images relevant to your service (you can see them in the composer file), use //docker image ls// to list images and //docker save -o ...// to save each one of them as a tar file:
 <code bash> <code bash>
- > mkdir /data/daemons/service/docker-export +mkdir /data/daemons/service/docker-export 
- > docker image ls +docker image ls 
- > docker image save -o /data/daemons/service/docker-export/image_name.tar image-id +docker image save -o /data/daemons/service/docker-export/image_name.tar image-id 
- > chown service:service /data/daemons/service/docker-export/image_name.tar+chown service:service /data/daemons/service/docker-export/image_name.tar
 </code> </code>
 (repeat for each image for the service!) (repeat for each image for the service!)
Line 285: Line 303:
 If your container uses also volumes, copy them to your //service// user, as root: If your container uses also volumes, copy them to your //service// user, as root:
 <code bash> <code bash>
- > mkdir /data/daemons/service/volumes +mkdir /data/daemons/service/volumes 
- > cp -a /var/lib/docker/volumes/service-image /data/daemons/service/volumes +cp -a /var/lib/docker/volumes/service-image /data/daemons/service/volumes 
- > chown service:service -R /data/daemons/service/volumes+chown service:service -R /data/daemons/service/volumes
 </code> </code>
 (repeat for each volume for the service!) (repeat for each volume for the service!)
Line 293: Line 311:
 Now, as user //server//, import the images and create the volumes: Now, as user //server//, import the images and create the volumes:
 <code bash> <code bash>
- > su - service +su - service 
- > podman load -i docker-export/image_name.tar+podman load -i docker-export/image_name.tar
 </code> </code>
  
Line 310: Line 328:
  
 <code bash> <code bash>
- > cp -a /var/lib/docker/volumes/my-volume /data/daemons///service///.local/share/containers/storage/volumes +cp -a /var/lib/docker/volumes/my-volume /data/daemons///service///.local/share/containers/storage/volumes 
 </code> </code>
  
Line 320: Line 338:
 - ping is restricted and cannot b performed from containers. If you need to enable it, type as root: - ping is restricted and cannot b performed from containers. If you need to enable it, type as root:
 <code bash> <code bash>
- > sysctl -w "net.ipv4.ping_group_range=0 2000000" +sysctl -w "net.ipv4.ping_group_range=0 2000000" 
 </code> </code>
 This can be made permanent in **/etc/sysctl.d**. This can be made permanent in **/etc/sysctl.d**.
Line 328: Line 346:
 - Running as simple user a container will not be allowed to bind to ports under 1024. Some ill-designed containers will insist on this. The only recurse (except don't use such broken images) is to allow ports for normal users with: - Running as simple user a container will not be allowed to bind to ports under 1024. Some ill-designed containers will insist on this. The only recurse (except don't use such broken images) is to allow ports for normal users with:
 <code bash> <code bash>
- > sysctl -w "net.ipv4.ip_unprivileged_port_start=80"+sysctl -w "net.ipv4.ip_unprivileged_port_start=80"
 </code> </code>
 This can be made permanent in **/etc/sysctl.d**. This can be made permanent in **/etc/sysctl.d**.

This website uses technical cookies only. No information is shared with anybody or used in any way but provide the website in your browser.

More information