Bug#1002997: podman: Please provide a default /etc/containers/storage.conf
Hi, This completed Pull Request upstream should solve the issue of needing to manually add a storage.conf by adjusting the defaults to use the in- kernel overlayfs by default instead of vfs which is slow and very heavy on disk-space. https://github.com/containers/storage/issues/1570 https://github.com/containers/storage/pull/1571 Is there any chance this fix could be backported/cherrypicked to Debian Stable Bookworm? It's functionally equivalent to just setting the storage driver in a storage.conf configuration file, but is just the default instead. Recently moved to Bookworm and the first thing I noticed was my disk was completely full after only running a few containers since vfs was the default. Kind regards, Cian.
Bug#1002997: podman: Please provide a default /etc/containers/storage.conf
Hi, This completed Pull Request upstream should solve the issue of needing to manually add a storage.conf by adjusting the defaults to use the in- kernel overlayfs by default instead of vfs which is slow and very heavy on disk-space. https://github.com/containers/storage/issues/1570 https://github.com/containers/storage/pull/1571 Is there any chance this fix could be backported/cherrypicked to Debian Stable Bookworm? It's functionally equivalent to just setting the storage driver in a storage.conf configuration file, but is just the default instead. Recently moved to Bookworm and the first thing I noticed was my disk was completely full after only running a few containers since vfs was the default. Kind regards, Cian.
Bug#1002997: podman: Please provide a default /etc/containers/storage.conf
On Mon, Jan 10, 2022 at 4:15 AM Giuseppe Scrivano wrote: > > > @Giuseppe Scrivano what do you think? > > please keep in mind that unprivileged overlay mounts cannot use > metacopy. You still need root access on the host (CAP_SYS_ADMIN in > the initial user namespace) in order to use metacopy=on. > > While it is safe to pull random images from the network and expect they > cannot exploit the system to gain access to files outside the image > itself, there is no guarantee when you are using a handcrafted storage > repository as you seem to imply with the pen drive example. > There are so many things that can be abused that metacopy=on is the last > I'd worry about :-) For such cases, I suggest to use rootless, and rely > on the kernel to limit what the unpriviled user can do. The quote comes directly from the kernel documentation. So with that rationale, maybe the option 'metacopy=on' should be set upstream at https://github.com/containers/storage/blob/375f77c66685b14fc580daad2dc6df607fb86dee/storage.conf#L95 ? That way, Debian would pick up the change on the next upstream update. Best, -rt
Bug#1002997: podman: Please provide a default /etc/containers/storage.conf
Valentin Rothberg writes: > Hi folks, > > Thanks for pulling me in. > > On Sun, Jan 9, 2022 at 11:15 PM Reinhard Tartler wrote: > > Control: reassign -1 storage-common > Control: affects -1 podman > > Hi Philip, > > Thank you for your bug report. I'll defer to our overlay expert, Giuseppe. > > The Debian equivalent to Fedora's package 'containers-common' has the same > name in debian, and does ship a 'storage.conf' file in > /usr/share/containers/storage.conf. This is so that the local > administrator can copy it to /etc/containers/storage.conf and do local > modifications. The Debian package copies the storage.conf from the upstream > source verbatim. As you can see at > > https://github.com/containers/storage/blob/375f77c66685b14fc580daad2dc6df607fb86dee/storage.conf#L95, > the mount option 'metacopy=on' is missing even upstream. > > I am not sure why the Fedora package decided to patch the configuration file > -- I couldn't find a comment in the .src.rpm that you linked. Also, looking > at the kernel documentation you provided, it > seems your concerns re: security are justified, and the option seems to have > significant security implications: > > Do not use metacopy=on with untrusted upper/lower directories. Otherwise it > is possible that an attacker can create a handcrafted file with appropriate > REDIRECT and METACOPY xattrs, and > gain access to file on lower pointed by REDIRECT. This should not be > possible on local system as setting “trusted.” xattrs will require > CAP_SYS_ADMIN. But it should be possible for untrusted > layers like from a pen drive. > > I'm not sure whether enabling it by default is a good idea. I need to think > more about this. > > @Giuseppe Scrivano what do you think? please keep in mind that unprivileged overlay mounts cannot use metacopy. You still need root access on the host (CAP_SYS_ADMIN in the initial user namespace) in order to use metacopy=on. While it is safe to pull random images from the network and expect they cannot exploit the system to gain access to files outside the image itself, there is no guarantee when you are using a handcrafted storage repository as you seem to imply with the pen drive example. There are so many things that can be abused that metacopy=on is the last I'd worry about :-) For such cases, I suggest to use rootless, and rely on the kernel to limit what the unpriviled user can do. Regards, Giuseppe > I'd also appreciate hearing additional opinions on this, and have copied > some friends from podman upstream. Do you happen to know what's the > background / thinking in Fedora with enabling the > option metacopy=on? > > Happy New Year! > > -rt > > On Sun, Jan 2, 2022 at 9:51 AM Philip wrote: > > Package: podman > Version: 3.0.1+dfsg1-3+b2 > Severity: wishlist > > Dear Maintainer, > > I had some problems running the dockerized version of the Unifi controller > jacobalberty/unifi-docker > with podman on Debian. > On a Fedora system, starting the container only takes a few seconds. > On a Debian system, it can take about 5 minutes. > > The reason is that on the Fedora system the mount-option metacopy=on > (see [1] for this mount option) is set for the container overlayfs via a > default /etc/containers/storage.conf. > That makes quite the difference for this specific image because it does a > `chown unifi:unifi /usr/lib/unifi` during startup. > chown-ing these 6k files is fast with metacopy=on (on Fedora). > Without the option (on Debian), I think the files will be copied instead of > only their metadata, making it rather slow. > > So the solution for me was to copy /etc/containers/storage.conf from a > Fedora system. If anyone has a similar problem, the file can be extracted > from the > src rpm of the containers-common package which can be downloaded at [2]. > > IMO it would be useful if Debian would also include a default > /etc/containers/storage.conf. > Thanks for considering this! > However I'm not sure if metacopy=on is a good idea from a security > perspective. > > Best > Philip > > -- System Information: > Debian Release: 11.2 >APT prefers stable-updates >APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, > 'stable') > Architecture: amd64 (x86_64) > > Kernel: Linux 5.10.0-10-amd64 (SMP w/2 CPU threads) > Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), > LANGUAGE=en_US:en > Shell: /bin/sh linked to /usr/bin/dash > Init: systemd (via /run/systemd/system) > LSM: AppArmor: enabled > > Versions of packages podman depends on: > ii conmon 2.0.25+ds1-1.1 > ii containernetworking-plugins 0.9.0-1+b6 > ii crun 0.17+dfsg-1 > ii golang-github-containers-common 0.33.4+ds1-1 > ii init-system-helpers 1.60 > ii iptables 1.8.7-1 > ii libc62.31-13+deb11u2 > ii libdevmapper1.02.1
Bug#1002997: podman: Please provide a default /etc/containers/storage.conf
Hi folks, Thanks for pulling me in. On Sun, Jan 9, 2022 at 11:15 PM Reinhard Tartler wrote: > Control: reassign -1 storage-common > Control: affects -1 podman > > Hi Philip, > > Thank you for your bug report. I'll defer to our overlay expert, Giuseppe. > > The Debian equivalent to Fedora's package 'containers-common' has the same > name in debian, and does ship a 'storage.conf' file in > /usr/share/containers/storage.conf. This is so that the local administrator > can copy it to /etc/containers/storage.conf and do local modifications. The > Debian package copies the storage.conf from the upstream source verbatim. > As you can see at > https://github.com/containers/storage/blob/375f77c66685b14fc580daad2dc6df607fb86dee/storage.conf#L95, > the mount option 'metacopy=on' is missing even upstream. > > I am not sure why the Fedora package decided to patch the configuration > file -- I couldn't find a comment in the .src.rpm that you linked. Also, > looking at the kernel documentation you provided, it seems your concerns > re: security are justified, and the option seems to have significant > security implications: > > Do not use metacopy=on with untrusted upper/lower directories. Otherwise >> it is possible that an attacker can create a handcrafted file with >> appropriate REDIRECT and METACOPY xattrs, and gain access to file on lower >> pointed by REDIRECT. This should not be possible on local system as setting >> “trusted.” xattrs will require CAP_SYS_ADMIN. But it should be possible for >> untrusted layers like from a pen drive. > > > I'm not sure whether enabling it by default is a good idea. I need to > think more about this. > @Giuseppe Scrivano what do you think? > I'd also appreciate hearing additional opinions on this, and have copied > some friends from podman upstream. Do you happen to know what's the > background / thinking in Fedora with enabling the option metacopy=on? > > Happy New Year! > > -rt > > > On Sun, Jan 2, 2022 at 9:51 AM Philip wrote: > >> Package: podman >> Version: 3.0.1+dfsg1-3+b2 >> Severity: wishlist >> >> Dear Maintainer, >> >> I had some problems running the dockerized version of the Unifi >> controller jacobalberty/unifi-docker >> with podman on Debian. >> On a Fedora system, starting the container only takes a few seconds. >> On a Debian system, it can take about 5 minutes. >> >> The reason is that on the Fedora system the mount-option metacopy=on >> (see [1] for this mount option) is set for the container overlayfs via a >> default /etc/containers/storage.conf. >> That makes quite the difference for this specific image because it does a >> `chown unifi:unifi /usr/lib/unifi` during startup. >> chown-ing these 6k files is fast with metacopy=on (on Fedora). >> Without the option (on Debian), I think the files will be copied instead >> of only their metadata, making it rather slow. >> >> So the solution for me was to copy /etc/containers/storage.conf from a >> Fedora system. If anyone has a similar problem, the file can be extracted >> from the >> src rpm of the containers-common package which can be downloaded at [2]. >> >> IMO it would be useful if Debian would also include a default >> /etc/containers/storage.conf. >> Thanks for considering this! >> However I'm not sure if metacopy=on is a good idea from a security >> perspective. >> >> Best >> Philip >> >> -- System Information: >> Debian Release: 11.2 >> APT prefers stable-updates >> APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, >> 'stable') >> Architecture: amd64 (x86_64) >> >> Kernel: Linux 5.10.0-10-amd64 (SMP w/2 CPU threads) >> Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), >> LANGUAGE=en_US:en >> Shell: /bin/sh linked to /usr/bin/dash >> Init: systemd (via /run/systemd/system) >> LSM: AppArmor: enabled >> >> Versions of packages podman depends on: >> ii conmon 2.0.25+ds1-1.1 >> ii containernetworking-plugins 0.9.0-1+b6 >> ii crun 0.17+dfsg-1 >> ii golang-github-containers-common 0.33.4+ds1-1 >> ii init-system-helpers 1.60 >> ii iptables 1.8.7-1 >> ii libc62.31-13+deb11u2 >> ii libdevmapper1.02.1 2:1.02.175-2.1 >> ii libgpgme11 1.14.0-1+b2 >> ii libseccomp2 2.5.1-1+deb11u1 >> >> Versions of packages podman recommends: >> ii buildah 1.19.6+dfsg1-1+b6 >> ii catatonit 0.1.5-2 >> ii fuse-overlayfs1.4.0-1 >> ii golang-github-containernetworking-plugin-dnsname 1.1.1+ds1-4+b7 >> ii slirp4netns 1.0.1-2 >> ii uidmap1:4.8.1-1 >> >> Versions of packages podman suggests: >> pn containers-storage >> pn docker-compose >> >> -- no debconf information >> >> >> [1]: >> https://www.ke
Bug#1002997: podman: Please provide a default /etc/containers/storage.conf
Control: reassign -1 storage-common Control: affects -1 podman Hi Philip, Thank you for your bug report. The Debian equivalent to Fedora's package 'containers-common' has the same name in debian, and does ship a 'storage.conf' file in /usr/share/containers/storage.conf. This is so that the local administrator can copy it to /etc/containers/storage.conf and do local modifications. The Debian package copies the storage.conf from the upstream source verbatim. As you can see at https://github.com/containers/storage/blob/375f77c66685b14fc580daad2dc6df607fb86dee/storage.conf#L95, the mount option 'metacopy=on' is missing even upstream. I am not sure why the Fedora package decided to patch the configuration file -- I couldn't find a comment in the .src.rpm that you linked. Also, looking at the kernel documentation you provided, it seems your concerns re: security are justified, and the option seems to have significant security implications: Do not use metacopy=on with untrusted upper/lower directories. Otherwise it > is possible that an attacker can create a handcrafted file with appropriate > REDIRECT and METACOPY xattrs, and gain access to file on lower pointed by > REDIRECT. This should not be possible on local system as setting “trusted.” > xattrs will require CAP_SYS_ADMIN. But it should be possible for untrusted > layers like from a pen drive. I'm not sure whether enabling it by default is a good idea. I need to think more about this. I'd also appreciate hearing additional opinions on this, and have copied some friends from podman upstream. Do you happen to know what's the background / thinking in Fedora with enabling the option metacopy=on? Happy New Year! -rt On Sun, Jan 2, 2022 at 9:51 AM Philip wrote: > Package: podman > Version: 3.0.1+dfsg1-3+b2 > Severity: wishlist > > Dear Maintainer, > > I had some problems running the dockerized version of the Unifi controller > jacobalberty/unifi-docker > with podman on Debian. > On a Fedora system, starting the container only takes a few seconds. > On a Debian system, it can take about 5 minutes. > > The reason is that on the Fedora system the mount-option metacopy=on > (see [1] for this mount option) is set for the container overlayfs via a > default /etc/containers/storage.conf. > That makes quite the difference for this specific image because it does a > `chown unifi:unifi /usr/lib/unifi` during startup. > chown-ing these 6k files is fast with metacopy=on (on Fedora). > Without the option (on Debian), I think the files will be copied instead > of only their metadata, making it rather slow. > > So the solution for me was to copy /etc/containers/storage.conf from a > Fedora system. If anyone has a similar problem, the file can be extracted > from the > src rpm of the containers-common package which can be downloaded at [2]. > > IMO it would be useful if Debian would also include a default > /etc/containers/storage.conf. > Thanks for considering this! > However I'm not sure if metacopy=on is a good idea from a security > perspective. > > Best > Philip > > -- System Information: > Debian Release: 11.2 > APT prefers stable-updates > APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, > 'stable') > Architecture: amd64 (x86_64) > > Kernel: Linux 5.10.0-10-amd64 (SMP w/2 CPU threads) > Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), > LANGUAGE=en_US:en > Shell: /bin/sh linked to /usr/bin/dash > Init: systemd (via /run/systemd/system) > LSM: AppArmor: enabled > > Versions of packages podman depends on: > ii conmon 2.0.25+ds1-1.1 > ii containernetworking-plugins 0.9.0-1+b6 > ii crun 0.17+dfsg-1 > ii golang-github-containers-common 0.33.4+ds1-1 > ii init-system-helpers 1.60 > ii iptables 1.8.7-1 > ii libc62.31-13+deb11u2 > ii libdevmapper1.02.1 2:1.02.175-2.1 > ii libgpgme11 1.14.0-1+b2 > ii libseccomp2 2.5.1-1+deb11u1 > > Versions of packages podman recommends: > ii buildah 1.19.6+dfsg1-1+b6 > ii catatonit 0.1.5-2 > ii fuse-overlayfs1.4.0-1 > ii golang-github-containernetworking-plugin-dnsname 1.1.1+ds1-4+b7 > ii slirp4netns 1.0.1-2 > ii uidmap1:4.8.1-1 > > Versions of packages podman suggests: > pn containers-storage > pn docker-compose > > -- no debconf information > > > [1]: > https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html#metadata-only-copy-up > [2]: > https://kojipkgs.fedoraproject.org//packages/containers-common/1/32.fc35/src/containers-common-1-32.fc35.src.rpm > > -- regards, Reinhard
Bug#1002997: podman: Please provide a default /etc/containers/storage.conf
Package: podman Version: 3.0.1+dfsg1-3+b2 Severity: wishlist Dear Maintainer, I had some problems running the dockerized version of the Unifi controller jacobalberty/unifi-docker with podman on Debian. On a Fedora system, starting the container only takes a few seconds. On a Debian system, it can take about 5 minutes. The reason is that on the Fedora system the mount-option metacopy=on (see [1] for this mount option) is set for the container overlayfs via a default /etc/containers/storage.conf. That makes quite the difference for this specific image because it does a `chown unifi:unifi /usr/lib/unifi` during startup. chown-ing these 6k files is fast with metacopy=on (on Fedora). Without the option (on Debian), I think the files will be copied instead of only their metadata, making it rather slow. So the solution for me was to copy /etc/containers/storage.conf from a Fedora system. If anyone has a similar problem, the file can be extracted from the src rpm of the containers-common package which can be downloaded at [2]. IMO it would be useful if Debian would also include a default /etc/containers/storage.conf. Thanks for considering this! However I'm not sure if metacopy=on is a good idea from a security perspective. Best Philip -- System Information: Debian Release: 11.2 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 5.10.0-10-amd64 (SMP w/2 CPU threads) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages podman depends on: ii conmon 2.0.25+ds1-1.1 ii containernetworking-plugins 0.9.0-1+b6 ii crun 0.17+dfsg-1 ii golang-github-containers-common 0.33.4+ds1-1 ii init-system-helpers 1.60 ii iptables 1.8.7-1 ii libc62.31-13+deb11u2 ii libdevmapper1.02.1 2:1.02.175-2.1 ii libgpgme11 1.14.0-1+b2 ii libseccomp2 2.5.1-1+deb11u1 Versions of packages podman recommends: ii buildah 1.19.6+dfsg1-1+b6 ii catatonit 0.1.5-2 ii fuse-overlayfs1.4.0-1 ii golang-github-containernetworking-plugin-dnsname 1.1.1+ds1-4+b7 ii slirp4netns 1.0.1-2 ii uidmap1:4.8.1-1 Versions of packages podman suggests: pn containers-storage pn docker-compose -- no debconf information [1]: https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html#metadata-only-copy-up [2]: https://kojipkgs.fedoraproject.org//packages/containers-common/1/32.fc35/src/containers-common-1-32.fc35.src.rpm
