Re: [systemd-devel] Feedback sought: can we drop cgroupv1 support soon?
Hey there, I believe Christian may have relayed some of this already but on my side, as much as I can sympathize with the annoyance of having to support both cgroup1 and cgroup2 side by side, I feel that we're sadly nowhere near the cut off point. >From what I can gather from various stats we have, over 90% of LXD users are still on distributions relying on CGroup1. That's because most of them are using LTS releases of server distributions and those only somewhat recently made the jump to cgroup2: - RHEL 9 in May 2022 - Ubuntu 22.04 LTS in April 2022 - Debian 11 in August 2021 OpenSUSE is still on cgroup1 by default in 15.4 for some reason. All this is also excluding our two largest users, Chromebooks and QNAP NASes, neither of them made the switch yet. I honestly wouldn't be holding deprecating cgroup1 on waiting for those few to wake up and transition. Both ChromeOS and QNAP can very quickly roll it out to all their users should they want to. It's a bit trickier for OpenSUSE as it's used as the basis for SLES and so those enterprise users are unlikely to see cgroup2 any time soon. Now all of this is a problem because: - Our users are slow to upgrade. It's common for them to skip an entire LTS release and those that upgrade every time will usually wait 6 months to a year prior to upgrading to a new release. - This deprecation would prevent users of anything but the most recent release from running any newer containers. As it's common to switch to newer containers before upgrading the host, this would cause some issues. - Unfortunately the reverse is a problem too. RHEL 7 and derivatives are still very common as a container workload, as is Ubuntu 16.04 LTS. Unfortunately those releases ship with a systemd version that does not boot under cgroup2. That last issue has been biting us a bit recently but it's something that one can currently workaround by forcing systemd back into hybrid mode on the host. With the deprecation of cgroup1, this won't be possible anymore. You simply won't be able to have both CentOS7 and Fedora XYZ running in containers on the same system as one will only work on cgroup1 and the other only on cgroup2. Now this doesn't bother me at all for anything that's end of life, but RHEL 7 is only reaching EOL in June 2024 and while Ubuntu 16.04 is officially EOL, Canonical provides extended support (ESM) on it until April 2026. So given all that, my 2 cents would be that ideally systemd should keep supporting cgroup1 until June 2024 or shortly before that given the usual leg between releasing systemd and it being adopted by Linux distros. This would allow for most distros to have made it through two long term releases shipping with cgroup2, making sure the vast majority of users will finally be on cgroup2 and will also allow for those cgroup1-only workloads to have gone away. I guess that would mean holding on to cgroup1 support until EOY 2023 or thereabout? Stéphane On Thu, Jul 21, 2022 at 5:55 AM Christian Brauner wrote: > > [Cc Stéphane and Serge] > > On Thu, Jul 21, 2022 at 11:03:49AM +0200, Lennart Poettering wrote: > > Heya! > > > > It's currently a terrible mess having to support both cgroupsv1 and > > cgroupsv2 in our codebase. > > > > cgroupsv2 first entered the kernel in 2014, i.e. *eight* years ago > > (kernel 3.16). We soon intend to raise the baseline for systemd to > > kernel 4.3 (because we want to be able to rely on the existance of > > ambient capabilities), but that also means, that all kernels we intend > > to support have a well-enough working cgroupv2 implementation. > > > > hence, i'd love to drop the cgroupv1 support from our tree entirely, > > and simplify and modernize our codebase to go cgroupv2-only. Before we > > do that I'd like to seek feedback on this though, given this is not > > purely a thing between the kernel and systemd — this does leak into > > some userspace, that operates on cgroups directly. > > > > Specifically, legacy container infra (i.e. docker/moby) for the > > longest time was cgroupsv1-only. But as I understand it has since been > > updated, to cgroupsv2 too. > > > > Hence my question: is there a strong community of people who insist on > > using newest systemd while using legacy container infra? Anyone else > > has a good reason to stick with cgroupsv1 but really wants newest > > systemd? > > > > The time where we'll drop cgroupv1 support *will* come eventually > > either way, but what's still up for discussion is to determine > > precisely when. hence, please let us know! > > In general, I wouldn't mind dropping cgroup1 support in the future. > > The only thing I immediately kept thinking about is what happens to > workloads that have a v1 cgroup layout on the host possibly with an > older systemd running container workloads using a newer distro with a > systemd version without cgroup1 support. > > Think Ubuntu 18.04 host running a really new Ubuntu LTS that has a > version of systemd with cgroup1 support already dropped. P
Re: [systemd-devel] EXTERNAL: Re: Failed to Create Symbolic Link During Build
Vito, You're correct - it's a typo due to Microsoft auto correction. It is lib/systemd/system. I just discovered that the systemd.mak file provided in Buildroot contains: define SYSTEMD_INSTALL_SERVICE_AUTOVT ln -sf ../../../lib/systemd/system/getty@.service \ $(TARGET_DIR)/lib/systemd/system/autovt@.service I'll try changing /lib to /usr/lib -Original Message- From: Vito Caputo Sent: Thursday, July 21, 2022 2:16 PM To: Otten, Jeffrey (US) Cc: systemd-devel@lists.freedesktop.org Subject: EXTERNAL: Re: [systemd-devel] Failed to Create Symbolic Link During Build On Thu, Jul 21, 2022 at 03:33:34PM +, Otten, Jeffrey wrote: > I'm trying to build systemd 243.4 using Buildroot 2019.11. The lines listed > below are generated during the build. I've googled but haven't come across > any discussion of this issue. > The directory > /home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system listed > in the error below does not exist. However, the sym link does exist in the > directory > /home/ottenj/elinux/build/buildroot/fvlinux/target/usr/lib/system/system. > Does anyone have a suggestion about how to fix this error? > > ln -sf ../../../lib/system/system/getty@.service > /home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system/a > utovt@.service > ln: failed to create symbolic link > '/home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system/ > autovt@.service': No such file or directory > > At a glance that path looks wrong, typically it's "lib/systemd/system/" not "lib/system/system/" Regards, Vito Caputo
Re: [systemd-devel] Failed to Create Symbolic Link During Build
On Thu, Jul 21, 2022 at 03:33:34PM +, Otten, Jeffrey wrote: > I'm trying to build systemd 243.4 using Buildroot 2019.11. The lines listed > below are generated during the build. I've googled but haven't come across > any discussion of this issue. > The directory > /home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system listed > in the error below does not exist. However, the sym link does exist in the > directory > /home/ottenj/elinux/build/buildroot/fvlinux/target/usr/lib/system/system. > Does anyone have a suggestion about how to fix this error? > > ln -sf ../../../lib/system/system/getty@.service > /home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system/autovt@.service > ln: failed to create symbolic link > '/home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system/autovt@.service': > No such file or directory > > At a glance that path looks wrong, typically it's "lib/systemd/system/" not "lib/system/system/" Regards, Vito Caputo
[systemd-devel] Failed to Create Symbolic Link During Build
I'm trying to build systemd 243.4 using Buildroot 2019.11. The lines listed below are generated during the build. I've googled but haven't come across any discussion of this issue. The directory /home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system listed in the error below does not exist. However, the sym link does exist in the directory /home/ottenj/elinux/build/buildroot/fvlinux/target/usr/lib/system/system. Does anyone have a suggestion about how to fix this error? ln -sf ../../../lib/system/system/getty@.service /home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system/autovt@.service ln: failed to create symbolic link '/home/ottenj/elinux/build/buildroot/fvlinux/target/lib/system/system/autovt@.service': No such file or directory Jeff Otten F-22 Software Architect 817-762-2498
Re: [systemd-devel] Feedback sought: can we drop cgroupv1 support soon?
[Cc Stéphane and Serge] On Thu, Jul 21, 2022 at 11:03:49AM +0200, Lennart Poettering wrote: > Heya! > > It's currently a terrible mess having to support both cgroupsv1 and > cgroupsv2 in our codebase. > > cgroupsv2 first entered the kernel in 2014, i.e. *eight* years ago > (kernel 3.16). We soon intend to raise the baseline for systemd to > kernel 4.3 (because we want to be able to rely on the existance of > ambient capabilities), but that also means, that all kernels we intend > to support have a well-enough working cgroupv2 implementation. > > hence, i'd love to drop the cgroupv1 support from our tree entirely, > and simplify and modernize our codebase to go cgroupv2-only. Before we > do that I'd like to seek feedback on this though, given this is not > purely a thing between the kernel and systemd — this does leak into > some userspace, that operates on cgroups directly. > > Specifically, legacy container infra (i.e. docker/moby) for the > longest time was cgroupsv1-only. But as I understand it has since been > updated, to cgroupsv2 too. > > Hence my question: is there a strong community of people who insist on > using newest systemd while using legacy container infra? Anyone else > has a good reason to stick with cgroupsv1 but really wants newest > systemd? > > The time where we'll drop cgroupv1 support *will* come eventually > either way, but what's still up for discussion is to determine > precisely when. hence, please let us know! In general, I wouldn't mind dropping cgroup1 support in the future. The only thing I immediately kept thinking about is what happens to workloads that have a v1 cgroup layout on the host possibly with an older systemd running container workloads using a newer distro with a systemd version without cgroup1 support. Think Ubuntu 18.04 host running a really new Ubuntu LTS that has a version of systemd with cgroup1 support already dropped. People do actually do stuff like that. Stéphane and Serge might know more about actual use-cases in that area. But fwiw, we did have people show up with this and related problems for the last 5 years or so at conferences. Christian
[systemd-devel] Feedback sought: can we drop cgroupv1 support soon?
Heya! It's currently a terrible mess having to support both cgroupsv1 and cgroupsv2 in our codebase. cgroupsv2 first entered the kernel in 2014, i.e. *eight* years ago (kernel 3.16). We soon intend to raise the baseline for systemd to kernel 4.3 (because we want to be able to rely on the existance of ambient capabilities), but that also means, that all kernels we intend to support have a well-enough working cgroupv2 implementation. hence, i'd love to drop the cgroupv1 support from our tree entirely, and simplify and modernize our codebase to go cgroupv2-only. Before we do that I'd like to seek feedback on this though, given this is not purely a thing between the kernel and systemd — this does leak into some userspace, that operates on cgroups directly. Specifically, legacy container infra (i.e. docker/moby) for the longest time was cgroupsv1-only. But as I understand it has since been updated, to cgroupsv2 too. Hence my question: is there a strong community of people who insist on using newest systemd while using legacy container infra? Anyone else has a good reason to stick with cgroupsv1 but really wants newest systemd? The time where we'll drop cgroupv1 support *will* come eventually either way, but what's still up for discussion is to determine precisely when. hence, please let us know! Thanks, Lennart -- Lennart Poettering, Berlin
