Re: [gentoo-dev] Update on the 23.0 profiles
Michael Orlitzky wrote: On Sun, 2024-04-07 at 15:07 +0200, Andreas K. Huettel wrote: tl;dr can we turn them back off in the profile? In any scenario where they are beneficial, there's a better place to put them. Easily doable with lzma, if there is consensus for it. Slightly more complex for zstd since this affects gcc and binutils. Still doable though. Thanks: * https://bugs.gentoo.org/928932 * https://bugs.gentoo.org/928933 I know this thread is only for people actually involved in Gentoo decision making, but I'll add my 2c anyway. I'm sure nobody is surprised that I support Michael Orlitzky here 100%. My personal "dream" is to have a Gentoo in the future where *all* compression is optional, only enabled by those who want it, not forced on anybody. In my opinion the importance of compression in general diminishes every year that goes by as naturally the trend in storage space has to be that it increases. So compression will increasingly become a) an extra undesirable security risk (it's quite complex to write and maintain which only increases rather than decreases the likelihood of security issues) and b) a cpu cycle waster (cpu resources will likely remain more precious than storage). I'd love to eventually see a Gentoo where most upstream source is pulled in untouched and uncompressed by default and if people want compression they can enable it. So I would hope that as each new profile release comes, Gentoo becomes less chained to any particular compression libraries than it was before, not more. But I'm aware this is unrealistic today from the pov of Gentoo infra. Still, I'm allowed to dream right? P.S. This is not a demand, just 2c, and yes, I do know ultimately the ones who role their sleeves up and submit patches will decide these things. Eddie
Re: [gentoo-dev] Update on the 23.0 profiles
On Sun, 2024-04-07 at 15:07 +0200, Andreas K. Huettel wrote: > > tl;dr can we turn them back off in the profile? In any scenario where > > they are beneficial, there's a better place to put them. > > Easily doable with lzma, if there is consensus for it. > > Slightly more complex for zstd since this affects gcc and binutils. > Still doable though. > Thanks: * https://bugs.gentoo.org/928932 * https://bugs.gentoo.org/928933
Re: [gentoo-dev] Update on the 23.0 profiles
On Mon, 2024-04-08 at 01:22 +0100, Alex Boag-Munroe wrote: > On Sun, 7 Apr 2024 at 22:09, Michael Orlitzky wrote: > > > What I am saying is that I want the freedom to not have things > > pointlessly enabled on my systems, because similar problems (and worse) > > happen all day every day. The less exposure I have, the better. The > > liblzma backdoor was timely because it will prevent most people from > > telling me I'm being paranoid, but it could have been USE=anything on > > any other day. Moving the defaults out of the high-level profiles will > > give control back to the user, hence my complaint about it. > > > > I agree, to be honest. The spirit of profiles has always felt like it > switches on safe/sane defaults that you'd expect for the name (a > desktop plasma profile switches on all the useful desktop USE flags, a > basic profile enables the bare minimum for a bootable system, etc), > giving an expected functionality in the resulting outcome of a > re-merge of world. Precisely. > Outside of this, preferred compression tools, preferred editors > etc...should be up to the user, or implied in the profile name if it's > going to be switched on in the profile defaults. I don't use zstd > myself, I prefer xz or lz4 depending on my purpose. It's on my system > because some things I chose to have required it. It feels un-Gentoo > for me to have zstd around _just because_, which the profile default > would bring into play. > It's not a "preferred compression tool". "Preferred compression tool" is selected via adding the package to your @world set. The flag is used for enable specific functionality on packages. This function may be limited to being able to optionally compress something. But it could e.g. also be responsible for being able to, say, open a specific file format (and I'm not talking of explicitly .xz compressed files) or a database, or receive proper interoperability elsewhere. The cost of enabling support for a compression library that's already installed by default (because you need it to unpack distfiles) is very little compared to the cost of suddenly discovering that things don't work. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Update on the 23.0 profiles
On Sun, 7 Apr 2024 at 22:09, Michael Orlitzky wrote: > What I am saying is that I want the freedom to not have things > pointlessly enabled on my systems, because similar problems (and worse) > happen all day every day. The less exposure I have, the better. The > liblzma backdoor was timely because it will prevent most people from > telling me I'm being paranoid, but it could have been USE=anything on > any other day. Moving the defaults out of the high-level profiles will > give control back to the user, hence my complaint about it. > I agree, to be honest. The spirit of profiles has always felt like it switches on safe/sane defaults that you'd expect for the name (a desktop plasma profile switches on all the useful desktop USE flags, a basic profile enables the bare minimum for a bootable system, etc), giving an expected functionality in the resulting outcome of a re-merge of world. Outside of this, preferred compression tools, preferred editors etc...should be up to the user, or implied in the profile name if it's going to be switched on in the profile defaults. I don't use zstd myself, I prefer xz or lz4 depending on my purpose. It's on my system because some things I chose to have required it. It feels un-Gentoo for me to have zstd around _just because_, which the profile default would bring into play. -- Ninpo
Re: [gentoo-dev] Update on the 23.0 profiles
On Sun, 2024-04-07 at 16:48 +0200, Michał Górny wrote: > > So, what you're basically saying, is that the best Gentoo response right > now would be to frantically remove LZMA support everywhere? I'm sure > that would be so much better than our response of masking vulnerable > versions and issuing a statement. > Only in the sense that people who park their cars in the bike lane are basically Hitler. This really has nothing to do with the xz thing. The timing was funny, that's all. What I am saying is that I want the freedom to not have things pointlessly enabled on my systems, because similar problems (and worse) happen all day every day. The less exposure I have, the better. The liblzma backdoor was timely because it will prevent most people from telling me I'm being paranoid, but it could have been USE=anything on any other day. Moving the defaults out of the high-level profiles will give control back to the user, hence my complaint about it.
Re: [gentoo-dev] Update on the 23.0 profiles
On Sun, 2024-04-07 at 08:51 -0400, Michael Orlitzky wrote: > On Sun, 2024-04-07 at 14:35 +0200, Andreas K. Huettel wrote: > > > > Uhh, I dont really remember, I think some Chinese-sounding guy asked > > me for it... (j/k) > > It is remarkably bad timing. How it looks: Gentoo's response to the xz > incident is to have me rebuild my entire system with everything that > could possibly be linked to liblzma, linked to liblzma. Even on the > hardened profiles, and with no easy way to prevent it. So, what you're basically saying, is that the best Gentoo response right now would be to frantically remove LZMA support everywhere? I'm sure that would be so much better than our response of masking vulnerable versions and issuing a statement. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Update on the 23.0 profiles
Am Sonntag, 7. April 2024, 14:51:55 CEST schrieb Michael Orlitzky: > On Sun, 2024-04-07 at 14:35 +0200, Andreas K. Huettel wrote: > > > > Uhh, I dont really remember, I think some Chinese-sounding guy asked > > me for it... (j/k) > > It is remarkably bad timing. How it looks: Gentoo's response to the xz > incident is to have me rebuild my entire system with everything that > could possibly be linked to liblzma, linked to liblzma. Even on the > hardened profiles, and with no easy way to prevent it. Well, we're now working with the best-audited compression library ever, I guess. > tl;dr can we turn them back off in the profile? In any scenario where > they are beneficial, there's a better place to put them. Easily doable with lzma, if there is consensus for it. Slightly more complex for zstd since this affects gcc and binutils. Still doable though. -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Update on the 23.0 profiles
On Sun, 2024-04-07 at 14:35 +0200, Andreas K. Huettel wrote: > > Uhh, I dont really remember, I think some Chinese-sounding guy asked > me for it... (j/k) It is remarkably bad timing. How it looks: Gentoo's response to the xz incident is to have me rebuild my entire system with everything that could possibly be linked to liblzma, linked to liblzma. Even on the hardened profiles, and with no easy way to prevent it. > Jokes aside, we did have bz2 in the default useflags for ages, and > at the time this made a lot of sense since xz/lzma and zstd were > steadily becoming the most prevalent compression algorithms. The flags that predate IUSE defaults can of course be forgiven. What is improved by having these flags in every profile, vs only (say) the desktop profiles, or in IUSE defaults? Here's what is made worse: I can't undo it. To maintain the status quo (I was quite happy to not have 100 packages pointlessly linked to liblzma last week), what I want to do is set USE="-lzma -zstd". But if I do that, then it overrides the IUSE defaults for the few packages whose maintainers are doing the right thing, and setting IUSE="+lzma +zstd" where it is important. For example, sys-apps/kmod, where the defaults ensure that your system isn't made unbootable by accident. The other option is for me to perpetually maintain a list of everything that uses lzma/zstd inside my package.use. And to avoid the problem above, I now have to read the ebuilds to see which ones have defaults and which ones don't. This is also not a great way to spend my time. tl;dr can we turn them back off in the profile? In any scenario where they are beneficial, there's a better place to put them.
Re: [gentoo-dev] Update on the 23.0 profiles
Am Sonntag, 7. April 2024, 04:03:01 CEST schrieb Michael Orlitzky: > On Sat, 2024-04-06 at 17:06 +0200, Andreas K. Huettel wrote: > > Hi all, > > > > so here's a small update on the state of the 23.0 profiles: > > > > Why was this silently added to make.defaults for all 23.0 profiles? > > > # This just makes sense nowadays, if only for distfiles... > > USE="lzma zstd" Uhh, I dont really remember, I think some Chinese-sounding guy asked me for it... (j/k) Jokes aside, we did have bz2 in the default useflags for ages, and at the time this made a lot of sense since xz/lzma and zstd were steadily becoming the most prevalent compression algorithms. And for anyone interested in the timeline, this was one of the first additions. commit 99a7cb9e0b1728ca75242ddfee6357dc008bd1cd Author: Andreas K. Hüttel AuthorDate: Sun Nov 13 19:26:40 2022 +0100 Commit: Andreas K. Hüttel CommitDate: Sun Nov 13 19:27:36 2022 +0100 profiles: Set USE="xz zstd" in 23.0 -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Update on the 23.0 profiles
> > Most 17.x profiles have been downgraded to "exp". > > I could imagine there is a reason to downgrade those back to 'exp', > could you elaborate a bit on that? > > Isn't it bit strange that a 'stable' profiles gets downgraded back to > 'exp'? Then again, I am not sure about the implications of this nor > about the rationale behind it. Mostly so the load on the CI does not suddenly double. There's no real reason why we can't keep a few of the profiles stable. Also not much reason to do that though... > > However, I also notice that there is a outstanding PR that reverts that > [1]. Maybe we should introduce a new state 'oldstable' or so? > > - Flow > > > 1: https://github.com/gentoo/gentoo/pull/35871 > -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] Update on the 23.0 profiles
On 06/04/2024 17.06, Andreas K. Huettel wrote: Hi all, so here's a small update on the state of the 23.0 profiles: Thanks for the update and the work on the 23.0 profiles. :) Most 17.x profiles have been downgraded to "exp". I could imagine there is a reason to downgrade those back to 'exp', could you elaborate a bit on that? Isn't it bit strange that a 'stable' profiles gets downgraded back to 'exp'? Then again, I am not sure about the implications of this nor about the rationale behind it. However, I also notice that there is a outstanding PR that reverts that [1]. Maybe we should introduce a new state 'oldstable' or so? - Flow 1: https://github.com/gentoo/gentoo/pull/35871 OpenPGP_0x8CAC2A9678548E35.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Update on the 23.0 profiles
On Sat, 2024-04-06 at 17:06 +0200, Andreas K. Huettel wrote: > Hi all, > > so here's a small update on the state of the 23.0 profiles: > Why was this silently added to make.defaults for all 23.0 profiles? > # This just makes sense nowadays, if only for distfiles... > USE="lzma zstd" It doesn't help with distfiles in any way, wasn't discussed here, doesn't belong there, and creates a mess on systems where they should be disabled. Use per-package defaults if they're important for certain packages.
[gentoo-dev] Update on the 23.0 profiles
Hi all, so here's a small update on the state of the 23.0 profiles: * For all arches, the 23.0 profiles are now marked at the same stability status (mostly for the CI and pkgcheck) as before the 17.x profiles. Most 17.x profiles have been downgraded to "exp". * All stage downloads (with the exception of hppa, where our builder is a tad slow) are now based on 23.0. * For all ISO / other boot media, the specs have been changed as of today, so that the next succeeding build will also be based on 23.0. The new builds need testing, so if you happen to have suitable hardware around... * With that we can nail down the timetable for the next steps: 2024-06-06 (in 2 months): deprecation of the 17.x profiles; binpkg builders for the 17.x profiles are stopped 2025-06-06 (1 year later): removal of the 17.x profiles from profiles.desc -??-?? (when infra has moved :o): removal of the 17.x profile trees in gentoo.git Comments and feedback welcome. I would really prefer to *not* extend the time until deprecation though, since maintenance (and CPU time) goes down as soon as we don't build twice the amount of packages... In general, our installation ISO need in my opinion some review regarding the package set and the detailed build instructions. As long as they work that is not really urgent though. Cheers, Andreas -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, comrel, toolchain, base-system, perl, libreoffice) https://wiki.gentoo.org/wiki/User:Dilfridge signature.asc Description: This is a digitally signed message part.
