Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
> My machines should actually do some useful stuff, like running my Nagios and > a bunch of nightly builds (CMake, libarchive, things like that). For that, > I'd like to have the actual system to work. Given the amount of breakage I > find when doing stabilizations I suspect this is not going to happen. Just to make that slightly more clear: the keywordings and stabilizations happen in their own chroots. But the main install of the system is what I like to have really working. Eike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
Sorry about a long delay responding, I ended up being offline until the end of last week and I've had quite a lot of catching up. Anyway, let me begin by addressing a sentiment expressed independently in several responses and which could be summarised as "just come and help". A laudable idea in theory - except as a project run entirely by unpaid volunteers, we can neither hire more people nor demand that developers work on more things than they already do. It might sound harsh but if working in particle physics (which like most public-sector research suffers from chronic shortage of manpower comparing to the amount of things to do, and which is nowadays based primarily on large-scale collaborations whose leadership has only minimal authority over individual participants) has taught me anything, it's that it is better to do a good job at two things than a mediocre one at ten. Moving on to specific comments: On 18/10/2021 01:50, Sam James wrote: > - Most failures found via arch testing _aren't_ arch-specific, but they serve as a useful quality check. That is, > usually, we're not held back by some odd e.g. SIGBUS that nobody knows how to fix. Possibly true (I've got no evidence to make a definite statement either way) - but there is a point in testing, or in pretty much any technical activity, when the amount of work required to polish something further begins to strongly outweigh the benefits. Moreover, the above doesn't really sound to me like a case in defence of stabilisation on exotic arches; quite the opposite in fact. > - Encourage developers to run test suites on their packages. This is a modern part of Gentoo development > and isn't optional if a package has a functioning test suite which isn't hell to get running - i.e. you should really > _try_. People who do not do this yet should be taken behind the chemicals shed and sho... I mean, be very much ashamed of themselves. Not sure what that has got to do with arch testing though, given what kind of hardware most of us do Gentoo development on. > - We drop any large suites of packages at least to ~arch where they're problematic. In addition to the dependency-creep problem already mentioned by Michał, I am not convinced that arbitrarily declaring some package or other not worthy of stable status on arch X would make the user experience on this arch better than downgrading the whole arch to ~X. Furthermore, I am pretty sure arch testers would then have to keep track of which packages must not be stabilised where - meaning more work. On 18/10/2021 01:25, Thomas Deutschmann wrote: Could you please elaborate what you are expecting from this change? I.e. will this solve any problem (please name it)? Will it allow us to move forward where we are blocked at the moment (please name it)? One part of this has already been mentioned by the others, i.e. all too often low activity on these arches ends up delaying overall progress of things such security issues for ALL Gentoo users. Another is that IMHO there are way too few people active in these arch teams to keep up with the work load - even including sam's activity pretty much all over the place, which at this rate I fear will result in him burning out soon, things are far from great. On 15/10/2021 22:40, Rolf Eike Beer wrote: > My machines should actually do some useful stuff, like running my Nagios and a > bunch of nightly builds (CMake, libarchive, things like that). For that, I'd > like to have the actual system to work. Given the amount of breakage I find > when doing stabilizations I suspect this is not going to happen. Maybe, maybe not... If my experience with RISC-V keywording is anything to go by, a lot of breakage comes from unexpected interactions due to throwing everything but a kitchen sink on a single system - which having to deal with stabilisation makes more likely, especially on an arch which does not see many new keywording requests (on riscv, which is still quite active in this respect, I simply run all keywording tests with --oneshot and regularly distclean the system). On 14/10/2021 18:10, Michał Górny wrote: > While we're discussing it, maybe we should start by defining a clear > criteria for platform support tiers? Like: what are the requirements > for a platform to maintain stable keywords? Then the decisions could > look less arbitrary, and people would have a clear way of knowing what > they need to do if they wish the platform to continue having stable > keywords. Not a bad idea but I wonder how much effort we might want to throw at this, especially given we're not Red Hat or SUSE. -- MS
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
Am Montag, 18. Oktober 2021, 03:08:52 CEST schrieb John Helmert III: > On Mon, Oct 18, 2021 at 02:25:47AM +0200, Thomas Deutschmann wrote: > > On 2021-10-14 15:40, Marek Szuba wrote: > > > WDYT? > > > > Could you please elaborate what you are expecting from this change? > > > > I.e. will this solve any problem (please name it)? Will it allow us to > > move forward where we are blocked at the moment (please name it)? > > A security bug, for example, is currently blocked for almost a month > waiting for hppa stabilization [1], and this isn't the first time > we've had to wait for a "slower" arch on a security bug. I had a system outage of my machine that I run the stabilizations on a few weeks back, and then the bug actually slipped through. Which is no excuse, but… -get access to hake (the hppa dev machine) and help doing it. This could fail if the machine is trying to build a stage or so, which takes multiple days… -if you feel that something is missing come over to #gentoo-hppa and ping us about that I'm running libgcrypt tests right now, should hopefully be done until tomorrow. Eike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On 2021-10-18 19:07, Michał Górny wrote: Security team arbitrarily deciding that an architecture is unsupported while otherwise it's supported in Gentoo doesn't change anything. Sure, you can close bugs and pretend that a problem doesn't exist... except that you can't if you can't remove the old version because of keywords. You won't see me defending the idea of allowing stable architectures without security support (this was before I joined Gentoo and I never liked it). But this is what we have for more than 10 years now. However, this was never an arbitrary decision. It was something between arch teams and security project but in the end it was always the arch team's decision because they are the ones doing the work (like "Sorry, we cannot keep up..." -"Well, that's bad but now we have to deal with that"). Anyway, I think we are losing focus on topic. I am still waiting for Marecki to answer the motivation behind this. And to quote you: Sure, you can close bugs and pretend that a problem doesn't exist Sadly, you can say the same for dropping stable keywords (and I think we are not that far away if I understand [1] correctly), not? That's why I asked for the motivation behind this and what people are expecting to become better/what problem will be solved after that change. We haven't yet talked about the risk of broken deptrees because some tooling will ignore non-stable architectures by default. [1] https://archives.gentoo.org/gentoo-dev/message/a3c7a6cb7596a5ff9102e4d819a52d9c -- Regards, Thomas Deutschmann / Gentoo Linux Developer fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On Mon, 2021-10-18 at 17:09 +0200, Thomas Deutschmann wrote: > On 2021-10-18 03:08, John Helmert III wrote: > > A security bug, for example, is currently blocked for almost a month > > waiting for hppa stabilization [1], and this isn't the first time > > we've had to wait for a "slower" arch on a security bug. > > Excuse me? How is this possible? > > We have that Gentoo Vulnerability Treatment Policy and HPPA isn't listed > in supported architectures. That problem was resolved in 2018 [1]. > Security team arbitrarily deciding that an architecture is unsupported while otherwise it's supported in Gentoo doesn't change anything. Sure, you can close bugs and pretend that a problem doesn't exist... except that you can't if you can't remove the old version because of keywords. -- Best regards, Michał Górny
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On 2021-10-18 03:08, John Helmert III wrote: A security bug, for example, is currently blocked for almost a month waiting for hppa stabilization [1], and this isn't the first time we've had to wait for a "slower" arch on a security bug. Excuse me? How is this possible? We have that Gentoo Vulnerability Treatment Policy and HPPA isn't listed in supported architectures. That problem was resolved in 2018 [1]. [1] https://archives.gentoo.org/gentoo-announce/message/196e45cde209d1ed25bd42e679739cf5 -- Regards, Thomas Deutschmann / Gentoo Linux Developer fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On Mon, Oct 18, 2021 at 02:25:47AM +0200, Thomas Deutschmann wrote: > On 2021-10-14 15:40, Marek Szuba wrote: > > WDYT? > > Could you please elaborate what you are expecting from this change? > > I.e. will this solve any problem (please name it)? Will it allow us to > move forward where we are blocked at the moment (please name it)? A security bug, for example, is currently blocked for almost a month waiting for hppa stabilization [1], and this isn't the first time we've had to wait for a "slower" arch on a security bug. [1] https://bugs.gentoo.org/795480 signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
> On 18 Oct 2021, at 01:50, Sam James wrote: > > > >> On 14 Oct 2021, at 14:40, Marek Szuba wrote: >> >> Dear everyone, >> >> Following some private discussions, I feel quite strongly now that it would >> both considerably improve certain processes and make our use of limited >> manpower more efficient were we to further reduce the number of stable >> arches in Gentoo Linux. Specifically, I propose to drop >> - hppa, >> - ppc, >> - sparc, >> - x86 >> to ~arch-only status. > > [snip] > > My suggested actions: > > [snip] > > - We drop any large suites of packages at least to ~arch where they're > problematic. A good place > to start would probably be scientific stuff which isn't a test dependency (or > likely to become one) > in future of e.g. the Python stack. There's quite a few niche sci > applications stable on e.g. x86 > which probably don't have a need to be. > One more while it's in my head: - Try break the assumption in developers' heads (mine too!) that we should stable x86 while we're stabling amd64 and so on, if it's not already stable there. Try reduce growing the x86 stable base unless somebody wants it. >> [snip] > > best, > sam > signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
> On 14 Oct 2021, at 14:40, Marek Szuba wrote: > > Dear everyone, > > Following some private discussions, I feel quite strongly now that it would > both considerably improve certain processes and make our use of limited > manpower more efficient were we to further reduce the number of stable arches > in Gentoo Linux. Specifically, I propose to drop > - hppa, > - ppc, > - sparc, > - x86 > to ~arch-only status. I'm not sure we should go down this route. Dakon's email covers a lot of the reasons, but I'll try to add to it my own rationale too: - Most failures found via arch testing _aren't_ arch-specific, but they serve as a useful quality check. That is, usually, we're not held back by some odd e.g. SIGBUS that nobody knows how to fix. - We're not really helping users by making such a change. Any problems which prevent stabilisation still exist. We're just reducing the quality of the Gentoo experience for users on these arches. My suggested actions: - As referenced below, make more developers aware they're welcome to have access to our various exotic hardware! - Encourage developers to run test suites on their packages. This is a modern part of Gentoo development and isn't optional if a package has a functioning test suite which isn't hell to get running - i.e. you should really _try_. - Further, encouraging tatt/pkg-testing-tools like Dakon suggested before pushing new ebuilds. A wiki page I've started might prove helpful too: https://wiki.gentoo.org/wiki/User:Sam/Useful_scripts#Testing. - We drop any large suites of packages at least to ~arch where they're problematic. A good place to start would probably be scientific stuff which isn't a test dependency (or likely to become one) in future of e.g. the Python stack. There's quite a few niche sci applications stable on e.g. x86 which probably don't have a need to be. The gist being, I think we can focus our efforts (and try educate + encourage others to help) without completely shutting the door here. I'm quite happy helping with these arches right now (although hppa is problematic due to the speed of our current hardware, we are wondering if we can get some other kit) as long as we all continue to chip in. But more help is very welcome and desired. What would probably help more than anything else right now is dropping stable keywords for irrelevant packages (not wasting time on some stablereqs where nobody is probably using that $application on $arch) and having a tool to easily report bugs so I don't waste time copying/pasting logs. (slyfox actually mentioned his desire for such a tool in his farewell post: https://trofi.github.io/posts/226-farewell-gentoo-dev.html). Now, addressing the rest of the email: > Note that this does NOT mean we intend to drop support for those arches > altogether. > > There are IMHO several good reasons for this: > - most of the arches from this list are quite dated and either aren't really > developed upstream any more or got superseded by newer ones (for the record, > it's been 18 years since the first amd64 CPUs came out) But users of this hardware can only really get by on Gentoo without super-super-frequent updates. Not all versions added to ~arch to get stabilised and also once stabled are less likely to have e.g. build failures so less wasted time. > - we have got very few people actually supporting these arches, and in case > of hppa there is also the hardware bottleneck. Subsequently, stabilisation > requests often take a long time to resolve I think a better way of tackling this is to make developers aware they can even access a lot of this hardware! I don't think many developers realise they're welcome to have access to our various arch testing machines -- they're not just for a select few. We want more help! > [snip] best, sam signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On 2021-10-14 15:40, Marek Szuba wrote: WDYT? Could you please elaborate what you are expecting from this change? I.e. will this solve any problem (please name it)? Will it allow us to move forward where we are blocked at the moment (please name it)? I am really curious what you are going to expect to change by this keyword change and why you want to change current status at all (motivation). -- Regards, Thomas Deutschmann / Gentoo Linux Developer fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On Thu, Oct 14, 2021 at 03:40:02PM +0200, Marek Szuba wrote: > Dear everyone, > > Following some private discussions, I feel quite strongly now that it > would both considerably improve certain processes and make our use of > limited manpower more efficient were we to further reduce the number of > stable arches in Gentoo Linux. Specifically, I propose to drop > - hppa, > - ppc, > - sparc, > - x86 > to ~arch-only status. > > Note that this does NOT mean we intend to drop support for those arches > altogether. > > There are IMHO several good reasons for this: > - most of the arches from this list are quite dated and either aren't > really developed upstream any more or got superseded by newer ones (for > the record, it's been 18 years since the first amd64 CPUs came out) > - we have got very few people actually supporting these arches, and in > case of hppa there is also the hardware bottleneck. Subsequently, > stabilisation requests often take a long time to resolve > - feedback we receive, e.g. by Bugzilla, suggests that Gentoo on at > least some of these arches have got very, very few users > - last but by no means least, my personal experience from the last > several years suggests that running ~arch is reasonably trouble-free > these days > > WDYT? For the record, I'm fine with this. x86 being on the list sort of caught my attention, but it does seem to fall into the superceeded category, so it should be fine. Even though running ~arch may be mostly trouble-free, this isn't really relevant to the discussion imo. If you run ~arch, you should be prepared for possible breakage at any time and be able to recover from it. William signature.asc Description: PGP signature
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On Fri, 2021-10-15 at 23:40 +0200, Rolf Eike Beer wrote: > We have already removed many stable packages from hppa, just to reduce the > amount of work. If sparc really becomes a problem I suspect that dropping > most > of the multimedia or whatever stuff there could also reduce the amount of > work > needed. For the record, I'm not quite sure if dropping large sets of packages to ~arch is actually a good idea. While it's fine for some leaf packages, the Python packages have proven to grow new dependencies quite fast. In the end, dropping stable keywords may result in only having to reintroduce them soon afterwards, with lots of extra work. -- Best regards, Michał Górny
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
Am Donnerstag, 14. Oktober 2021, 15:40:02 CEST schrieb Marek Szuba: > Dear everyone, > > Following some private discussions, I feel quite strongly now that it > would both considerably improve certain processes and make our use of > limited manpower more efficient were we to further reduce the number of > stable arches in Gentoo Linux. Specifically, I propose to drop > - hppa, > - sparc, > to ~arch-only status. > > There are IMHO several good reasons for this: > - we have got very few people actually supporting these arches, and in > case of hppa there is also the hardware bottleneck. Subsequently, > stabilisation requests often take a long time to resolve > - last but by no means least, my personal experience from the last > several years suggests that running ~arch is reasonably trouble-free > these days > > WDYT? Reducing to what I have a personal opinion about. For quite a while I have been more or less the arch testing team for hppa and sparc, the latter reduced since ago and sam meanwhile utilize even faster machines to do much of the the sparc work (yay!). Running these machines is a bumpy ride. Things break quite regularly, besides the arch-independent breakage like missing dependencies or similar things, which I also find quite regularly. My machines should actually do some useful stuff, like running my Nagios and a bunch of nightly builds (CMake, libarchive, things like that). For that, I'd like to have the actual system to work. Given the amount of breakage I find when doing stabilizations I suspect this is not going to happen. My fear is that I'll be rebuilding stuff because there is an upgrade, and then back because there was an update, and in between I have to find out what actually went wrong. That's close to what I'm doing now, with the difference that the main system meanwhile can do it's work because it usually is unaffected, and I can decide to ignore the problem for one or another day until I'm bored enough to fight the breakage again. So from my limited PoV this would likely even increase the work that I have to do, or the pressure to do it in time to fix the system up to a point where it works. We have already removed many stable packages from hppa, just to reduce the amount of work. If sparc really becomes a problem I suspect that dropping most of the multimedia or whatever stuff there could also reduce the amount of work needed. Another note: these machines are quite slow, especially the hppa ones, when compared with a modern PC with SSD and tons of RAM. I would really _really_ welcome it if people could just run tatt for stabilizations on amd64 in a regularly empty chroot. It finds tons of stuff with missing dependencies or useflags (USE=static is always good for trouble) that I would otherwise run into on the slow machines. If you fix only half of the things before it hits the minor arches, which is not limited to the above list, it will greatly reduce the pain for everyone with a vintage fetish. So, do what I can't stop you from doing, but at least for me dropping hppa will likely not reduce any pain, and if sparc really is a problem than dropping some packages will likely do the same thing also. Oh, and maybe mark some for fonts and stuff ALLARCHES ;) Eike (aka Dakon) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On 14.10.2021 16:40, Marek Szuba wrote: Dear everyone, Following some private discussions, I feel quite strongly now that it would both considerably improve certain processes and make our use of limited manpower more efficient were we to further reduce the number of stable arches in Gentoo Linux. Specifically, I propose to drop - hppa, - ppc, - sparc, - x86 to ~arch-only status. [..] WDYT? There arches are mostly exotic these days, so marking them unstable is only going to reflect it more (which is right, respective arch teams might still support the stable profiles to make sure we are fine with the deptree),
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On giovedì 14 ottobre 2021 15:40:02 CEST Marek Szuba wrote: > WDYT? I agree for arches that have exotic hardware but I'd keep x86 since testing can be done on amd64 via 32bit chroot. On the other hand I'm pretty sure we have few x86 users so, sooner or later, x86 will go into ~arch as well. Agostino
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On 14.10.2021 20.10, Michał Górny wrote: > On Thu, 2021-10-14 at 15:40 +0200, Marek Szuba wrote: >> Dear everyone, >> >> Following some private discussions, I feel quite strongly now that it >> would both considerably improve certain processes and make our use of >> limited manpower more efficient were we to further reduce the number of >> stable arches in Gentoo Linux. Specifically, I propose to drop >> - hppa, >> - ppc, >> - sparc, >> - x86 >> to ~arch-only status. >> Yes please. Still confused why people by default push KEYWORDS="~amd64 ~x86", but I guess they're the most compatible with each other. > > On one hand, I fully realize that these platforms are a hassle (hppa > and x86 especially). On the other hand, I wouldn't want to basically go > tell Dakon "sorry, you're doing a good job but we've arbitrarily decided > it's not worth your effort". Isn't this just strengthening the point; there's one guy behind all work ;) > > While we're discussing it, maybe we should start by defining a clear > criteria for platform support tiers? Like: what are the requirements > for a platform to maintain stable keywords? Then the decisions could > look less arbitrary, and people would have a clear way of knowing what > they need to do if they wish the platform to continue having stable > keywords. > ++ -- juippis OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On Thu, 2021-10-14 at 15:40 +0200, Marek Szuba wrote: > Dear everyone, > > Following some private discussions, I feel quite strongly now that it > would both considerably improve certain processes and make our use of > limited manpower more efficient were we to further reduce the number of > stable arches in Gentoo Linux. Specifically, I propose to drop > - hppa, > - ppc, > - sparc, > - x86 > to ~arch-only status. > I don't have a strong opinion either way. On one hand, I fully realize that these platforms are a hassle (hppa and x86 especially). On the other hand, I wouldn't want to basically go tell Dakon "sorry, you're doing a good job but we've arbitrarily decided it's not worth your effort". While we're discussing it, maybe we should start by defining a clear criteria for platform support tiers? Like: what are the requirements for a platform to maintain stable keywords? Then the decisions could look less arbitrary, and people would have a clear way of knowing what they need to do if they wish the platform to continue having stable keywords. -- Best regards, Michał Górny
Re: [gentoo-dev] [RFC] Moving more architectures to ~arch only
On 2021.10.14 14:40, Marek Szuba wrote: > Dear everyone, > > Following some private discussions, I feel quite strongly now that it > would both considerably improve certain processes and make our use of > limited manpower more efficient were we to further reduce the number > of > stable arches in Gentoo Linux. Specifically, I propose to drop > - hppa, > - ppc, > - sparc, > - x86 > to ~arch-only status. > > Note that this does NOT mean we intend to drop support for those > arches > altogether. > > There are IMHO several good reasons for this: > - most of the arches from this list are quite dated and either > aren't > really developed upstream any more or got superseded by newer ones > (for > the record, it's been 18 years since the first amd64 CPUs came out) > - we have got very few people actually supporting these arches, and > in > case of hppa there is also the hardware bottleneck. Subsequently, > stabilisation requests often take a long time to resolve > - feedback we receive, e.g. by Bugzilla, suggests that Gentoo on at > least some of these arches have got very, very few users > - last but by no means least, my personal experience from the last > several years suggests that running ~arch is reasonably trouble-free > these days > > WDYT? > > -- > Marecki > > > Only x86 raised an eyebrow here but only one, and not very far. It has to come sooner or later, so if not now, then when? Datapoint: On the forums, x86 installs are either done by mistake or by users who know what they are doing on a 32 bit SoC, The first set of users will be helped, the second set know what they are doing. In case its not clear after all that waffle, I'll go with the flow. -- Regards, Roy Bamford (Neddyseagoon) a member of elections gentoo-ops forum-mods arm64 pgpu7gIwFkHei.pgp Description: PGP signature
