On Wed, 2022-10-19 at 07:13 +0200, Martin Pitt wrote: > Hello Adam, > > Adam Williamson [2022-10-18 10:37 -0700]: > > that would define a critical path for Server. That would mean updates > > containing any of those packages or their dependencies would be > > 'critical path' updates, meaning they have higher requirements to be > > pushed stable (+2 karma or 14 days waiting, as opposed to +1 karma or 7 > > days waiting for regular updates) > > What is the reason for increasing the waiting period, what would it achieve?
It's essentially a byproduct. Please see the devel@ discussion I linked. We do not currently have any other good way to gate updates on the openQA tests. We can invent one but it's substantially more work than just adding things to the critpath. I'm 75% done with the "do it by adding things to critpath" approach because nobody objected to that idea on devel@ , so it would've been great to have this feedback in *that* discussion :( The issues you raise are all reasonable, but kind of not something new: we've never had an official way to override the critical path requirements if we think a package has good enough CI, so if you want to call that a problem, it's been a problem forever. We can leave cockpit out of the definition if you like, but that would mean it won't be gated by the distribution-wide gating policy. As I mentioned in the devel@ discussion, an alternative is to configure gating in the package repos - see https://gating-greenwave.readthedocs.io/en/latest/policies.html#remoterule-configure-additional-policies - but this requires the package maintainer(s) to maintain the gating policy. It also has the disadvantage that we don't resolve dependencies like we do for the critpath. If we put cockpit in the critpath definition, then cockpit *and all its dependencies* are added to the critpath, and thus will have the tests run and be gated on them. So if some dependency of cockpit breaks cockpit, we'll find out, and the update will be gated. If we just have cockpit in the openQA scheduler allowlist and add a gating.yaml to the cockpit repo, we do not find out if some dependency of cockpit breaks cockpit before it happens (unless that dep happens to be in the critical path some other way). CCing devel@ since this brings up points wider than the Server context. The rest of Martin's mail is below for reference. > - It sets the wrong motivation, as it indiscriminately penalizes all of these > updates. You can not land faster (or at all) by adding proper CI, and > instead can blame "the community" for not having spotted a regression once > the update gets in eventually. > > - It will easily lead to packages not being updated at all any more. We > release every 14 days. Updates in the current stable are usually getting > karma feedback rather easily, but we also support the previous stable, or > the pending fork (like Fedora 37 a few weeks ago), and they usually don't > get *any* karma. We can work around that by giving karma ourself from our > team after gating tests get green [1], but that is just fighting the > process > and gives zero new information. > > - It is a step backwards. CI helps to increase velocity of the distribution, > this step decreases it. This is walking away from CI and towards more human > interaction. > > - It tries to address the problem at the wrong place. What we are sorely > missing in Fedora is reverse dependency testing, i.e. making sure that a > package update does not break *other* packages. New selinux-policy, > systemd, > or kernel versions break the distro all the time, but they wouldn't be > covered by the "critical path". Human manual testing of e.g. freeipa or > cockpit *could* spot this, but (1) it takes much more work than just > running > the automated tests that we already have, and (2) it barks at the wrong > tree. > > As a workaround, we run Cockpit's integration tests (which are kinda sorta > "Fedora/RHEL server QA") against fedora-37 + updates-testing twice a week, and > occasionally we are able to -1 a package update in bodhi that breaks things. > But 7 days are more than enough for that. Our problem has been that at least > in > the past, e.g. selinux-policy updates got like +5 "WFM" karma and went into > -updates in *hours*, way too fast to catch it with that method. A human "WFM" > is simply not good enough for these low-level package updates, and it also > doesn't address the "test consumers of that package" problem. > > So in summary, I don't believe this is helpful at all, sorry. > > > and openQA would test them automatically and they would be blocked from > > stable if the tests fail. > > That is a good move, of course. Any failing gating test should block updates > (for every package), otherwise we'll never fix them. (Developers can waive, of > course, but without that you wouldn't even know) > > Martin > > [1] And I'd automate that, hello https://github.com/osbuild/fedora-bot > _______________________________________________ > server mailing list -- ser...@lists.fedoraproject.org > To unsubscribe send an email to server-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/ser...@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
