It sounds to me like the problem is "how do we best use the available automated test resources?" so I'll answer accordingly. Ignore me if I misunderstood ;-)
We currently have a small list of packages that are gated behind openQA, and insufficient openQA resources to expand this list to all packages. We also have a gating system based on karma, where users provide the QA manually. At least, one hopes. The karma system has some configurability for how much karma is needed, and for how long, etc. What about a merger of those systems, where the openQA results count as a type of user, contributing to the status? Give each package a "QA Priority" that ranges from "full gating" to "five second rule", where the openQA resources and the user work together to prioritize and test packages according to need: * full gating requires all openQA tests to pass AND meet karma requirements, openQA does these first. * partial gating is like above, but either one (openQA or karma) can be sufficient if enough time passes. * reject only allows an openQA failure to reject an update, but the lack of openQA success need not stop it if it has enough karma. (glibc uses this paradigm - a ci/cd failure is an automatic reject, but a ci/cd pass adds nothing to the consensus needed) ... and so on down the list. Each user, including openQA, can "vote" a pass or fail, and the rules say how all those passes and fails result in the update's overall pass or fail. This would allow the openQA resources to prioritize updates that it knows *needs* openQA results, which allocating spare cycles to packages that could benefit from testing but don't require it. It also means that additional openQA resources can be put to use immediately, while still allowing for growth in critical path size, without being wasted on unseen results. _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
