One extra argument for waiving the requirement: this package does most of its work on first-boot of an instance; we have to manually build new GCE images specifically to be able to test it. Users who just enable -proposed and upgrade (even if they then reboot) will not be performing meaningful testing of the package.
On Wed, Aug 08, 2018 at 04:10:26PM +0200, Lukasz Zemczak wrote: > Hello, > > In my humble opinion, for this particular case I would be inclined > toward lifting the aging requirement. Yes, it's not a completely > straightforward thing, even though I usually approved of releasing > early of this package before after validation was performed. From my > point of view a strong enough rationale has been provided for that. > Balint's original e-mail states: > "No one else is expected to test those packages in the remaining time > of default required minimal age." > I think in case of this package this is true. As already mentioned by > Steve, the gce-compute-image-packages binaries are *only* used on > cloud instances. This basically means almost absolutely no one will be > using the versions from -proposed, meaning the 7 days are wasted on > idle waiting. My rationale below: > > To me the aging period is always a safety buffer for when a package > enters -proposed, during which it can get additional testing (or > 'dogfooding') by both interested and unrelated parties. In my eyes > though, usually what this means in practice is to give users that are > affected by the selected bug or are generally interested in the given > package time to pick it up from -proposed and give it a spin > themselves. Formally for us to consider an SRU to be verified requires > only one tested giving a +1 on the package, after providing test > results. The aging period (besides the obvious case of letting it run > on systems of users that always run with -proposed enabled for testing > purposes) serves the purpose of giving additional time for those > 'other than the verifying person' affected users to actually test the > package and report any regressions that the original verifier could > have missed. For this, there have to be such users - and those users > need to be able to perform the validation steps as noted in the test > case. > > In the case of gce-compute-image-packages, considering the nature of > the package itself, the test case, the SRU exception and the release > process of the actual package itself, I think we can be pretty > confident that the only people aware of SRU updates of those packages > are: the uploader, the SRU team and GCE upstream. These people are > probably also the only ones that can fully test the packages (since > those are usually new upstream releases), not counting cases where > there are specific isolated LP bugs involved in the upload (for those > I would say aging can still make some sense). > > All those small arguments, in my opinion, sum up into a decent enough > rationale for ignoring the aging period in this case. > > Cheers, > > > On 7 August 2018 at 22:27, Robie Basak <[email protected]> wrote: > > Hi Steve, > > > > Summary: I still think we should have an actual justification for > > waiving the aging requirement. I still don't think that we've actually > > been given a justification in this case. Give me one and I'll > > reconsider, but IMHO without any justification, we should either change > > policy to drop it wholesale, or maintain our current policy of having > > it. > > > > The rest of my response is mostly about why we have the general policy > > in the first place, rather than this specific case. > > > > On Tue, Aug 07, 2018 at 11:41:31AM -0700, Steve Langasek wrote: > >> That is certainly the purpose of the aging period, but I think this > >> overstates its value in the SRU process overall. You say you believe you > >> have seen this happen in practice, to block a buggy SRU before it reached > >> -updates. But broadly speaking: > >> > >> - dist-upgrading (or trying to use update-manager) with -proposed is a bad > >> idea for a production machine, because packages in -proposed are known > >> not to be consistently installable throughout the SRU lifecycle > >> - therefore, running with -proposed enabled is not something that is > >> broadly socialized (nor should be) as a way for users to contribute to > >> the quality of Ubuntu > >> - therefore, random user testing coverage of SRUs is sparse, whether we > >> wait 7 days or 30. > > > > Nevertheless, I think that wider random user testing coverage of > > packages in proposed should be something we aim to achieve. This is > > something I've mentioned before[1]. I'm confident that we have enough of > > a community willing to live on the edge that they would be willing to do > > this: if we make it easy for them and publish this as "a way to help > > out". Regardless of whether or not we can do this today, removing the > > aging period moves us in the opposite direction. > > > >> There is *some* value to an aging period, which is why I have never > >> advocated dropping the default aging period entirely. But the value is > >> small, not straightforwardly quantifiable, and doesn't outweigh the cost in > >> human developer time of multiple round-trips with the SRU team; so whenever > >> anyone asks to waive the waiting period, and I don't have /specific/ > >> concerns about the SRU, my answer will be yes. > > > > I disagree. Since aging is something we consider worthy enough to have > > in our policy by default, I think that waiving the aging period in a > > specific case should require at least _some_ justification. "I want to > > do it" or "It isn't needed" is, IMHO, not in itself a valid reason. > > Accepting those statements would in practice mean that aging doesn't > > exist in the SRU process for all those who disagree with it; in that > > case, why have an aging period in our policy at all? If we're going to > > accept that, we might as well remove the requirement entirely. > > > > On the other hand, give me an actual justification and I'd be happy to > > consider it. I just don't feel like we've been given a justification at > > all in this case, apart from "I don't like it". > > > > As for requiring an extra round-trip with the SRU team, I don't think > > this applies to SRUs in -proposed for which all the bugs are green and > > the SRU process properly followed in the SRU report by the time the > > aging period expires. In practice, I see these all being released to > > -updates promptly and without further interaction. > > > > Most delays are caused by Ubuntu developers not following documented SRU > > procedure, and it is that which causes extra round trips. That can be > > fixed by uploading Ubuntu developers easily enough; I don't think it's > > necessary to compromise quality (since we both agree that there is > > *some* value to an aging period) to do it. > > > >> I note that there's no language at all in the wiki page at > >> https://wiki.ubuntu.com/StableReleaseUpdates which talks about waiving the > >> waiting period. I think that is an oversight; I recall seeing written > >> language at one point about this, but I can't find it now (including in the > >> history of this wiki page). Other long-time members of the SRU team can > >> check me on this, but my understanding of the actual policy is that any > >> member of the SRU team has the authority to waive the 7-day period any time > >> they think it is appropriate to do so. I think we should fix the wiki > >> documentation to match. > > > > My understanding matches yours. I have always understood any single SRU > > team member to have the delegated authority to waive the aging > > requirement on a case-by-case basis. What I think we're discussing here > > is when the SRU team believes they should. > > > >> > In this case, I think user-based regression alert opportunity still > >> > applies. Presumably the reason for this package to exist in the Ubuntu > >> > archive is that there are users. Removing the aging requirement removes > >> > the opportunity for those users to tell us about problems. > >> > >> The package in question, gce-compute-image-packages, is only installed in > >> cloud instances. I believe this means that the organic regression test > >> coverage by users is LESS than it would be for another package, because the > >> test coverage we do get tends to be from individually-deployed machines (a > >> personal desktop/laptop or a one-off server), which is less likely to > >> happen > >> in a cloud where deployments are intended to be done at scale.[1] > > > > Users managing cloud deployments done at scale following best practice > > already have the ability to test such deployments automatically, > > including packages in proposed, at virtually no extra maintenance cost. > > > > That we currently don't recommend that users enable -proposed wholesale > > is a use case that, in my view, we should fix (whether by "fixing" > > -proposed or by providing some other mechanism). I see the current > > inability as a "bug", and I don't think the existence of this bug should > > be used as a justification for removing the aging period; that would > > make it impossible to solve the problem. > > > >> And in cases where we already have an SRU exception, which dictates that > >> there must be strong upstream QA for the package before it is pushed to the > >> SRU queue, I think it's all the more reasonable to omit the default aging > >> period as part of that SRU exception. > > > > Strong upstream QA certainly does shift the balance of regression risk > > in the "safer" direction. However, packages with SRU exceptions > > generally end up with riskier stable uploads, such as feature additions. > > So that's the counterpoint that could shift the balance back and cancel > > this out. Going in the other direction, it might well be a reasonable > > mitigation for the extra regression risk in permitting wholesale > > upstream feature releases into Ubuntu stable releases to require a > > _longer_ aging period. I'm not suggesting we do this, but I hope this > > example helps illustrate my point. > > > > Robie > > > > [1] > > https://lists.ubuntu.com/archives/ubuntu-release/2016-October/003948.html > > > > -- > > Ubuntu-release mailing list > > [email protected] > > Modify settings or unsubscribe at: > > https://lists.ubuntu.com/mailman/listinfo/ubuntu-release > > > > > > -- > Łukasz 'sil2100' Zemczak > Foundations Team > [email protected] > www.canonical.com > > -- > Ubuntu-release mailing list > [email protected] > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-release -- Ubuntu-release mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-release
