Elsewhere, on 26 April 2011 08:13, Martin Pitt <[email protected]> wrote: > Hello Martin, > > Martin Pool [2011-04-21 18:48 +1000]: >> On the whole I'm not sure [manually verifying all SRU bugs in bzr] was a >> good use of time: I think we tend >> to have problems not so much when we fail to fix the bug but rather >> when we break something else in doing so. Manually testing the bug is >> probably not going to catch that, and is probably redundant with >> testing we did and the original reporter did when it was merged >> upstream. > > I agree. Indeed I'm much more concerned about regression testing, > which is of course hard to describe in general for all SRUs. So we > usually resort to minimal patches and have reporters test the actual > package in a real environment. Strictly speaking this is not true > regression testing, but the next best thing to what we can reasonably > achieve. > > In the bzr case however, we can do proper regression testing, because > it already has a huge test suite. Indeed the MRE says: > > conditions: test suite running during package build from Ubuntu > 11.04 on; SRU verification should run test suite in installed sytem > > From my POV doing the latter and then reporting back to one of the > bugs with "test suite showed no regressions for the package in > foo-proposed" would suffice here. I think that was the original intent > when we discussed the MRE. > > We handle things in a similar way for other MREs like postgresql or > Firefox: We run standard tests (manual or automatic suites) for > signing them off.
I would like to ask the tech board to approve an edit <https://wiki.ubuntu.com/StableReleaseUpdates/MicroReleaseExceptions> to say something along these lines, in the interests of people getting changes without wasted effort or inconsistent handling. Thanks Martin -- technical-board mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/technical-board
