Excerpts from John Griffith's message of 2013-08-27 09:42:37 -0700: > On Tue, Aug 27, 2013 at 10:26 AM, Alex Gaynor <[email protected]> wrote: > > > I wonder if there's any sort of automation we can apply to this, for > > example having known rechecks have "signatures" and if a failure matches > > the signature it auto applies the recheck. > > > > I think we kinda already have that, the recheck list and the bug ID > assigned to it no? Automatically scanning said list and doing the recheck > automatically seems like overkill in my opinion. At some point human > though/interaction is required and I don't think it's too much to ask a > technical contributor to simply LOOK at the output from the test runs > against their patches and help out a bit. At the very least if you didn't > test your patch yourself and waited for Jenkins to tell you it's broken I > would hope that a submitter would at least be motivated to fix their own > issue that they introduced. >
It is worth thinking about though, because "ask a technical contributor to simply LOOK" is a lot more expensive than "let a script confirm the failure and tack it onto the list for rechecks". Ubuntu has something like this going for all of their users and it is pretty impressive. Apport and/or whoopsie see crashes and look at the backtraces/coredumps/etc and then (with user permission) submit a signature to the backend. It is then analyzed and the result is this: http://errors.ubuntu.com/ Known false positives are shipped along side packages so that they do not produce noise, and known points of pain for debugging are eased by including logs and other things in bug reports when users are running the dev release. This results in a much better metric for what bugs to address first. IIRC update-manager also checks in with a URL that is informed partially by this data about whether or not to update packages, so if there is a high fail rate early on, the server side will basically signal update-manager "don't update right now". I'd love to see our CI system enhanced to do all of the pattern matching to group failures by common patterns, and then when a technical contributor looks at these groups they have tons of data points to _fix_ the problem rather than just spending their precious time identifying it. The point of the recheck system, IMHO, isn't to make running rechecks easier, it is to find and fix bugs. _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
