On 27/06/14 07:40 -0400, Sean Dague wrote:
It's clear that lots of projects want 3rd Party CI information on patches. But it's also clear that 6 months into this experiment with a lot of 3rd Party CI systems, the Gerrit UI is really not great for this.A couple of things have fallen out of this. 3rd Party CI bots outnumber Human comments on changes on some projects (Nova / Neutron). That has an impact on the readability of the votes section (on a neutron change the files in the change are rarely above the fold), the readability of the comments. 3rd Party CI systems haven't become all that reliable. They fall into the same problems that Jenkins hits with cloud networking, race bugs in OpenStack, but also new bugs around site configs. It's kind of a testament to how much we've learned about how to keep the machine running that the upstream CI system, even with all it's faults, still trends more reliable than most of the 3rd Party systems. Commenting in Gerrit was to eventually get towards voting in Gerrit. But my experience at this point is reviewers are at CI fatigue and are mostly not paying attention to the votes. Heck, when we're dealing with a bunch of bugs in the gate most reviewers want to ignore the Jenkins votes too, which is why you get the recheck grinding behavior. This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. I'm not signing up for this particular mission, but I wanted to stick it out there to figure out if the idea had merrit, and if it did, if it excited anyone to enough to dive on it.
I agree all those comments cause way to much noise. The most important bit of the comment is the link that takes you to the CI logs so, it may be probably enough to just have 1 link that will take someone to the builds of the 3rd party CI for that specific change. Along the lines of what you just proposed, what if each 3rd Party CI username in the review's list just links to this "builds-list" resources that a developer can go to to check the logs. Would that be too much of a hack for gerrit? I'd like to avoid us to create yet-another-monitoring-resource that we'll have to go to in order to figure out what's going on. FWIW, the above could also be done for our Jenkins. Overall, +1 for the email and idea. Flavio -- @flaper87 Flavio Percoco
pgpwpRJnlxUen.pgp
Description: PGP signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
