It's clear that lots of projects want 3rd Party CI information on patches. But it's also clear that 6 months into this experiment with a lot of 3rd Party CI systems, the Gerrit UI is really not great for this.
A couple of things have fallen out of this. 3rd Party CI bots outnumber Human comments on changes on some projects (Nova / Neutron). That has an impact on the readability of the votes section (on a neutron change the files in the change are rarely above the fold), the readability of the comments. 3rd Party CI systems haven't become all that reliable. They fall into the same problems that Jenkins hits with cloud networking, race bugs in OpenStack, but also new bugs around site configs. It's kind of a testament to how much we've learned about how to keep the machine running that the upstream CI system, even with all it's faults, still trends more reliable than most of the 3rd Party systems. Commenting in Gerrit was to eventually get towards voting in Gerrit. But my experience at this point is reviewers are at CI fatigue and are mostly not paying attention to the votes. Heck, when we're dealing with a bunch of bugs in the gate most reviewers want to ignore the Jenkins votes too, which is why you get the recheck grinding behavior. This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. I'm not signing up for this particular mission, but I wanted to stick it out there to figure out if the idea had merrit, and if it did, if it excited anyone to enough to dive on it. -Sean -- Sean Dague http://dague.net
signature.asc
Description: OpenPGP digital signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev