> > > However, it would be great if we could start devising a solution for > having > > "health" reports from the various CI systems. > > This report should report the following kind of information: > > - timestamp of last run > > - timestamp of last vote (a system might start job which then get aborted > > for CI infra problems) > > - % of success vs failures (not sure how important is that one but > provides > > a metric of comparison with upstream jenkins) > > - % of disagreements with jenkins (this might allow us to quickly spot > those > > CI systems which are randomly -1'ing patches) > > > > The percentage metrics might be taken over a 48 hours or 7 days > interval, or > > both. > > Does this idea sound reasonable? > > > This sounds like a very good idea. Now we just need to find someone > with the time to write this. :)
That's exactly what Stackalytics/DriverLog may do! It already collects the latest CI votes and it wouldn't be hard to calculate metrics based on them. Ilya 2014-06-16 18:11 GMT+04:00 Salvatore Orlando <[email protected]>: > > > > On 16 June 2014 15:58, Kyle Mestery <[email protected]> wrote: > >> On Mon, Jun 16, 2014 at 8:52 AM, Salvatore Orlando <[email protected]> >> wrote: >> > I will probably be unable, as usual, to attend today's CI meeting (falls >> > right around my dinner time). >> > I think it's a good idea to starting keeping track of the status of the >> > various CI systems, but I feel the etherpad will not work very well in >> the >> > long term. >> > >> Agreed. The etherpad was a starting point, I'll move this information >> to a wiki page later today. >> >> > However, it would be great if we could start devising a solution for >> having >> > "health" reports from the various CI systems. >> > This report should report the following kind of information: >> > - timestamp of last run >> > - timestamp of last vote (a system might start job which then get >> aborted >> > for CI infra problems) >> > - % of success vs failures (not sure how important is that one but >> provides >> > a metric of comparison with upstream jenkins) >> > - % of disagreements with jenkins (this might allow us to quickly spot >> those >> > CI systems which are randomly -1'ing patches) >> > >> > The percentage metrics might be taken over a 48 hours or 7 days >> interval, or >> > both. >> > Does this idea sound reasonable? >> > >> This sounds like a very good idea. Now we just need to find someone >> with the time to write this. :) >> >> > Also, regarding [1]. I agree that more is always better... but I would >> like >> > a minimum required set of tests to be enforced. >> > Is this something that can be achieved? >> > >> I think the tests that are in there are the minimum tests I'd like to >> see run. I'll clarify the language on the wiki page a bit. >> > > If the intention is to have the minimum set of test "we'd like to see run" > then it's perfect! > I was trying to say we should impose that set as a minimum requirement... > > >> >> > Salvatore >> > >> > [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting >> > >> > >> > >> > >> > On 16 June 2014 07:07, YAMAMOTO Takashi <[email protected]> wrote: >> >> >> >> hi, >> >> >> >> > My initial analysis of Neutron 3rd Party CI is here [1]. This was >> >> > somewhat correlated with information from DriverLog [2], which was >> >> > helpful to put this together. >> >> >> >> i updated the etherpad for ofagent. >> >> currently a single CI system is running tests for both of ofagent and >> ryu. >> >> is it ok? >> >> >> >> YAMAMOTO Takashi >> >> >> >> _______________________________________________ >> >> OpenStack-dev mailing list >> >> [email protected] >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> > >> > >> > _______________________________________________ >> > OpenStack-dev mailing list >> > [email protected] >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> >> _______________________________________________ >> OpenStack-dev mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
