Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-07 Thread Kyle Mestery
On Thu, Jul 3, 2014 at 6:12 AM, Salvatore Orlando sorla...@nicira.com wrote: Apologies for quoting again the top post of the thread. Comments inline (mostly thinking aloud) Salvatore On 30 June 2014 22:22, Jay Pipes jaypi...@gmail.com wrote: Hi Stackers, Some recent ML threads [1] and a

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-07 Thread Joe Gordon
-party-ci][neutron] What is Success exactly? On 06/30/2014 09:13 PM, Jay Pipes wrote: On 06/30/2014 07:08 PM, Anita Kuno wrote: On 06/30/2014 04:22 PM, Jay Pipes wrote: Hi Stackers, Some recent ML threads [1] and a hot IRC meeting today [2] brought up some legitimate questions

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-06 Thread Jay Pipes
On 07/03/2014 02:41 PM, Fawad Khaliq wrote: On Thu, Jul 3, 2014 at 10:27 AM, Kevin Benton blak...@gmail.com mailto:blak...@gmail.com wrote: This allows the viewer to see categories of reviews based upon their divergence from OpenStack's Jenkins results. I think evaluating

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Sullivan, Jon Paul
-Original Message- From: Anita Kuno [mailto:ante...@anteaya.info] Sent: 01 July 2014 14:42 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly? On 06/30/2014 09:13 PM, Jay Pipes wrote: On 06/30/2014 07:08 PM, Anita

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Salvatore Orlando
Apologies for quoting again the top post of the thread. Comments inline (mostly thinking aloud) Salvatore On 30 June 2014 22:22, Jay Pipes jaypi...@gmail.com wrote: Hi Stackers, Some recent ML threads [1] and a hot IRC meeting today [2] brought up some legitimate questions around how a

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Anita Kuno
On 07/03/2014 06:22 AM, Sullivan, Jon Paul wrote: -Original Message- From: Anita Kuno [mailto:ante...@anteaya.info] Sent: 01 July 2014 14:42 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly? On 06/30/2014 09:13 PM

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Anita Kuno
On 07/03/2014 07:12 AM, Salvatore Orlando wrote: Apologies for quoting again the top post of the thread. Comments inline (mostly thinking aloud) Salvatore On 30 June 2014 22:22, Jay Pipes jaypi...@gmail.com wrote: Hi Stackers, Some recent ML threads [1] and a hot IRC meeting today

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Sullivan, Jon Paul
-Original Message- From: Anita Kuno [mailto:ante...@anteaya.info] Sent: 03 July 2014 13:53 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly? On 07/03/2014 06:22 AM, Sullivan, Jon Paul wrote: -Original

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Anita Kuno
On 07/03/2014 09:52 AM, Sullivan, Jon Paul wrote: -Original Message- From: Anita Kuno [mailto:ante...@anteaya.info] Sent: 03 July 2014 13:53 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly? On 07/03/2014 06:22 AM

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Anita Kuno
On 07/03/2014 10:31 AM, Sullivan, Jon Paul wrote: -Original Message- From: Anita Kuno [mailto:ante...@anteaya.info] Sent: 03 July 2014 15:06 To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly? I guess you missed

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Anita Kuno
On 07/03/2014 01:27 PM, Kevin Benton wrote: This allows the viewer to see categories of reviews based upon their divergence from OpenStack's Jenkins results. I think evaluating divergence from Jenkins might be a metric worth consideration. I think the only thing this really reflects though

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Kevin Benton
Maybe we can require period checks against the head of the master branch (which should always pass) and build statistics based on the results of that. Otherwise it seems like we have to take a CI system's word for it that a particular patch indeed broke that system. -- Kevin Benton On Thu, Jul

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-07-03 Thread Kevin Benton
Yes, I can propose a spec for that. It probably won't be until Monday. Is that okay? On Thu, Jul 3, 2014 at 11:42 AM, Anita Kuno ante...@anteaya.info wrote: On 07/03/2014 02:33 PM, Kevin Benton wrote: Maybe we can require period checks against the head of the master branch (which should

[openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-06-30 Thread Jay Pipes
Hi Stackers, Some recent ML threads [1] and a hot IRC meeting today [2] brought up some legitimate questions around how a newly-proposed Stackalytics report page for Neutron External CI systems [2] represented the results of an external CI system as successful or not. First, I want to say

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-06-30 Thread Franck Yelles
Hi Jay, Couple of points. I support the fact that we need to define what is success is. I believe that the metrics that should be used are Voted +1 and Skipped. But to certain valid case, I would say that the Voted -1 is really mostly a metric of bad health of a CI. Most of the -1 are due to

Re: [openstack-dev] [third-party-ci][neutron] What is Success exactly?

2014-06-30 Thread Jay Pipes
On 06/30/2014 07:08 PM, Anita Kuno wrote: On 06/30/2014 04:22 PM, Jay Pipes wrote: Hi Stackers, Some recent ML threads [1] and a hot IRC meeting today [2] brought up some legitimate questions around how a newly-proposed Stackalytics report page for Neutron External CI systems [2] represented