Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-06 Thread Jay Pipes
On 07/04/2014 03:57 AM, Luke Gorrie wrote: On 3 July 2014 19:02, Jay Pipes mailto:jaypi...@gmail.com>> wrote: devstack-gate works very well for what it is supposed to do: Yeah, I would actually love to use devstack-gate. I tried that first. There are two problems for me as a user: First

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-04 Thread Luke Gorrie
On 3 July 2014 19:02, Jay Pipes wrote: > devstack-gate works very well for what it is supposed to do: > Yeah, I would actually love to use devstack-gate. I tried that first. There are two problems for me as a user: First I didn't manage to get it up and running reliably in a reasonable time fr

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-03 Thread Kevin Benton
>In short, you need to test every single proposed patch to the system fully and consistently, otherwise there's simply no point in running any tests at all, as you will spend an inordinate amount of time tracking down what broke what. I agree that every patch should be tested. However, since third

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-03 Thread Jay Pipes
On 07/03/2014 08:42 AM, Luke Gorrie wrote: On 3 July 2014 02:44, Michael Still mailto:mi...@stillhq.com>> wrote: The main purpose is to let change reviewers know that a change might be problematic for a piece of code not well tested by the gate Just a thought: A "sampling" approach co

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-03 Thread Jay Pipes
On 07/03/2014 03:49 AM, Luke Gorrie wrote: On 3 July 2014 02:44, Michael Still mailto:mi...@stillhq.com>> wrote: I have seen both. Normally there's a failure, reviewers notice, and then the developer spins trying out fixes by uploading new patch sets. Interesting. Yes, I can see that y

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-03 Thread Luke Gorrie
On 3 July 2014 02:44, Michael Still wrote: > The main purpose is to let change reviewers know that a change might > be problematic for a piece of code not well tested by the gate Just a thought: A "sampling" approach could be a reasonable way to stay responsive under heavy load and still give

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-03 Thread Luke Gorrie
On 3 July 2014 02:44, Michael Still wrote: > I have seen both. Normally there's a failure, reviewers notice, and > then the developer spins trying out fixes by uploading new patch sets. > Interesting. Yes, I can see that you need fast response from CIs to support that scenario. 12-hour edit-comp

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-02 Thread Michael Still
On Thu, Jul 3, 2014 at 4:33 AM, Luke Gorrie wrote: > > On 30 June 2014 21:04, Kevin Benton wrote: >> >> As a maintainer of a small CI system that tends to get backed up during >> milestone rush hours, it would be nice if we were allowed up to 12 hours. >> However, as a developer this seems like t

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-02 Thread Luke Gorrie
On 2 July 2014 20:33, Luke Gorrie wrote: > I'd love to see links to such reviews, if anybody has some? (I've only > seen positive reviews and false-negative reviews from 3rd party CIs so far > in my limited experience.) > I didn't say what I meant: reviews where a 3rd party CI has genuinely obje

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-02 Thread Luke Gorrie
On 30 June 2014 21:04, Kevin Benton wrote: > As a maintainer of a small CI system that tends to get backed up during > milestone rush hours, it would be nice if we were allowed up to 12 hours. > However, as a developer this seems like too long to have to wait for the > results of a patch. > Inter

Re: [openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-07-02 Thread Anita Kuno
On 06/30/2014 03:04 PM, Kevin Benton wrote: > Hello all, > > The subject of 3rd party CI voting responses came up in the 3rd-party IRC > meeting today.[1] We would like to get feedback from the larger dev > community on what acceptable response times are for third party CI systems. > > As a maint

[openstack-dev] [third party] - minimum response time for 3rd party CI responses

2014-06-30 Thread Kevin Benton
Hello all, The subject of 3rd party CI voting responses came up in the 3rd-party IRC meeting today.[1] We would like to get feedback from the larger dev community on what acceptable response times are for third party CI systems. As a maintainer of a small CI system that tends to get backed up dur