Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

John, It's a good option. Let's try it! Also, we can try to find/implement something like [13] for ostestr. [13] https://github.com/mahmoudimus/nose-timer Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Mon, Mar 7, 2016 at 7:16 PM, John Griffith wrote: > > >

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 2016-03-07 23:54:49 +0800 (+0800), Duncan Thomas wrote: > Complexity can be tricky to spot by hand, and expecting reviewers to get it > right all of the time is not a reasonable expectation. > > My ideal would be something that processes the commit and the jenkins logs, > extracts the timing

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On Mon, Mar 7, 2016 at 8:57 AM, Knight, Clinton wrote: > > > On 3/7/16, 10:45 AM, "Eric Harney" wrote: > > >On 03/06/2016 09:35 PM, John Griffith wrote: > >> On Sat, Mar 5, 2016 at 4:27 PM, Jay S. Bryant > >> >>>

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 3/7/16, 10:45 AM, "Eric Harney" wrote: >On 03/06/2016 09:35 PM, John Griffith wrote: >> On Sat, Mar 5, 2016 at 4:27 PM, Jay S. Bryant >>>> wrote: >> >>> Ivan, >>> >>> I agree that our testing needs improvement. Thanks for starting this

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 7 March 2016 at 23:45, Eric Harney wrote: > > > I'm not really sure that writing a "hacking" check for this is a > worthwhile investment. (It's not a hacking check really, but something > more like what you're describing, but that's beside the point.) > > We should just

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 03/06/2016 09:35 PM, John Griffith wrote: > On Sat, Mar 5, 2016 at 4:27 PM, Jay S. Bryant > wrote: > >> Ivan, >> >> I agree that our testing needs improvement. Thanks for starting this >> thread. >> >> With regards to adding a hacking check for tests that run

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On Sat, Mar 5, 2016 at 4:27 PM, Jay S. Bryant wrote: > Ivan, > > I agree that our testing needs improvement. Thanks for starting this > thread. > > With regards to adding a hacking check for tests that run too long ... are > you thinking that we would have a

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

Ivan, I agree that our testing needs improvement. Thanks for starting this thread. With regards to adding a hacking check for tests that run too long ... are you thinking that we would have a timer that checks or long running jobs or something that checks for long sleeps in the testing

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 02/03, Ivan Kolodyazhny wrote: > I'll try to implement such scenario and step-by-step guideline soon. > That would be fantastic!! Thank you very much Looking forward to it. :-) Cheers, Gorka. > Regards, > Ivan Kolodyazhny, > http://blog.e0ne.info/ > > On Wed, Mar 2, 2016 at 5:16 PM, Eric

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

Hi, I will try to be short. - Voting unit test coverage job is ready, and you can just use it as is from rally source code: you need this file https://github.com/openstack/rally/blob/master/tests/ci/cover.sh and this change in tox:

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 03/02/2016 04:11 PM, Gorka Eguileor wrote: > On 02/03, Ivan Kolodyazhny wrote: >> Eric, >> >> There are Gorka's patches [10] to remove API Races >> >> >> [10] >> https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified >> > I looked at Rally a long

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

I'll try to implement such scenario and step-by-step guideline soon. Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Wed, Mar 2, 2016 at 5:16 PM, Eric Harney wrote: > On 03/02/2016 10:07 AM, Ivan Kolodyazhny wrote: > > Eric, > > > > For now, we test Cinder API with

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 03/02/2016 10:07 AM, Ivan Kolodyazhny wrote: > Eric, > > For now, we test Cinder API with some concurrency only with Rally, so, IMO, > it's reasonable get more scenarios for API races fixes. > > It's not a hard task to implement new scenarios, they are pretty simple: > [11] and [12] > Sure,

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 02/03, Ivan Kolodyazhny wrote: > Eric, > > There are Gorka's patches [10] to remove API Races > > > [10] > https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified > I looked at Rally a long time ago so apologies if I'm totally off base here,

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

Eric, For now, we test Cinder API with some concurrency only with Rally, so, IMO, it's reasonable get more scenarios for API races fixes. It's not a hard task to implement new scenarios, they are pretty simple: [11] and [12] [11]

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

:* OpenStack Development Mailing List (not for usage questions) < > openstack-dev@lists.openstack.org> > *Subject:* Re: [openstack-dev] [cinder] Proposal: changes to our current > testing process > > > > Eric, > > > > There are Gorka's patches [10] to remove API Races

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org> Subject: Re: [openstack-dev] [cinder] Proposal: changes to our current testing process Eric, There are Gorka's patches [10] to remove API Races [10] https://review.openstack.org/#/q/project:ope

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 03/02/2016 09:36 AM, Ivan Kolodyazhny wrote: > Eric, > > There are Gorka's patches [10] to remove API Races > > > [10] > https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified > > Regards, > Ivan Kolodyazhny, > http://blog.e0ne.info/ > So

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

Eric, There are Gorka's patches [10] to remove API Races [10] https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Wed, Mar 2, 2016 at 4:27 PM, Eric Harney wrote: > On

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

On 03/02/2016 06:25 AM, Ivan Kolodyazhny wrote: > Hi Team, > > Here are my thoughts and proposals how to make Cinder testing process > better. I won't cover "3rd party CI's" topic here. I will share my opinion > about current and feature jobs. > > > Unit-tests > >- Long-running tests. I

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

That would be great. The boot from volume test remains one of the top failure scenarios, both in the gate, and for end users. It would be really good to get a handle on why and get that shored up. -Sean On 03/02/2016 08:18 AM, D'Angelo, Scott wrote: > +1 to making the testing process

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

+1 to making the testing process better. It has been discussed that services could/should consider devoting some or all of a release cycle to stability and/or quality. I propose the Cinder team makes improving and fixing the tests and test process a priority for the Newton cycle. Scott D'Angelo

Re: [openstack-dev] [cinder] Proposal: changes to our current testing process

Hi Ivan! On Wed, Mar 2, 2016 at 1:25 PM, Ivan Kolodyazhny wrote: > Hi Team, > > Here are my thoughts and proposals how to make Cinder testing process > better. I won't cover "3rd party CI's" topic here. I will share my opinion > about current and feature jobs. > > > Unit-tests >