Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-22 Thread Matthew Treinish
On Thu, Aug 22, 2013 at 04:36:18PM -0500, Ben Nemec wrote: > On 2013-08-22 16:28, Matthew Treinish wrote: > >Another option that I've thought about is making the testr-full > >jobs voting on > >the check queue. This way it will raise parallel failures to > >peoples attention > >but not increase the

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-22 Thread Ben Nemec
On 2013-08-22 16:28, Matthew Treinish wrote: Another option that I've thought about is making the testr-full jobs voting on the check queue. This way it will raise parallel failures to peoples attention but not increase the number of gate resets. The only tradeoff here is that it will make the

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-22 Thread Matthew Treinish
On Wed, Aug 14, 2013 at 06:13:18PM -0300, Thierry Carrez wrote: > Matthew Treinish wrote: > > Also, if anyone has any input on what threshold they feel is good enough > > for this I'd welcome any input on that. For example, do we want to ensure > > a >= 1:1 match for job success? Or would somethin

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-16 Thread Matthew Treinish
On Fri, Aug 16, 2013 at 01:03:57PM -0500, Ben Nemec wrote: > >>>Getting this in before the H3 rush would be very helpful. When we made > >>>the switch with Nova's unittests we fixed as many of the test bugs > >>>that we could find, merged the change to switch the test runner, then > >>>treated all

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-16 Thread Ben Nemec
On 2013-08-14 16:10, Matthew Treinish wrote: On Wed, Aug 14, 2013 at 11:05:35AM -0500, Ben Nemec wrote: On 2013-08-13 16:39, Clark Boylan wrote: >On Tue, Aug 13, 2013 at 1:25 PM, Matthew Treinish > wrote: >> >>Hi everyone, >> >>So for the past month or so I've been working on getting tempest >>t

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-15 Thread Joshua Harlow
+1 good idea I was wondering this myself. Sent from my really tiny device... On Aug 14, 2013, at 1:07 PM, "Alexius Ludeman" mailto:l...@lexinator.com>> wrote: I kind of high jacked another thread with my testr problems, but I want to reiterate it directly on this one as they are my pain point

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-14 Thread Thierry Carrez
Matthew Treinish wrote: > Also, if anyone has any input on what threshold they feel is good enough > for this I'd welcome any input on that. For example, do we want to ensure > a >= 1:1 match for job success? Or would something like 90% as stable as the > serial job be good enough considering the

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-14 Thread Matthew Treinish
On Wed, Aug 14, 2013 at 11:05:35AM -0500, Ben Nemec wrote: > On 2013-08-13 16:39, Clark Boylan wrote: > >On Tue, Aug 13, 2013 at 1:25 PM, Matthew Treinish > > wrote: > >> > >>Hi everyone, > >> > >>So for the past month or so I've been working on getting tempest > >>to work stably > >>with testr in

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-14 Thread Robert Collins
On 15 August 2013 08:02, Alexius Ludeman wrote: > I kind of high jacked another thread with my testr problems, but I want to > reiterate it directly on this one as they are my pain points I had from the > transition. > > testr does not appear to support --exclusion[1] or --stop[2]. 1 is tricky to

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-14 Thread Dan Prince
- Original Message - > From: "Matthew Treinish" > To: openstack-dev@lists.openstack.org > Sent: Tuesday, August 13, 2013 4:25:14 PM > Subject: [openstack-dev] Migrating to testr parallel in tempest > > > Hi everyone, > > So for the past mon

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-14 Thread Alexius Ludeman
I kind of high jacked another thread with my testr problems, but I want to reiterate it directly on this one as they are my pain points I had from the transition. testr does not appear to support --exclusion[1] or --stop[2]. I have a work around for --exclusion, by: testr list-tests | egrep -v r

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-14 Thread Ben Nemec
On 2013-08-13 16:39, Clark Boylan wrote: On Tue, Aug 13, 2013 at 1:25 PM, Matthew Treinish wrote: Hi everyone, So for the past month or so I've been working on getting tempest to work stably with testr in parallel. As part of this you may have noticed the testr-full jobs that get run on the

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-13 Thread Clark Boylan
On Tue, Aug 13, 2013 at 1:25 PM, Matthew Treinish wrote: > > Hi everyone, > > So for the past month or so I've been working on getting tempest to work > stably > with testr in parallel. As part of this you may have noticed the testr-full > jobs that get run on the zuul check queue. I was using th

Re: [openstack-dev] Migrating to testr parallel in tempest

2013-08-13 Thread Jay Pipes
On 08/13/2013 04:25 PM, Matthew Treinish wrote: Hi everyone, So for the past month or so I've been working on getting tempest to work stably with testr in parallel. As part of this you may have noticed the testr-full jobs that get run on the zuul check queue. I was using that job to debug some

[openstack-dev] Migrating to testr parallel in tempest

2013-08-13 Thread Matthew Treinish
Hi everyone, So for the past month or so I've been working on getting tempest to work stably with testr in parallel. As part of this you may have noticed the testr-full jobs that get run on the zuul check queue. I was using that job to debug some of the more obvious race conditions and stability