Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-05 Thread James E. Blair
Kashyap Chamarthy writes: > If it reduces nondeterministic spam for the CI Infra, and makes us > achieve the task at hand, sure. [/me need to educate himself a > bit more on the Zuul pipeline infrastructure.] > > Worth filing this (and your 'idle pipeline' thought below) in the Zuul > tracker he

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-03 Thread Kashyap Chamarthy
On Fri, Jul 01, 2016 at 02:35:34PM +, Jeremy Stanley wrote: > On 2016-07-01 15:39:10 +0200 (+0200), Kashyap Chamarthy wrote: > > [Snip description of some nice debugging.] > > > > > I'd really love it if there was > > > > > > 1. the ability to request checking of just specific jobs eg > > >

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Daniel P. Berrange
On Fri, Jul 01, 2016 at 02:13:46PM +, Jeremy Stanley wrote: > Have you considered just writing a throwaway devstack-gate change > which overrides the gate_hook to run that one suspect Tempest test, > say, a thousand times in a loop? Would be far more efficient if you > don't need to be concerne

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Daniel P. Berrange
On Fri, Jul 01, 2016 at 02:35:34PM +, Jeremy Stanley wrote: > On 2016-07-01 15:39:10 +0200 (+0200), Kashyap Chamarthy wrote: > > [Snip description of some nice debugging.] > > > > > I'd really love it if there was > > > > > > 1. the ability to request checking of just specific jobs eg > > >

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Timofei Durakov
This option is already available for live-migration job, as it's hooks are under nova project. On Fri, Jul 1, 2016 at 5:13 PM, Jeremy Stanley wrote: > Have you considered just writing a throwaway devstack-gate change > which overrides the gate_hook to run that one suspect Tempest test, > say, a

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Jeremy Stanley
On 2016-07-01 15:39:10 +0200 (+0200), Kashyap Chamarthy wrote: > [Snip description of some nice debugging.] > > > I'd really love it if there was > > > > 1. the ability to request checking of just specific jobs eg > > > > "recheck gate-tempest-dsvm-multinode-full" > > Yes, this would rea

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Jeremy Stanley
Have you considered just writing a throwaway devstack-gate change which overrides the gate_hook to run that one suspect Tempest test, say, a thousand times in a loop? Would be far more efficient if you don't need to be concerned with all the environment setup/teardown overhead. -- Jeremy Stanley

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Kashyap Chamarthy
On Thu, Jun 30, 2016 at 02:44:36PM +0100, Daniel P. Berrange wrote: [Snip description of some nice debugging.] > I'd really love it if there was > > 1. the ability to request checking of just specific jobs eg > > "recheck gate-tempest-dsvm-multinode-full" Yes, this would really be desir

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Timofei Durakov
Hi, talking about live-migration job. If you are OK to re-run tempest tests multiple times on single environment you could wrap this line: https://github.com/openstack/nova/blob/master/nova/tests/live_migration/hooks/run_tests.sh#L30 in for-cycle. Timofey. On Fri, Jul 1, 2016 at 10:31 AM, Spence

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-07-01 Thread Spencer Krum
This is some neat work. I like it. We have a graph of nodepool nodes in use in grafana[1]. Looking at a 30 day window, its pretty easy to tell that the weekends are low activity. Running big tests like this on a Saturday would be great. Of course, there won't be much infra support on that day. W

Re: [openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-06-30 Thread Matt Riedemann
On 6/30/2016 8:44 AM, Daniel P. Berrange wrote: A bunch of people in Nova and upstream QEMU teams are trying to investigate a long standing bug in live migration[1]. Unfortuntely the bug is rather non-deterministic - eg on the multinode-live-migration tempest job it has hit 4 times in 7 days, whi

[openstack-dev] [nova][infra][ci] bulk repeating a test job on a single review in parallel ?

2016-06-30 Thread Daniel P. Berrange
A bunch of people in Nova and upstream QEMU teams are trying to investigate a long standing bug in live migration[1]. Unfortuntely the bug is rather non-deterministic - eg on the multinode-live-migration tempest job it has hit 4 times in 7 days, while on multinode-full tempest job it has hit ~70 ti