Re: [openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
I actually don't, for the reasons Clark brought up. All this does is optimizes for people that don't run unit tests locally, and make all the jobs take longer. Everything except the tempest jobs should be easy to run locally. So this would effectively penalize the people that do the right thing, to advantage people that aren't. -Sean On 12/05/2013 07:17 PM, Michael Still wrote: > I like this idea. > > Michael > > On Thu, Dec 5, 2013 at 4:37 PM, Peter Portante > wrote: >> Has anybody considered changing how check and gate jobs work such that >> the tempest and grenade checks only run once the docs/pep8/unittests >> jobs all succeed? >> >> It seems like they complete so much quicker that folks can fix those >> bugs before having to wait hours for the other jobs. >> >> Thoughts, concerns, problems with that? >> >> Sincerely, -peter >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
On Thu, Dec 5, 2013 at 3:33 PM, Christopher Yeoh wrote: > On Fri, Dec 6, 2013 at 9:32 AM, Clark Boylan wrote: >> >> On Thu, Dec 5, 2013 at 2:37 PM, Peter Portante >> wrote: >> > Has anybody considered changing how check and gate jobs work such that >> > the tempest and grenade checks only run once the docs/pep8/unittests >> > jobs all succeed? >> > >> > It seems like they complete so much quicker that folks can fix those >> > bugs before having to wait hours for the other jobs. >> > >> > Thoughts, concerns, problems with that? >> > >> We actually did this for a while where we ran pep8 first and after >> doing it found that the additional 5-10 minutes that were added to >> passing runs was more painful than either waiting for results or >> checking the Zuul status page. There is the additional problem that we >> occasionally run out of slaves to run specific jobs while being able >> to run others. If we run out of slaves capable of running pep8 jobs we >> sit and wait even longer (though there is work being done to make >> those slaves scale out much better). >> > > Is there any way to start them all off in parallel and then abort the long > running tempest > ones early if the short running pep8 or unittests fail? > I don't think Zuul is capable of doing that today. Even if we could do that it wouldn't really save any resources. The tempest tests are run on single use slaves that are thrown away after being used. We would just throw them away and need to build new slaves in the current and proposed scenarios. The other drawback to this approach is that you cancel the tempest jobs after throwing 6 slaves away because pep8 failed. After pep8 is fixed you find that a tempest test is broken by the change. You have now burned through 12 slaves instead of 6 to get this info. I think it is best to provide reviewers (and change proposers) with as much information upfront as possible. Clark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
I like this idea. Michael On Thu, Dec 5, 2013 at 4:37 PM, Peter Portante wrote: > Has anybody considered changing how check and gate jobs work such that > the tempest and grenade checks only run once the docs/pep8/unittests > jobs all succeed? > > It seems like they complete so much quicker that folks can fix those > bugs before having to wait hours for the other jobs. > > Thoughts, concerns, problems with that? > > Sincerely, -peter > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
On Fri, Dec 6, 2013 at 9:15 AM, Sergey Lukjanov wrote: > Why developers need to wait for other jobs? Zuul shows current progress > well. > > They don't need to wait, but it would free up resources for other jobs that might pass to get tested earlier. > > On Friday, December 6, 2013, Peter Portante wrote: > >> Has anybody considered changing how check and gate jobs work such that >> the tempest and grenade checks only run once the docs/pep8/unittests >> jobs all succeed? >> >> It seems like they complete so much quicker that folks can fix those >> bugs before having to wait hours for the other jobs. >> >> Thoughts, concerns, problems with that? >> >> Sincerely, -peter >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > -- > Sincerely yours, > Sergey Lukjanov > Savanna Technical Lead > Mirantis Inc. > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
On Fri, Dec 6, 2013 at 9:32 AM, Clark Boylan wrote: > On Thu, Dec 5, 2013 at 2:37 PM, Peter Portante > wrote: > > Has anybody considered changing how check and gate jobs work such that > > the tempest and grenade checks only run once the docs/pep8/unittests > > jobs all succeed? > > > > It seems like they complete so much quicker that folks can fix those > > bugs before having to wait hours for the other jobs. > > > > Thoughts, concerns, problems with that? > > > We actually did this for a while where we ran pep8 first and after > doing it found that the additional 5-10 minutes that were added to > passing runs was more painful than either waiting for results or > checking the Zuul status page. There is the additional problem that we > occasionally run out of slaves to run specific jobs while being able > to run others. If we run out of slaves capable of running pep8 jobs we > sit and wait even longer (though there is work being done to make > those slaves scale out much better). > > Is there any way to start them all off in parallel and then abort the long running tempest ones early if the short running pep8 or unittests fail? > The other thing worth mentioning is that we should all be running > `tox` before pushing code :) I don't always do it, but getting > everyone into that habit should reduce the vast number of patchsets > with simple failures. > > +1! Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
On Thu, Dec 5, 2013 at 2:37 PM, Peter Portante wrote: > Has anybody considered changing how check and gate jobs work such that > the tempest and grenade checks only run once the docs/pep8/unittests > jobs all succeed? > > It seems like they complete so much quicker that folks can fix those > bugs before having to wait hours for the other jobs. > > Thoughts, concerns, problems with that? > We actually did this for a while where we ran pep8 first and after doing it found that the additional 5-10 minutes that were added to passing runs was more painful than either waiting for results or checking the Zuul status page. There is the additional problem that we occasionally run out of slaves to run specific jobs while being able to run others. If we run out of slaves capable of running pep8 jobs we sit and wait even longer (though there is work being done to make those slaves scale out much better). If we do want to try this again and make the non tempest jobs run first we would need to update Zuul. Zuul's layout file is currently only capable of expressing that one test run before others and not some set of tests. At one point there was talk of adding this but believe interest declined once we decided to stop doing this altogether. The other thing worth mentioning is that we should all be running `tox` before pushing code :) I don't always do it, but getting everyone into that habit should reduce the vast number of patchsets with simple failures. Clark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
Why developers need to wait for other jobs? Zuul shows current progress well. On Friday, December 6, 2013, Peter Portante wrote: > Has anybody considered changing how check and gate jobs work such that > the tempest and grenade checks only run once the docs/pep8/unittests > jobs all succeed? > > It seems like they complete so much quicker that folks can fix those > bugs before having to wait hours for the other jobs. > > Thoughts, concerns, problems with that? > > Sincerely, -peter > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Gate / Check jobs fail fast mode - don't start tempest until docs/pep8/unittests pass?
Has anybody considered changing how check and gate jobs work such that the tempest and grenade checks only run once the docs/pep8/unittests jobs all succeed? It seems like they complete so much quicker that folks can fix those bugs before having to wait hours for the other jobs. Thoughts, concerns, problems with that? Sincerely, -peter ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
