++ On 08/09/2013 06:07 PM, Dan Prince wrote: > In general I agree with what has been outline in Jim's reply below. Jim, I > have a couple replies inline though :) > > Bob, I think calling what we talked about a "gate" is probably the wrong > word. This is third party testing running as a pre-gate check which we are > trying to make a bit more formal. Can you prevent bad code from landing by > running tests as soon as someone prevents a branch: Yes!. Can you gate > everything and guarantee nothing slips in: No. Is it worth doing? I think so > because at the end of the day preventing bad code from landing is always > good. Much of the SmokeStack improvements we talked about (like isolating > package build failures from torpedo/test failures) is going to happen anyway. > So what we end up with is basically something where we can confidently at > pre-gate time run tests and provide feedback in the form of a -1 in the > verified column without someone like me having to approve it. I understand > you'd like that to be a -2 if the tests fail... but is automatically posting > a -1 good enough for now. > > > ----- Original Message ----- >> From: "James E. Blair" <[email protected]> >> To: "Bob Ball" <[email protected]> >> Cc: [email protected] >> Sent: Friday, August 9, 2013 11:56:28 AM >> Subject: Re: [OpenStack-Infra] Smokestack posting +/-2 votes >> >> Bob Ball <[email protected]> writes: >> >>> What are the thoughts on how an external system can add gating >>> comments? >> >> This is very unlikely to happen. >> >> Multiple gating systems for one project (or set of projects in our case) >> will not work. Zuul does quite a bit of work to try to merge changes as >> quickly as possible, including speculative execution (where it builds >> git trees for several projects that represent the expected repo states >> in the future after many changes merge). It also recognizes updates to >> the current state of changes in review, determining whether they can >> merge based on current review criteria, dependencies, etc. >> >> In other words, Zuul needs a global and authoritative view of what is >> happening. If there is an outside influence that may cause an >> unpredictable change (such as a random -2 showing up), Zuul will not be >> able to make the correct choices. >> >> There is another important issue -- we can not gate on something that is >> not run by the infrastructure team. We have a responsibility to the >> developers of the project to keep this system running. If something >> breaks, any one of several people on the (open; anyone is welcome!) >> infrastructure team needs to be able to fix it. If we started running >> gate tests on an external system and it stopped working, development on >> the entire project would stop, and we would be powerless to fix it. >> >> On a constructive note, much of what smokestack does could be >> incorporated into the current infrastructure. The "devstack" jenkins >> nodes are becoming increasingly inaccurately named, as they are already >> running tests that have nothing to do with devstack. Think of them as >> general purpose single-use jenkins slaves. It would not be too >> difficult to have a jenkins job build packages, and then have further >> jenkins jobs use those packages to stand up a multi-node openstack (if >> you need three nodes, write three jobs that each run on a single-use >> full-access slave). Obviously this would be quite resource intensive, >> but I believe it's possible. > > > Obviously I'm very interested in this idea. Using packages and configuration > management in multi-node environments to test upstream is really the reason > SmokeStack exists. Once upon a time we did have some Jenkins jobs that used > packages and once devstack came around interest just wasn't there for us to > maintain those (in the upstream openstack Jenkins even). If things had gone > differently... well lets just say there may not be a SmokeStack around today. > Perhaps that job was before its time though and at the time the community > seem violently against using packages of any sort in upstream. Two years > later: if there is an open door here I'd like to pursue it... it is a lot of > work though and I certainly think there is a good bit to learn from the > workflow we use in SmokeStack now. In the meantime there is still a good bit > of ground to hold with SmokeStack which isn't easily swapped out. > > So this is encouraging... and we have lots of work to do. :) > > >> >> The problem, as I understand it, is that we can't run Xen nested on any >> of our current cloud providers. If there is no way to do that, then I >> believe we would need a new source of test resources for this. I think >> we talked about this a bit at the summit, but I think in general if >> someone wants to provide new testing resources, they would need to be >> sufficient to support our resource usage (which is large and growing), >> supported by a real ops team (because we aren't one) with something like >> an SLA. We discussed some ideas around bare metal testing at the summit >> here: https://etherpad.openstack.org/havana-bare-metal-testing >> >> Even without gating, the advisory reporting from smokestack is hugely >> valuable. I think any increase in reliability and speed (such as >> automating the failure detection as you were talking about) will be >> perceived by the review community and they will act accordingly, giving >> it significant weight in their reviews. > > Exactly. > >> >> -Jim >> >> _______________________________________________ >> OpenStack-Infra mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> > > _______________________________________________ > OpenStack-Infra mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >
_______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
