On 22 June 2015 at 23:03, Matt Riedemann <[email protected]> wrote: > > > On 6/22/2015 4:38 PM, Matt Riedemann wrote: >> >> >> >> On 6/22/2015 4:32 PM, Russell Bryant wrote: >>> >>> On 06/22/2015 05:23 PM, Matt Riedemann wrote: >>>> >>>> The check-tempest-dsvm-cells job has been in nova's check queue since >>>> January as non-voting and has been stable for a couple of weeks now, so >>>> before it's regressed melwitt proposed a change to making it voting and >>>> gating on nova changes [1]. >>>> >>>> I raised a concern in that change that the tempest-dsvm-cells job is not >>>> in the check queue for tempest or devstack changes, so if a change is >>>> merged in tempest/devstack which breaks the cells job, it will block >>>> nova changes from merging. >>>> >>>> mtreinish noted that tempest already has around 30 jobs running against >>>> it right now in the check queue, so he'd prefer that another one isn't >>>> added since the nova API shouldn't be different in the case of cells, >>>> but we know there are quirks. That can be seen from the massive regex >>>> of excluded tests for the tempest-dvsm-cells job [2].
That sounds like a good call. For the record, cells v2 should fix this major issue, which is awesome. >>>> If we could turn that regex list into tempest configurations, I think >>>> that would make it possible to not have to run tempest changes through >>>> the cells job in the check queue but also feel reasonably confident that >>>> changes to tempest that use the config options properly won't break the >>>> cells job (and block nova in the gate). >>>> >>>> This is actually something we should do regardless of voting or not on >>>> nova since new tests get added which might not fall in the regex and >>>> break the cells job. So I'm going to propose some changes so that the >>>> regex will be moved to devstack-gate (regex exodus (tm)) and we'll work >>>> on whittling down the regex there (and run those d-g changes against the >>>> tempest-dsvm-cells job in the experimental queue). >>>> >>>> The question for the nova team is, shall we make the tempest-dsvm-cells >>>> job voting on nova changes knowing that the gate can be broken with a >>>> change to tempest that isn't caught in the regex? In my opinion I think >>>> we should make it voting so we don't regress cells with changes to nova >>>> that go unnoticed with the non-voting job today. Cells v2 is a nova >>>> priority for Liberty so we don't want setbacks now that it's stable. I would be tempted to risk it, given the large gain, but I am a little biased too. But with us controlling the regex, that seems a much easier call to say yes. >>>> If a change does land in tempest which breaks the cells job and blocks >>>> nova, we (1) fix it or (2) modify the regex so it's excluded until fixed >>>> as has been done up to this point. >>>> >>>> I think we should probably mull this over in the ML and then vote on it >>>> in this week's nova meeting. >>>> >>>> [1] https://review.openstack.org/#/c/190894/ >>>> [2] >>>> >>>> http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004 >>>> >>>> >>>> >>> >>> Regarding your "regex exodus", I recently added something for this. In >>> another project, I'm setting the regex in a file I keep in the code repo >>> instead of project-config. >>> >>> support for DEVSTACK_GATE_SETTINGS in devstack-gate: >>> https://review.openstack.org/190321 >>> >>> usage in a job definition: https://review.openstack.org/190325 >>> >>> a DEVSTACK_GATE_SETTINGS file that sets DEVSTACK_GATE_TEMPEST_REGEX: >>> https://review.openstack.org/186894 >>> >>> It all seems to be working for me, except I still need to tweak my regex >>> to get the job passing, but at least I can do that without updating >>> project-config now. >>> >> >> Awesome, that is way cleaner. I'll go that route instead, thanks! >> > > Here is the change that moves the regex into nova: > > https://review.openstack.org/#/c/194411/ This seems like a good best of both worlds. Awesome stuff. Thanks, John __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
