On 06/22/15 at 04:23pm, Matt Riedemann wrote:
The check-tempest-dsvm-cells job has been in nova's check queue since
January as non-voting and has been stable for a couple of weeks now,
so before it's regressed melwitt proposed a change to making it
voting and gating on nova changes [1].
I raised a concern in that change that the tempest-dsvm-cells job is
not in the check queue for tempest or devstack changes, so if a
change is merged in tempest/devstack which breaks the cells job, it
will block nova changes from merging.
mtreinish noted that tempest already has around 30 jobs running
against it right now in the check queue, so he'd prefer that another
one isn't added since the nova API shouldn't be different in the case
of cells, but we know there are quirks. That can be seen from the
massive regex of excluded tests for the tempest-dvsm-cells job [2].
If we could turn that regex list into tempest configurations, I think
that would make it possible to not have to run tempest changes
through the cells job in the check queue but also feel reasonably
confident that changes to tempest that use the config options
properly won't break the cells job (and block nova in the gate).
This is actually something we should do regardless of voting or not
on nova since new tests get added which might not fall in the regex
and break the cells job. So I'm going to propose some changes so
that the regex will be moved to devstack-gate (regex exodus (tm)) and
we'll work on whittling down the regex there (and run those d-g
changes against the tempest-dsvm-cells job in the experimental
queue).
The question for the nova team is, shall we make the
tempest-dsvm-cells job voting on nova changes knowing that the gate
can be broken with a change to tempest that isn't caught in the
regex? In my opinion I think we should make it voting so we don't
regress cells with changes to nova that go unnoticed with the
non-voting job today. Cells v2 is a nova priority for Liberty so we
don't want setbacks now that it's stable.
I agree, though I'm pretty biased.
If a change does land in tempest which breaks the cells job and
blocks nova, we (1) fix it or (2) modify the regex so it's excluded
until fixed as has been done up to this point.
During our efforts last cycle to reduce the number of failing tests it
seemed that we had regressions based on Tempest changes 2 times. Once
around security group logic being added to tests and again when the
networking setup changed during tests that created an instance.
The security group change led to exclusions and the networking change we
were able to fix with a devstack change, and a Tempest flag iirc.
I think we should probably mull this over in the ML and then vote on
it in this week's nova meeting.
[1] https://review.openstack.org/#/c/190894/
[2]
http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
