On 6/16/2017 9:46 AM, Eric Harney wrote:
On 06/16/2017 10:21 AM, Sean McGinnis wrote:
I don't think merging tests that are showing failures, then blacklisting
them, is the right approach. And as Eric points out, this isn't
necessarily just a failure with Ceph. There is a legitimate logical
issue with what this particular test is doing.
But in general, to get back to some of the earlier points, I don't think
we should be merging tests with known breakages until those breakages
can be first addressed.
As another example, this was the last round of this, in May:
https://review.openstack.org/#/c/332670/
which is a new tempest test for a Cinder API that is not supported by
all drivers. The Ceph job failed on the tempest patch, correctly, the
test was merged, then the Ceph jobs broke:
https://bugs.launchpad.net/glance/+bug/1687538
https://review.openstack.org/#/c/461625/
This is really not a sustainable model.
And this is the _easy_ case, since Ceph jobs run in OpenStack infra and
are easily visible and trackable. I'm not sure what the impact is on
Cinder third-party CI for other drivers.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
This is generally why we have config options in Tempest to not run tests
that certain backends don't implement, like all of the backup/snapshot
volume tests that the NFS job was failing on forever.
I think it's perfectly valid to have tests in Tempest for things that
not all backends implement as long as they are configurable. It's up to
the various CI jobs to configure Tempest properly for what they support
and then work on reducing the number of things they don't support. We've
been doing that for ages now.
--
Thanks,
Matt
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
