On 3/10/2017 3:02 PM, Andrea Frittoli wrote:
We had a couple of sessions related to this topic at the PTG [0][1]. We agreed that we want to still maintain integration tests only in Tempest, which means that API micro versions that have no integration impact can be tested via functional tests.
To be clear, "integration" here means tests that span operations across multiple services, correct? Like a compute test that first creates a port in the networking service and a volume in the block storage service and then uses those to create a server and maybe take a snapshot of it which is then verified was uploaded to the image service.
The non-integration things are self-contained in a single service, like if all you need to do is create an aggregate, show it's details and validate the response, at a particular microversion, we can just do that in nova functional tests, and it's not necessary in Tempest.
It might be worth having a definition of this policy in the Tempest docs so when people ask this question again you can just point at the docs.
In terms of which versions we test in the gate, for nova we always run with min_microversion = None and max_microversion = latest, which means that all tests will be executed. Since micro versions are incremental, and each micro version usually involves no or one test on Tempest side, I think it will be a while before this becomes an issue for the common gate.
We test max_microversion=latest only on master. On the devstack stable branch we cap the max_microversion, e.g.:
https://github.com/openstack-dev/devstack/blob/stable/newton/lib/tempest#L339 -- Thanks, Matt __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
