On 7/5/2017 4:16 AM, Ferenc Horváth wrote:
I looked at how novaclient's functional test suite works.
As far as I understand there is a functional environment in tox
in which all the test cases are executed in a common dsvm job.
Obviously, the test cases are using the CLI to communicate
with devstack.
My current idea is that we could break down the problem to
at least three work items. So, from a (very) high-level view, we
will need the following things to implement the aforementioned
nova-integration environment:
1. A new tox environment, which will run test from a directory
(which might be /nova/tests/functional/integration for example).
If it's a new tox env, I'd put them in nova/tests/integration, and leave
the 'functional' subdirectory to the "tox -e functional" tests.
2. A new dsvm job, which will execute only the new integration
environment.
Yes.
3. Some way to communicate with devstack. In case of tempest
or novaclient this layer is already implemented, but my guess is
that we will have to implement new clients/fixtures in nova.
It probably depends on what needs to be tested. If you're going through
the REST API, then a simple REST client using keystoneauth1 is probably
good enough. We can use os-client-config for auth credentials like the
novaclient functional job uses.
I think the most critical is #3, but if we can discuss this and the idea
is mature enough then I'd like to start with proposing tests for evacuate.
Note that for evacuate testing, we'd need a multi-node job (minimum of 2
nodes), and we'd also need to run the tests serially since evacuate
requires that the nova-compute service is 'down' before you can evacuate
instances from it.
The novaclient functional job runs it's tests serially, but is single node.
If we wanted to test evacuate cheaply without an entirely new test / job
infrastructure, we could add a multi-node novaclient functional job
which runs serially and tests evacuate for us - we have all of the CLIs
we'd need to do that and the majority of the infra and test framework
plumbing is already in place to do this.
It seems a bit odd to build that into novaclient functional testing
rather than in something that runs against nova changes though, i.e. we
can't verify fixes to evacuate in nova itself, and if we break something
in evacuate because we aren't testing it in nova, then novaclient
changes will all be blocked until that's fixed.
We'd have to weigh the pros/cons to having some testing in the
short-term with novaclient vs the effort it would take to get a new job
framework setup for nova.
Another alternative is we should be able to test evacuate with the
in-tree functional tests using fixtures. That kind of testing only gives
us so much coverage since we have to stub some things out, but it would
at least test the mechanics of an evacuate through all of the services.
--
Thanks,
Matt
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
