Now that we have achieved the goal of parallel tempest in the gate using
testr we have to be careful that we don't introduce tests that are
flaky. This may be obvious to many of you but we should include some
information in the tempest README. Here is a start. Improvements are
welcome and I will soon add this to the README.
-David
A new test only has to pass twice to get into the gate. The default
tenant isolation prevents most tests from trying to access the same
state in a racy way, but there are some apis, particularly in the
whitebox and admin area (also watch out for the cli tests) that affect
more global resources. Such tests can cause race failures. In some cases
a lock can be used to serialize execution for a set of tests. An example
is AggregatesAdminTest. Races between methods in a class is not a
problem because parallelization is at the class level, but if there is a
json and xml version of the class there could still be a problem.
Reviewers need to keep on top of these issues.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
