Hi everyone, I have been reading the etherpad from the Paris summit wrt to moving the functional tests into their respective projects (https://etherpad.openstack.org/p/kilo-crossproject-move-func-tests-to-projects). I am mostly interested this from the nova project perspective. However, I still have a lot of questions.
For example, is it permissible (or a good idea) to use the python- *clients as a library for the tasks? I know these were not allowed in Tempest, but I don't see why they couldn't be used here (especially since, AFAIK, there is no testing done on the SDK clients themselves). Another question is also about a difference between Tempest and these new functional tests. In nova's case, it would be very useful to actually utilize the libvirt library in order to touch the hypervisor itself. In Tempest, it's not allowed to do that. Would it make sense to be able to make calls to libvirt within a nova functional test? Basically, since Tempest was a "public" only library, there needs to be a different set of rules as to what can and can't be done. Even the definition of what exactly a functional test is should be more clearly stated. For example, I have been working on a project for some nova tests that also use the glance and keystone clients (since I am using the python SDK clients). I saw this quote from the etherpad: " Many "api" tests in Tempest require more than one service (eg, nova api tests require glance) Is this an API test or an integration test or a functional test? sounds to me like cross project integration tests +1+1" I would disagree that a functional test should belong to only one project. IMHO, a functional test is essentially a black box test that might span one or more projects, though the projects should be related. For example, I have worked on one of the new features where the config drive image property is set in the glance image itself, rather than specified during the nova boot call. I believe that's how a functional test can be defined. A black box test which may require looking "under the hood" that Tempest does not allow. Has there been any other work or thoughts on how functional testing should be done? Thanks, Sean Toner _______________________________________________ OpenStack-dev mailing list OpenStackfirstname.lastname@example.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev