Hello, We (agordeev and vkozhukalov) had a discussion about functional testing flow described here https://etherpad.openstack.org/p/IronicDevstackTesting.From our perspective it has some non-critical, but still notable flaws. If you ask me, it looks a bit strange to create testing networks and VMs during devstack run using virsh and shell. Maybe more suitable to use libvirt python API for this purpose and create test environment right before launching testing scenario (aka setUp stage). We know that according to tempest development requirements it is not allowed to hit hypervisors directly ( http://docs.openstack.org/developer/tempest/overview.html ), but using libvirt is not hitting hypervisor directly. We described alternative testing flow at the same document https://etherpad.openstack.org/p/IronicDevstackTesting.
There are some reasons why it is more correct to create VMs inside tempest itself, not inside devstack. - Firstly, we possibly want to have several functional testing scenarios and every scenario needs to have clean testing environment, so we can just add setUp and tearDown stages for creating and destroying testing envs. - Secondly, virsh has some performance issues if you deal with >30 VMs (it is not our case for now but who knows). Besides, as we found out ssh power driver also uses virsh for VM power management and does it ineffectively looking over all VMs (virsh dumpxml for every VM and grep MAC). It is possible to significantly improve testing power management performance if we substitute ssh power driver with driver which uses libvirt python API and lookups VM by UUID, not by MAC. - Third point here is that adding nodes into ironic is also a part of ironic functionality and it is supposed to be tested as well as others. Besides, if something goes wrong during adding nodes into ironic, we'll get testing scenario failed, not devstack failed. It is what we usually expect from testing procedure. - And finally, using tempest with python libvirt API we can create widely customizable testing envs, it is not possible if we use devstack for that purpose. Ok. It is just our point of view. If the decision about how to implement creating testing VMs already made, we'll make it according to above mentioned description. As far as original flow was suggested by infra and QA gangs, it means that they a priori agree with that flow and we'll merge it much faster than any other one. Let us know what you think. Your feedback is highly appreciated :) Thanks!
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev