Good to know. I think it's a good idea to implement a common compute verifier after instances booted. Maybe we can define different checking levels so that it can be leveraged by different test cases. I will see what I can do.
On 24/06/14 22:27, Sean Dague wrote: > On 06/24/2014 01:29 AM, Fei Long Wang wrote: >> Greetings, >> >> We're leveraging the scenario test of Tempest to do the end-to-end >> functional test to make sure everything work great after upgrade, >> patching, etc. And We're happy to fill the gaps we found. However, I'm a >> little bit confused about the test policy from the scenario test >> perspective, especially comparing with the API test. IMHO, scenario test >> will cover some typical work flows of one specific service or mixed >> services, and it would be nice to make sure the function is really >> working instead of just checking the object status from OpenStack >> perspective. Is that correct? >> >> For example, live migration of Nova, it has been covered in API test of >> Tempest (see >> https://github.com/openstack/tempest/blob/master/tempest/api/compute/test_live_block_migration.py). >> But as you see, it just checks if the instance is Active or not instead >> of checking if the instance can be login/ssh successfully. Obviously, >> from an real world view, we'd like to check if it's working indeed. So >> the question is, should this be improved? If so, the enhanced code >> should be in API test, scenario test or any other places? Thanks you. > The fact that computes aren't verified fully during the API testing is > mostly historical. I think they should be. The run_ssh flag used to be > used for this, however because of some long standing race conditions in > the networking stack, that wasn't able to be turned on in upstream > testing. My guess is that it's rotted now. > > We've had some conversations in the QA team about a compute verifier > that would be run after any of the compute jobs to make sure they booted > correctly, and more importantly, did a very consistent set of debug > capture when they didn't. Would be great if that's something you'd like > to help out with. > > -Sean > > > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Cheers & Best regards, Fei Long Wang (???) -------------------------------------------------------------------------- Senior Cloud Software Engineer Tel: +64-48032246 Email: [email protected] Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
