On 11/12/2013 10:25 PM, Robert Collins wrote: > We shouldn't really be changing the config of the cloud we're testing > - that stops us being run against actual prod clouds. > > Instead we should have some set of profiles we test against - and then > run different jobs for different profiles, avoiding the problem > entirely.
The issue here is we've got an API that's exposed to the admin to do this, which means for some folks, it would be nice to test this functionality is working. Per our community mantra "If it's not tested, it's assumed broken." - https://twitter.com/russellbryant/status/396889282155008000 That being said, in this case in particular, we should probably be more careful about disabling our only nova-compute during the tests, because, you know, that might be bad. Today the calls happen so quickly, I think we've just avoided the race entirely (the enable_disable test has been in the gate since H2). I've honestly never seen a race that we can track down to this in the gate. For now, I'd agree that enable_disable should probably be removed from a normal tempest run. I think we've started to find that we have a class of APIs that impact our runtime in a dramatic enough way, that we need to be really careful about how we tickle them. I'd be interested in ideas about how we do that. Remember we also have locking in our bag of tricks, which is what we need to do around all the aggregate tests, as those are admin tests on global state. -Sean > > -Rob > > On 13 November 2013 15:23, Zhi Kun Liu <liuzhi...@gmail.com> wrote: >> Hi all, >> >> In Tempest, we have some testcases that disable/enable services or agents. >> e.g. >> https://review.openstack.org/#/c/55271/1/tempest/api/compute/admin/test_services.py >> test_service_enable_disable >> test_disable_service_with_disable_reason >> >> Since Tempest run in parallel for now, I'm afraid of these testcases have >> possible impact on other tests. Anyone has ideas about it? Should we remove >> such testcases? >> >> Regards, >> Zhi Kun >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > -- Sean Dague http://dague.net
signature.asc
Description: OpenPGP digital signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev