Re: [openstack-dev] [qa] Issue in Tempest API's tests when running with nosetest command?
Yes, Issue is in nose. I do not like nose due to these kind of assumption for test running :). There is no reason of using nose, I just ran that and counter this issue. I totally agree on your point not to spent time to fix these kind of issue. As you mentioned. may be blocking nose would be good idea. On Fri, May 23, 2014 at 11:12 AM, Matthew Treinish wrote: > On Fri, May 23, 2014 at 09:34:35AM +0900, GHANSHYAM MANN wrote: > > Tempest API’s XML test will run with 1 test failure when run with > Nosetest. > > When we run the tempest with nosetest command (ex -nosetest –v > > ./tempest/api/compute/test_xyz.py), it run *BaseComputeTest > > ::create_test_server & create_test_server_group *functions also as test > > with any of the API tests. Because nose looks for tests in directories > and > > modules whose names start with *test* and *Test*, or contain a '_', '.', > > or '-' followed by test or Test. > > So this is part of the many reasons we don't support or advise running > tempest > with nosetests. I brought this up a few months ago when I tried to be a bit > more aggressive about this and block nose from being run at all. [1] Is > there > any particular reason you are still trying to run Tempest with nosetests? > > > > > > > > > So for each API XML test, failure will happen in > create_test_server_group > > function as it’s XML client is not implemented (as expected). > > > > > > > > To fix/stop these kind of issue, one solution is to have some policy for > > Tempest not to name non test function with test_, Test_ etc so that nose > > would not consider those as test case. > > So trying to accommodate nose especially on this isn't going to happen. The > python docs clearly state test_* named methods are what should be used so > that > the test driver recognizes them as tests. Nose is what is being overzealous > and causing issues; the bug is in nose not tempest. I think this suggested > patch > or something similar has been pushed at least twice already. The problem > is even > after making this change large chunks of tempest still won't work because > of > other things with nose. (like testscenarios) It's really not worth the > review > effort to merge a change like this. We've been actually moving in the other > direction too and removed all the in tree references and workarounds > related to > nose during Icehouse. > > For a project of tempest's size and growth rate trying to workaround all > the > quirks of every test runner isn't really feasible especially because we're > not > running with anything but testr in the gate. That being said I think nose > might > be a slightly special case in that I know it doesn't work for several > reasons. > (only because it used to be the default) Other test runners probably > wouldn't > have nearly as many issues, if any at all. > > > > > > > > > Looking forward to have more opinions on this. > > > > > > > -Matt Treinish > > [1] > http://lists.openstack.org/pipermail/openstack-dev/2014-January/024673.html > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Thanks & Regards Ghanshyam Mann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Issue in Tempest API's tests when running with nosetest command?
On Fri, May 23, 2014 at 09:34:35AM +0900, GHANSHYAM MANN wrote: > Tempest API’s XML test will run with 1 test failure when run with Nosetest. > When we run the tempest with nosetest command (ex -nosetest –v > ./tempest/api/compute/test_xyz.py), it run *BaseComputeTest > ::create_test_server & create_test_server_group *functions also as test > with any of the API tests. Because nose looks for tests in directories and > modules whose names start with *test* and *Test*, or contain a '_', '.', > or '-' followed by test or Test. So this is part of the many reasons we don't support or advise running tempest with nosetests. I brought this up a few months ago when I tried to be a bit more aggressive about this and block nose from being run at all. [1] Is there any particular reason you are still trying to run Tempest with nosetests? > > > > So for each API XML test, failure will happen in create_test_server_group > function as it’s XML client is not implemented (as expected). > > > > To fix/stop these kind of issue, one solution is to have some policy for > Tempest not to name non test function with test_, Test_ etc so that nose > would not consider those as test case. So trying to accommodate nose especially on this isn't going to happen. The python docs clearly state test_* named methods are what should be used so that the test driver recognizes them as tests. Nose is what is being overzealous and causing issues; the bug is in nose not tempest. I think this suggested patch or something similar has been pushed at least twice already. The problem is even after making this change large chunks of tempest still won't work because of other things with nose. (like testscenarios) It's really not worth the review effort to merge a change like this. We've been actually moving in the other direction too and removed all the in tree references and workarounds related to nose during Icehouse. For a project of tempest's size and growth rate trying to workaround all the quirks of every test runner isn't really feasible especially because we're not running with anything but testr in the gate. That being said I think nose might be a slightly special case in that I know it doesn't work for several reasons. (only because it used to be the default) Other test runners probably wouldn't have nearly as many issues, if any at all. > > > > Looking forward to have more opinions on this. > > -Matt Treinish [1] http://lists.openstack.org/pipermail/openstack-dev/2014-January/024673.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa] Issue in Tempest API's tests when running with nosetest command?
Tempest API’s XML test will run with 1 test failure when run with Nosetest. When we run the tempest with nosetest command (ex -nosetest –v ./tempest/api/compute/test_xyz.py), it run *BaseComputeTest ::create_test_server & create_test_server_group *functions also as test with any of the API tests. Because nose looks for tests in directories and modules whose names start with *test* and *Test*, or contain a '_', '.', or '-' followed by test or Test. So for each API XML test, failure will happen in create_test_server_group function as it’s XML client is not implemented (as expected). To fix/stop these kind of issue, one solution is to have some policy for Tempest not to name non test function with test_, Test_ etc so that nose would not consider those as test case. Looking forward to have more opinions on this. -- Thanks Ghanshyam Mann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
