New issue 661: Automatic grouping of tests by fixture instances broken https://bitbucket.org/hpk42/pytest/issue/661/automatic-grouping-of-tests-by-fixture
davidkr: from [Automatic grouping of tests by fixture instances](http://pytest.org/latest/fixture.html#automatic-grouping-of-tests-by-fixture-instances) > pytest minimizes the number of active fixtures during test runs. If you have > a parametrized > fixture, then all the tests using it will first execute with one instance and > then finalizers are > called before the next fixture instance is created. Among other things, this > eases testing of > applications which create and use global state. However the grouping doesn't seem to be working in some cases. Below is an example of what I mean. Here's the test code... ``` #!python import pytest @pytest.yield_fixture(scope="session", params=["big mac", "whopper"]) def burger(request): yield request.param @pytest.yield_fixture(scope="session", params=["curlyfries", "regularfries"]) def fries(request): yield request.param def test_burgers(burger): print "test_burgers: {0}".format(burger) def test_combo1(burger, fries): print("test_combo1: {0}{1}".format(burger, fries)) ``` The output from running the tests.. ``` test_grouping.py::test_burgers[big mac] PASSED test_grouping.py::test_combo1[big mac-curlyfries] PASSED test_grouping.py::test_combo1[whopper-curlyfries] PASSED test_grouping.py::test_burgers[whopper] PASSED test_grouping.py::test_combo1[whopper-regularfries] PASSED test_grouping.py::test_combo1[big mac-regularfries] PASSED ``` So the *whopper* instance of the *burger* fixture was created before the *big mac* instance was finalized and all its dependent tests were executed. I ran into this with my own test suite when I change a session-scoped fixture that other fixtures depended on from using the ```@pytest.fixture``` decorator to using ```metafunc.parametrize```. At first I thought it was a bug with using ```metafunc.parametrize``` but when I created a simpler reproduction set of tests I still get the same issue with simply using the decorator. It's more obvious when using metafunc though. For e.g. if I change the above test suite to use metafunc by doing this.. ``` #!python import pytest def pytest_generate_tests(metafunc): if "burger" in metafunc.fixturenames: metafunc.parametrize("burger", argvalues=["big mac", "whopper"]) ``` ..then the order changes yet again, all instances of ```test_burger``` get run first and the output looks like this.... ``` test_grouping_metafunc.py::test_burgers[big mac] PASSED test_grouping_metafunc.py::test_burgers[whopper] PASSED test_grouping_metafunc.py::test_combo1[big mac-curlyfries] PASSED test_grouping_metafunc.py::test_combo1[whopper-curlyfries] PASSED test_grouping_metafunc.py::test_combo1[big mac-regularfries] PASSED test_grouping_metafunc.py::test_combo1[whopper-regularfries] PASSED ``` If grouping was working as documented I'd expect the correct order to be... ``` test_grouping_metafunc.py::test_burgers[big mac] PASSED test_grouping_metafunc.py::test_combo1[big mac-curlyfries] PASSED test_grouping_metafunc.py::test_combo1[big mac-regularfries] PASSED test_grouping_metafunc.py::test_burgers[whopper] PASSED test_grouping_metafunc.py::test_combo1[whopper-curlyfries] PASSED test_grouping_metafunc.py::test_combo1[whopper-regularfries] PASSED ``` I've tried with non-yield fixtures as well and that made no difference. _______________________________________________ pytest-commit mailing list pytest-commit@python.org https://mail.python.org/mailman/listinfo/pytest-commit
