Hello Holger, On 09-05-22 12:41 AM, holger krekel wrote: > are you aware that py.test runs test function in the order > in which they appear in the test module file, btw?
Is this by design? Can I always expect the functions and method be run in the defined order? Vis.: [quote] 'Tests usually run in the order in which they appear in the files. However, tests should not rely on running one after another, as this prevents more advanced usages: running tests distributedly or selectively, or in "looponfailing" mode, will cause them to run in random order.'[endquote] The keyword /usually/ suggests to me that that may not be the case always. > If one of the yielded tests fails, should the rest of the yielded tests > better not run at all? Correct. In my case, the rest of the yielded tests should not run. > Would you like to reuse the yielded test functions for other test cases? Usually not, but sometimes (if the tests are defined in reusable fashion), yes. > i'd like to understand your use case better. I get the impression > that something else/more than the current funcarg/generate mechanisms is > needed > to address it nicely. So please also state openly any problems/wishes you > have with the current yield-way of doing things. I gave some thought to this.. and let me explain: I have this (conceptually big) test case for which I want detailed reporting. This test case is the function ``test_typical_usecase``. All the test_* functions defined inside this function are parts of the parent test. If one of them fails, then of course the whole test is considered to be failed and thus rest of them should not run (this is a bug in my current test code as it continues to run them). I guess what I actually want out of this 'splitting' is fine-grained reporting. That is, if, say, test_import fails.. I should see FAIL for test_import so that I can immediately see where the problem is. [1] http://gist.github.com/115787 [2] http://gist.github.com/116260 Here, if test_import fails.. ``test_typical_usecase['test_import']`` is shown to be failing in [1] (fine-grained reporting) .. but this is not a correct way to do it.. as rest of the tests continue to run. In [2], test_typical_usecase is shown to be failing (not fine-grained reporting). Cheers, Sridhar _______________________________________________ py-dev mailing list py-dev@codespeak.net http://codespeak.net/mailman/listinfo/py-dev
