Hi Bogdan, On Sun, Apr 01, 2012 at 02:16 +1100, Bogdan Opanchuk wrote: > I am trying to wrap performance tests for my module into py.test. Each > testcase is, basically, a data preparation stage (can take some time) > and the test itself. Is there some way (hooks, plugins) to make > py.test print the value returned from the testcase (and make testcases > time whatever they want)? What I want is just a table like the one > after "py.test -v", but with return values instead of "PASSED". I > googled it and haven't found any solutions.
I don't know of a direct available example solution. > At the moment I am doing it as following: > 1. Overload "pytest_pyfunc_call()" and save returned result into > "pyfuncitem" parameter. > 2. Overload "pytest_runtest_makereport()", look for "perf" mark and > somehow output "item.result". > 3. I haven't decided about the way to output it yet, because I am > currently trying to find the place where "PASSED" is printed. do you want to consider performance regressions as a failure? > But this seems quite clunky and overcomplicated. Is there any better > way to do this? If not for the data preparation, the currently > existing time measuring system would probably be enough for me > (although I'd still miss the ability to print GFLOPS value instead of > seconds), but the data preparation is testcase-specific and I do not > want to move it to "pytest_generate_tests()" hook. > > Thank you in advance. Could you maybe provide a simple example test file and make up some example output that you'd like to see? Meanwhile, if you haven't already you might want to look at the output of "py.test --durations=10" and see about its implementation (mostly contained in _pytest/runner.py, grep for 'duration'). best, holger _______________________________________________ py-dev mailing list py-dev@codespeak.net http://codespeak.net/mailman/listinfo/py-dev
