On Tue, Jan 06, 2009 at 07:11:45AM -0800, Istvan Albert wrote: -> To this end I have partially reworked many of the test modules. You -> can see what I have done by getting my repository at: -> -> git clone http://atlas.bx.psu.edu/git/pygr-test2.git/ -> -> The new tests are located in the tests2 directory. The following -> issues have been worked on: -> -> - no more driveby-file-bombing: test data files live in the data -> directory, all temporary files are created in the temp subdirectory -> (that will be created automatically), the only file that I was unable -> to move to temp is the downloaded file.
Excellent! -> - no more access errors on windows (due to removing files without them -> being closed, windows would complain) That's great! -> - removed dependency on nose, tests are based on unittest. I found the -> old protest.py to be a little convoluted, since I don't know that the -> reason for running every testmodule in a new python interpreter was I -> just bypassed that, the advantage is that this makes it a mere 10 or -> so lines of actual code. So, I think the original dependency on nose could use a bit of explanation... I like nose because it can dramatically simplify your tests; in particular, nose allows you to write tests that contain less boilerplate than unittest tests. It also has a number of nifty plugins that let you use code coverage analysis, profiling, HTML output, etc. within the test framework. [0] Therefore I recommended it to Chris. However, Chris ran into some issues with nose itself that led to him ripping out the simplified syntax and simply inheriting from nose classes in an OO style. In practice this didn't lead to simpler test code! So the tests look very much like unittest tests anyway. In addition, nose does a fine job of discovering and executing unittest-style tests, and will even let you apply plugins to them; unlike py.test, nose is backwards-compatible with unittest. So with the caveat that I would (personally) very much like to be able to use nose as a test runner, I'm in favor of cleaning and simplifying. This is especially true if it makes the test suite run on Windows. -> The job is not finished but I'm putting it out here to get some -> feedback. [ ... ] -> One of the reasons I started this is to understand better what every -> component does. In the end I had some fun with it and I am willing to -> spend more time with it, provided this is something that is positively -> received. I like the basic idea and think it's a great simplification -- lack of (my) involvement and Chris's issues with pygr.Data led to 'protest' becoming overcomplicated, but it seemed like the lesser of two evils to just leave it alone while things worked! One problem I note is that, at least at the moment, runtest.py does not obviously fail even if one of the tests fails. That is, even if it fails on an early test, you still get an 'OK!' message at the bottom. The proper way to deal with this is to integrate all of the tests into a unittest test runner but I'm afraid I don't know how to do that. (...but I'm pretty sure it can be done). Another problem is that nose does not, by default, discover and run executable Python files; this is to avoid running scripts named 'test-blah'. You can override this with 'nosetests --exe', in which case nose discovers the tests under tests2/, aggregates them, and runs them all quite nicely. A third problem -- and you and Chris alluded to this in your later discussion -- is that currently the pygr.Data tests *have* to run in a separate process. There may also be pygr.Data persistence or other "global state" contamination that takes place; I'm not sure. This is what led Chris to have protest spawn off new Python processes for each test. Finally, I think we need a sys.path hack somewhere, as described in my other e-mail. It's bitten me too many times for me to believe that it *won't* cause problems for others... cheers, --titus [0] My impression (probably quite biased) is that nose is close to becoming a majority partner in the Python testing community. Between nose and py.test, the simplistic style of tests allowed by these two frameworks is becoming widely used. -- C. Titus Brown, [email protected] --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pygr-dev" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pygr-dev?hl=en -~----------~----~----~----~------~----~------~--~---
