While working on a test suite for unittest these past few weeks, I've
run across some behaviours that, while not obviously wrong, don't
strike me as quite right, either. Submitted for your consideration:
1) TestCase.tearDown() is only run if TestCase.setUp() succeeded. It
seems to me that tearDown() should always be run, regardless of any
failures in setUp() or the test method itself.
The case I'm considering is something like this, ie, a multi-part setUp():
> def setUp(self)
> lock_file(testfile) # open_socket(), connect_to_database(), etc
>
> something_that_raises_an_exception()
>
> def tearDown(self):
> if file_is_locked(testfile):
> unlock_file(testfile)
In this pseudo-code example, the file won't be unlocked if some later
operation in setUp() raises an exception. I propose that
TestCase.run() be changed to always run tearDown(), even if setUp()
raise an exception.
I'm undecided if this is a new feature (so it should go in for 2.6) or
a bug fix; I'm leaning toward the latter.
2) The TestLoader.testMethodPrefix attribute currently allows anything
to be assigned to it, including invalid objects and the empty string.
While the former will cause errors to be raised when one of
TestLoader's loadTestsFrom*() methods is called, the empty string will
raise no exception; rather, the loadTestsFrom*() methods will
interpret every possible attribute as being a test method, e.g.,
meaning you get things like assertEqual(), failUnlessEqual(), etc,
when TestLoader.loadTestsFromTestCase() is run.
I propose protecting testMethodPrefix with a property that validates
the assigned value, restricting input to non-empty instances of str. I
see this as a bug fix that should go in before 2.5-final.
3) TestLoader.loadTestsFromTestCase() accepts objects that are not
test cases and will happily look for appropriately-named methods on
any object you give it. This flexibility should be documented, or
proper input validation should be done (a bug fix for 2.5).
4) TestLoader.loadTestsFromName() (and by extension,
loadTestsFromNames(), too) raises an AttributeError if the name is the
empty string because -- as it correctly asserts -- the object does not
contain an attribute named ''. I recommend that this be tested for and
ValueError be raised (bug fix for 2.5).
This of course leads into the question of how much input validation
should be done on these names. Should loadTestsFrom{Name,Names}() make
sure the names are valid attribute names, or is this overkill?
5) When TestLoader.loadTestsFrom{Name,Names}() are given a name that
resolves to a classmethod on a TestCase subclass, the method is not
invoked. From the docs:
> The specifier name is a ``dotted name'' that may resolve either to a module,
> a test
> case class, a TestSuite instance, a test method within a test case class, or a
> callable object which returns a TestCase or TestSuite instance.
It is not documented which of these tests takes priority: is the
classmethod "a test method within a test case class" or is it a
callable? The same issue applies to staticmethods as well.
Once I get answers to these questions, I can finish off the last few
bits of the test suite and have it ready for 2.5-final.
Thanks,
Collin Winter
--
http://mail.python.org/mailman/listinfo/python-list
