Chris Jerdonek added the comment:
Nice/elegant idea. A couple comments:
(1) What will be the semantics of TestCase/subtest failures? Currently, it
looks like each subtest failure registers as an additional failure, meaning
that the number of test failures can exceed the number of test cases. For
example (with a single test case with 2 subtests):
$ ./python.exe test_subtest.py
FF
======================================================================
FAIL: test (__main__.MyTests) (i=0)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subtest.py", line 9, in test
self.assertEqual(0, 1)
AssertionError: 0 != 1
======================================================================
FAIL: test (__main__.MyTests) (i=1)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_subtest.py", line 9, in test
self.assertEqual(0, 1)
AssertionError: 0 != 1
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=2)
With the way I understand it, it seems like a subtest failure should register
as a failure of the TestCase as a whole, unless the subtests should be
enumerated and considered tests in their own right (in which case the total
test count should reflect this).
(2) Related to (1), it doesn't seem like decorators like expectedFailure are
being handled correctly. For example:
@unittest.expectedFailure
def test(self):
for i in range(2):
with self.subTest(i=i):
self.assertEqual(i, 0)
This results in:
Ran 1 test in 0.001s
FAILED (failures=1, unexpected successes=1)
In other words, it seems like the decorator is being applied to each subtest as
opposed to the test case as a whole (though actually, I think the first should
read "expected failures=1"). It seems like one subtest failing should qualify
as an expected failure, or are the semantics such that expectedFailure means
that every subtest must fail?
----------
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue16997>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
