Kit Yan Choi <k...@kychoi.org> added the comment:

For your test:

class T(unittest.TestCase):
    def test_f(self): raise TypeError()

If you run this test with unittest test runner, you should get this result:

E
======================================================================
ERROR: test_f (test_main.T)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_main.py", line 5, in test_f
    def test_f(self): raise TypeError()
TypeError

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)


I expect to get this behaviour even if the test is decorated with 
unittest.expectedFailure. However, currently we get a success.


Scenario:


You create a class named Duck with a method "quack".  Then you added a test, 
and test needs to call Duck.quack.

Later on for whatever reason, you need to decorate the test with 
expectedFailure.  The test passes with the expected failure.

Then you rename the "quack" method to "walk", but you forget to update the 
test.  Now the test is actually failing with an AttributeError, but you won't 
notice it because expectedFailure silences it.

In this scenario, it is important to differentiate a "test error" and a "test 
failure". A test has four status: success, failure, error, skipped.  I expect 
unittest.expectedFailure to make "failure" a "success" and a "success" a 
"failure", and it should leave "error" and "skipped" unchanged.

Please consider reopening this issue.

----------
nosy: +Kit Yan Choi

_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue38296>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to