New submission from Paul Ganssle <p.gans...@gmail.com>:
It seems that if you call skipTest *anywhere* in a test function, even in a
subTest, the *entire* function gets marked as "skipped", even if only one
sub-test is skipped.
Example:
import unittest
class SomeTest(unittest.TestCase):
def test_something(self):
for i in range(1, 3):
with self.subTest(i):
if i > 1:
self.skipTest('Not supported')
self.assertEqual(i, 1)
If you run `python3.7 -m unittest -v` on this, you get:
$ python -m unittest -v
test_something (test_mod.SomeTest) ... skipped 'Not supported'
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK (skipped=1)
Despite the fact that the test *was* run in the `i == 1` case. Similarly,
pytest marks this as a single skipped test:
========= test session starts =======
platform linux -- Python 3.7.1, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /tmp/test_mod, inifile:
collected 1 item
test_mod.py s [100%]
===== 1 skipped in 0.00 seconds =====
The solution to this is not obvious, unfortunately. One way to solve it would
be to have each subtest considered a "separate test". Another would be to
detect whether subTests have been skipped and expose only the *skipped* tests
as separate tests. You could also add a "partially skipped" marker, so to say,
"Some parts of this were skipped"
I suspect the right answer is some combination of these three answers -
possibly adding an extra-verbose mode that splits out all sub tests and having
skipped subTests default to being displayed separately.
Somewhat related to issue #30997.
----------
components: Tests
messages: 330527
nosy: ezio.melotti, michael.foord, p-ganssle, rbcollins
priority: normal
severity: normal
status: open
title: Using skipTest with subTest gives a misleading UI.
versions: Python 3.6, Python 3.7, Python 3.8
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue35327>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
