[issue30997] TestCase.subTest and expectedFailure

2017-07-23 Thread Louie Lu 

Louie Lu added the comment:

I'm successful to add a sub.expectedFailure() to subTest, but in current API, 
it isn't that will done.

First, you will not get any alarm or information that there is a  test been 
treat as "expected failure" at the result text.

Second, it just act like you putting @unittest.expectedFailure at the top of 
the test.

Third, current result didn't put expected failure as a result category, so it 
will not show it at the result if we skip some test in the subTest, and it will 
need to changed to add an expected failure list to handle multiple expected 
failure in a test.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30997] TestCase.subTest and expectedFailure

2017-07-23 Thread Ronald Oussoren 

Ronald Oussoren added the comment:

That's correct, I have a test like:


def test_something(self):

for info in CASES:
   with self.subTest(info):
  self.assert_something(info)

For some values of 'info' the test is known to fail and I want to mark those as 
exptectedFailure somehow and there doesn't appear to be a way to do so right 
now.

I'm currently doing:
with self.subTest(info):
   try:
   ... # actual test

   except:
   if info in KNOWN_FAILURES:
   self.skipTest()
   raise

That suppresses the test failures, but in an unclean way and without getting a 
warning when the testcase starts working again.

I could generate testcases manually the old fashioned way without using 
subTest, but that results in more complicated test code and requires rewriting 
a number of tests.

One possible design for making it possible to mark subTests as known failures 
it to return a value in the subTest context manager that has a method for 
marking the subTest:

 with self.subTest(...) as tc:
if ...:
   tc.expectedFailure(...)
 
... # actual test

I don't know how invasive such a change would be.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30997] TestCase.subTest and expectedFailure

2017-07-23 Thread Louie Lu 

Louie Lu added the comment:

There is a mailing-list discusss this before:

https://mail.python.org/pipermail/python-list/2016-June/710575.html


But after checking, current subTest contextmanager won't yield subtest, so the 
method of with self.subTest() as sub won't work.

Another problem is, inside the subTest, if there have expectfailed, it will 
raise _ShouldStop, causing the after test won't run it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30997] TestCase.subTest and expectedFailure

2017-07-23 Thread Louie Lu 

Louie Lu added the comment:

So for example:


for i in range(10):
with self.subTest(i=i):
DoSomething(i)

we have some platform specific in i=8, i=9, and you want to labeled as 
expectedFailure. Do I miss something?

--
nosy: +louielu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30997] TestCase.subTest and expectedFailure

2017-07-23 Thread Ronald Oussoren 

New submission from Ronald Oussoren:

It would be nice if there were a way to mark a TestCase.subTest as an expected 
failure.

I have a number of testcases that use the subTest feature and where a small 
subset of those tests are expected failures due to platform issues. It would be 
nice if it were possible to mark those tests as such. 

I'm currently using self.skipTest() to mark these subTests as special, but 
that's less than ideal.

--
components: Library (Lib)
messages: 298895
nosy: ezio.melotti, michael.foord, rbcollins, ronaldoussoren
priority: low
severity: normal
status: open
title: TestCase.subTest and expectedFailure
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com