[issue17519] unittest should not try to run abstract classes

[Stephen Thorne]

Stephen Thorne  added the comment:

I have done some experimentation here and thought through this feature request.

The concept we are trying to deliver is: "I would like to share functionality 
between test classes, by having an abstract parent, with concrete leaves"

The metaclass abc.ABCMeta provides functionality that means two things:

 - any class with this metaclass (so the class and all its subclasses, 
typically) that have @abc.abstractmethod or @abc.abstractproperty decorated 
methods will be treated as abstract
 - any class that is treated as abstract will raise an exception immediately, 
to make it clear to the programmer (and unit tests) that a programming error 
has occured.

Following this through, we end up with two ways in which this can go  wrong in 
unit testing if we ask our unit testing framework to not test abstract classes.

This is a complete example, with both failure modes illustrated:

Consider:

class AbstractTestCase(unittest.TestCase, metaclass=abc.ABCMeta):
  ...

class FooTest(AbstractTestCase):
  def foo(self):
return 1

In this case, AbstractTestCase will not be skipped: this is because without any 
abstract methods inside it: it's not actually considered 'abstract', and is a 
concrete class.

In the second case:

class AbstractTestCase(unittest.TestCase, metaclass=abc.ABCMeta):
  @abc.abstractmethod
  def foo(self):
...

  @abc.abstractmethod
   def bar(self):
...

class FooTest(AbstractTestCase):
  def foo(self):
return 1

In this case, because AbstractTestCase has 2 abstract methods, it will be 
skipped. No tests run. But also FooTest will be skipped because it has 1 
abstract method, and is therefore also abstract.

If this were a 'normal' program, we would see an exception raised when FooTest 
is instanciated, but because we're skipping tests in abstract classes, we skip 
all the tests and exit with success.

My gut feeling on this is that what we really want is a decorator that says: 
Skip this class, and only this class, explicitly. All subclasses are concrete, 
only this one is abstract.

--
nosy: +sthorne

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Nathaniel Manista]

Nathaniel Manista  added the comment:

michael.foord: I am now persuaded that the feature requested here ought be 
reconsidered (since my last comment there's been a lot of chatter about it 
behind closed doors at work, but I can at least cite 
https://github.com/abseil/abseil-py/issues/166 as a public example of the 
question coming up).

Would it be appropriate to file a new issue? My issue tracker training brought 
me up to believe that it's better to reopen an existing closed issue in a 
circumstance like this, but I respect that that may not be the custom in this 
issue tracker, and besides I lack the permission in this issue tracker to 
reopen this issue. 😛

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Nathaniel Manista]

Nathaniel Manista  added the comment:

In the years since this was considered and declined, I wonder if the facts have 
changed sufficiently to make it now worth doing?

I often find myself writing TestCases for interfaces, and those define test_* 
methods that call the interface under test, but of course my TestCase needs to 
be abstract because I'm only testing an interface and not a concrete 
implementation of that interface. It's also the case when I'm writing this kind 
of test that I wish to use a type-checker, and if I can have my abstract 
TestCase inherit from unittest.TestCase, that will satisfy my type-checker's 
questions about why I believe my TestCase has all kinds of assert* methods 
defined that it doesn't otherwise see.

I currently have the impression that if this is cheap enough to do, it may be 
worth doing just for the ergonomics alone? It mightn't make anything impossible 
become possible to do, but I forecast that it would make something difficult to 
do much more straightforward to do.

(I remain a fan of the all-powerful load_tests protocol, but... often it's nice 
to escape all the responsibility that comes with use of it.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Nathaniel Manista]

Change by Nathaniel Manista :


--
nosy: +Nathaniel Manista

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Ezio Melotti]

Changes by Ezio Melotti :


--
nosy: +ezio.melotti
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Michael Foord]

Michael Foord added the comment:

As David says, the current workaround is to provide a mixin (base) class that 
inherits from object. Because this doesn't inherit from TestCase there is no 
confusion.

We *may* add a class marker that allows you to provide TestCase subclasses that 
won't be run. I don't think we'll do this with the abstract class machinery as 
it is additional complication for no benefit.

--
assignee:  -> michael.foord
resolution:  -> rejected
stage:  -> committed/rejected
status: open -> closed
versions:  -Python 2.7, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[R. David Murray]

R. David Murray added the comment:

I leave it to Michael to decide if your suggestion is a good addition :)

I personally don't see anything wrong with fake_abc.py...the stdlib test suite 
uses that idom extensively.  Other developers insist it should be called a 
"mixin" instead of a base class, but I don't see the point of the distinction 
myself.

If you want an ABC so that certain required methods are supplied by the actual 
TestCases, you could omit TestCase in the ABC and only supply it in the actual 
TestCase.  I'm guessing you will find that ugly too :)

--
nosy: +michael.foord, r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Éric Piel]

Changes by Éric Piel :


Added file: http://bugs.python.org/file29545/fake_abc.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Éric Piel]

New submission from Éric Piel:

Since Python 2.6 there is the notion if abstract class (ABC). It could be 
useful to use it for test cases, but unittest doesn't support it. Typically, 
I'd like to test a bunch of classes which all should behave similarly (at least 
for some cases). So I'd like to have one abstract class containing many test 
cases, and a separate real tests classes which inherit from this abstract class.

Unfortunately, for now unittest tries to instantiate the abstract class, which 
fails. 

Note that I'm not the only one thinking of this, here is a mention of the same 
idea on stack overflow:
http://stackoverflow.com/questions/4566910/abstract-test-case-using-python-unittest

Attached are two small examples of test cases. test_abs.py shows what I think 
is a good usage of ABC, with unittest. It fails to run with this error:
TypeError: Can't instantiate abstract class VirtualTest with abstract methods 
important_num
fake_abc.py is typically what people end up doing for using abstract classes 
with unittests (that's what people used to do before ABC exists). It does work, 
but it's not really beautiful as VirtualTest uses self.assertGreater() and 
self.important_num which are not explicitly part of the class.

My guess is that the following patch to Lib/unittest/loader.py should be enough 
(but it's untested):
diff -r a2128cb22372 Lib/unittest/loader.py
--- a/Lib/unittest/loader.pyThu Mar 21 23:04:45 2013 -0500
+++ b/Lib/unittest/loader.pyFri Mar 22 12:22:46 2013 +0100
@@ -6,6 +6,7 @@
 import traceback
 import types
 import functools
+import inspect
 
 from fnmatch import fnmatch
 
@@ -74,7 +75,8 @@
 tests = []
 for name in dir(module):
 obj = getattr(module, name)
-if isinstance(obj, type) and issubclass(obj, case.TestCase):
+if (isinstance(obj, type) and issubclass(obj, case.TestCase) and
+not inspect.isabstract(test_class)):
 tests.append(self.loadTestsFromTestCase(obj))
 
 load_tests = getattr(module, 'load_tests', None)

--
components: Library (Lib)
files: test_abc.py
messages: 184959
nosy: Éric.Piel
priority: normal
severity: normal
status: open
title: unittest should not try to run abstract classes
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file29544/test_abc.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com