On 13/02/2010 04:01, Glyph Lefkowitz wrote:
[snipping some good points...]
Regarding the objection that setUp/tearDown for classes would run into
issues with subclassing, I propose to let the standard semantics of
subclasses do their job. Thus a subclass that overrides setUpClass or
tearDownClass is responsible for calling the base class's setUpClass
and tearDownClass (and the TestCase base class should provide empty
versions of both). The testrunner should only call setUpClass and
tearDownClass for classes that have at least one test that is
selected.
Yes, this would mean that if a base class has a test method and a
setUpClass (and tearDownClass) method and a subclass also has a test
method and overrides setUpClass (and/or tearDown), the base class's
setUpClass and tearDown may be called twice. What's the big deal? If
setUpClass and tearDownClass are written properly they should support
this.
Just to be clear: by "written properly" you mean, written as classmethods,
storing their data only on 'cls', right?
Heh, yes (answered several times in this thread already I think...).
If this behavior is undesired in a particular case, maybe what
was really meant were module-level setUp and tearDown, or the class
structure should be rearranged.
There's also a bit of an open question here for me: if subclassing is allowed,
and module-level setup and teardown are allowed, then what if I define a test
class with test methods in module 'a', as well as module setup and teardown,
then subclass it in 'b' which *doesn't* have setup and teardown... is the
subclass in 'b' always assumed to depend on the module-level setup in 'a'? Is
there a way that it could be made not to if it weren't necessary? What if it
stubs out all of its test methods? In the case of classes you've got the 'cls'
variable to describe the dependency and the shared state, but in the case of
modules, inheritance doesn't create an additional module object to hold on to.
This is also an interesting point. The 'naive' implementation, which I
think I prefer, only runs the setUpModule of modules actually containing
tests. Similarly setUpClass is only called on classes with actual tests,
although they may call up to base class implementations.
This has a couple of consequences, particularly for setUpModule. It
makes the basic rule:
* only use setUpModule for modules actually containing tests
* don't mix concrete TestCases (with tests) in the same module (using
setUpModule) as base classes for other tests
The use case (that I can think of) that isn't supported (and of course
all this needs to be documented):
* Having setUpModule / tearDownModule in modules that define base
classes and expecting them to be called around all tests that inherit
from those base classes
Having this in place makes the implementation simpler. If we explicitly
document that this rule may change and so users shouldn't rely on
setUpModule not being called for modules containing base classes, then
we are free to rethink it later. Not having this restriction at all is
possible, it just requires more introspection at TestSuite creation /
ordering time.
Note that setUpClass on a base class maybe called several times if
several base classes inherit and all call up to the base class
implementation. As it will be a class method the cls argument will be
different for each call.
Another question. If we are implementing TestCase.setUpClass as an
additional test then should it be reported *even* if it is only the
default (empty) implementation that is used?
The reason to have setUpClass implemented as a test is so that you can
report the failure *before* you run all the tests. Lots of unit test
users want a consistent number of tests every run - so we shouldn't
insert an extra test only on fail. The other alternative is to report a
setUpClass failure as part of the first test that depends on it - this
makes the implementation more, complex (having a pseudo-test represent
setUpClass / tearDownClass is convenient for ordering and isolating the
'magic' in one place - the rest of the test running infrastructure
doesn't need to know about setUpClass or Module).
If we do add a default setUpClass test for all TestCases it means extra
noise for test runs that don't use the new feature. Saying no it
shouldn't be shown means that we have to introspect test classes to see
if they inherit setUpClass from TestCase or from some intermediate base
class. Not hard just an extra complexity. One solution would be for
TestCase *not* to have default implementations, but it is probably nicer
for them to appear in the API.
I guess my preferred approach is to have a default implementation, but
not to create pseudo-tests for them if they aren't used.
All the best,
Michael
testresources very neatly sidesteps this problem by just providing an API to say
"this test case depends on that test resource", without relying on the grouping
of tests within classes, modules, or packages. Of course you can just define a
class-level or module-level resource and then have all your tests depend on it, which
gives you the behavior of setUpClass and setUpModule in a more general way.
-glyph
--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog
READ CAREFULLY. By accepting and reading this email you agree, on behalf of
your employer, to release me from all obligations and waivers arising from any
and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap,
clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and
acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in perpetuity, without
prejudice to my ongoing rights and privileges. You further represent that you
have the authority to release me from any BOGUS AGREEMENTS on behalf of your
employer.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
