On Tue, Feb 9, 2010 at 5:34 PM, Holger Krekel <[email protected]> wrote:
> On Tue, Feb 9, 2010 at 10:57 PM, Ben Finney <[email protected]>
> wrote:
>> Michael Foord <[email protected]> writes:
>>
>>> The next 'big' change to unittest will (may?) be the introduction of
>>> class and module level setUp and tearDown. This was discussed on
>>> Python-ideas and Guido supported them. They can be useful but are also
>>> very easy to abuse (too much shared state, monolithic test classes and
>>> modules). Several authors of other Python testing frameworks spoke up
>>> *against* them, but several *users* of test frameworks spoke up in
>>> favour of them. ;-)
>>
>> I think the perceived need for these is from people trying to use the
>> ‘unittest’ API for test that are *not* unit tests.
>>
Well the example I was talking about before is when some (critical)
resource needed for unittesting requires a very, very heavy
initialization process. I'll employ the most recent example (hope it
doesn't look like too much biased, it's just to illustrate the whole
picture ;o) which is unittests for a framework like Trac . In that
case it is critical to have a Trac environment, a ready-to-use DB and
backend, initialize the plugins cache by loading relevant plugins, run
the actions specified by each IEnvironmentSetup participant, sometimes
a ready to use repository (if testing code depending on Trac VCS API)
and more ... Just considering these cases someone could :
- Create a fake environment used as a stub
- But having a single global environment is not a good idea because
it would be very difficult to run multiple (independent) tests
concurrently (e.g. test multiple Trac plugins concurrently in a dedicated
CI environment). So an environment has to be started for every
test run and be as isolated as possible from other similar
stub environments
- The DB and backend can be replaced by using in-memory SQLite
connection
- Plugins cache and loading is mandatory as well running the actions
specified by each IEnvironmentSetup participant
- VCS can be mocked, but if it's needed it has to be initialized as well
And all this is needed to run *ANY* test of *ANY* kind (that includes
unittests ;o) . I hope that, up to this point, you all are convinced
of the fact that all this cannot be done for each TestCase instance.
That's why something like class-level setup | teardown might be useful
to get all this done just once ... but it's not enough
Something I consider a limitation of that approach is that it is a
little hard to control the scope of setup and teardown. For instance,
if I was trying to run Trac test suite I'd like to create the
environment stub just once, and not once for every (module | class)
containing tests. The current approach does not fit very well
scenarios like this (i.e. setup | teardown actions span even beyond
single modules ;o)
So that's why it seems that the approach included in Trac testing code
(i.e. a global shared fixture ) will still be needed, but AFAICR it
breaks a little the interface of TC class and setup and tear down has
to be performed from the outside.
OTOH another minimalistic framework I've been building on top of
`dutest` to cope with such scenarios (aka TracDuTest but not oficially
released yet :-/ ) seems to handle all those features well enough by
using doctest extraglobs or by modifying the global namespace at any
given time inside setUp and tearDown (thus hiding all this code from
doctests ;o).
> One nice bit is that you can for a given test module issue "py.test
> --funcargs"
> and get a list of resources you can use in your test function - by simply
> specifying them in the test function.
>
> In principle it's possible to port this approach to the stdlib - actually i
> consider to do it for the std-unittest- running part of py.test because
> people asked for it - if that proves useful i can imagine to refine it
> and offer it for inclusion.
>
Considering part of what I've mentioned above:
Q:
- How could py.test help in cases like this ?
- Considering the similitudes with unittest style (at least IMO)
I think I'd prefer something like PeckCheck to generate and run
parameterized TCs. What d'u think ? (I confess that I don't use
py.test , nose ... because I see they use too much magic & ...,
but that's just my *VERY* biased opinion, so I won't start
a war or alike ;o)
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
Nabble - Trac Users - Embedding pages? -
http://feedproxy.google.com/~r/TracGViz-full/~3/MWT7MJBi08w/Embedding-pages--td27358804.html
_______________________________________________
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
