Hash: SHA1

On 03/28/2011 10:04 AM, Jim Fulton wrote:
> On Sun, Mar 27, 2011 at 11:13 AM, Martin Aspeli
> <optilude+li...@gmail.com> wrote:
>> Hi,
>> On 27 March 2011 15:54, Uli Fouquet <u...@gnufix.de> wrote:
>>> The (limited) experiences with py.test, however, were awesome. Some
>>> points that are quite cool IMHO:
>>> - Easy finding of tests: just write some ``test_function`` in a
>>>  ``test_module`` and it will be found and executed. That also makes
>>>  py.test tests more readable and maybe more intuitive.
>> I'm not sure this is always a good idea. It feels a bit implicit, and
>> having a base class isn't really a big problem, IMHO. It seems a bit
>> like the kind of thing that sounds cool (look, it's even easier!), but
>> in practice makes little difference.
> +1
> +1
> +1
> This is especially important for doctests (and manuel) or any
> situation where setup is important and where you can't really guess.
>>> - py.test is more widespread in the Python community (that's my
>>>  impression; I can't proof it)
>> What about nose?
> It looks to me like a layerish mechanism might be possible in nose, or
> at least like
> zope.testing layers could be integrated with nose.
>>> - Support of unittest/unittest2: you can write standard lib setups
>>>  (defining TestCases; no need to also write testsuite-setup stuff) and
>>>  they will be found/executed. zope.testrunner for instance does not
>>>  support the new `setUpClass`/`tearDownClass` concept of unittest2
>>>  (yes, you would use layers in that case; but it might be nice if
>>>  zope.testrunner  would support also class-wide fixtures in
>>>  unittest2-style; people from other worlds might expect that to work).
>> zope.testing should definitely gain support for the new unittest2
>> hooks. That wouldn't be very hard, though. ;-)
> I assume you mean zope.testrunner.
>>> Main drawbacks I see on py.test side are:
>>> - Lack of layer support (yet). Maybe we can do something about that in
>>>  `zope.pytest` based on `plone.testing.layer`.
>>> - Limited doctest support. It is quite difficult (AFAIK) to define
>>>  fixtures for doctests or to even set the usual doctest options
>>>  (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests
>>>  are simply collected and executed and not much finetuning is possible.
>> With zope.testrunner, you *do* need a test_suite method to run
>> doctests. I think that's a good thing. Look at plone.testing's README
>> for examples.
> Again, +1
> If I were to use nose or py.tests, I would want to adopt an explicit
> style, which I believe is possible w nose.

FWIW, I thought the same, but haven't missed 'test_suite' appreciably
after a week or so of acclimatization to the nose discovery mechanism.
The only wrinkle I know of is that one doesn't use base classes for test
cases where the base class itself derives from unittest.TestCase:
instead, the base is just used as a "pure mixin".

>> FWIW, I think we should stop using .txt doctests for unit tests.
> I disagree, of course.
>> Doctests should be used to test *documentation* ("the examples are
>> valid").
> Manuel is *much* better for that.  (Of course, manuel is arguably a form of
> doctest.)

The Sphinx doctest integration[1] ('make doctest') supports both classic
"interpreter prompt" doctests and a more useful (IMHO) "code-output"
form, such as:

.. testcode::

   1+1        # this will give no output!
   print 2+2  # this will give output

.. testoutput::


[1] http://sphinx.pocoo.org/ext/doctest.html

>> For actual unit tests, writing tests in a unittest class is
>> almost always better in the long run. doctests don't scale well and
>> discourage the kind of ad-hoc "this seems broken, I'll just write a
>> quick test" or "I just fixed a bug, better add a regression test"
>> testing.
> You're just not using them correctly. :)

The vast majority of the doctest testcases in zope.* packages fall into
this category:  poor isolation, lots of edge cases which would obscure
any real narrative docs, of which there are almost none.  I believe the
conflict is intrinsic, here, and not just an accident of careless /
naive implementation:  exercising all the preconditions of the contract
of the function-under-test makes for really poor documentation, but is
essential to good testing.

> One thing I hate about unittest is the javiotic ceremony it involves.
> Doctests can cut down on the clutter a lot. I believe that that is
> py.test's strength as well..

"I'll take explicit for $1000, Alex."

> More generally, I'd love to see us adopt another test runner so that
> we can stop maintianing zope.testrunner.  When it was written at
> the turn of the century, there weren't good alternatives.  Personally,
> I think maintaining it is boring.

Nose and coverage have been pretty good to repoze.* and pyramid_*.

- -- 
Tres Seaver          +1 540-429-0999          tsea...@palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/


Zope-Dev maillist  -  Zope-Dev@zope.org
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope )

Reply via email to