Hi !
... sorry , I could not avoid to reply this message ...
On 5/20/13, Michael Foord <fuzzy...@voidspace.org.uk> wrote:
>
> On 20 May 2013, at 18:26, Mark Janssen <dreamingforw...@gmail.com> wrote:
>
>>>> I'm hoping that core developers don't get caught-up in the "doctests are
>>>> bad
>>>> meme".
>>>>
>>>> Instead, we should be clear about their primary purpose which is to
>>>> test
>>>> the examples given in docstrings.
>>>> In other words, doctests have a perfectly legitimate use case.
>>>
>>> But more than just one ;-) Another great use has nothing to do with
>>> docstrings: using an entire file as "a doctest". This encourages
>>> writing lots of text explaining what you're doing,. with snippets of
>>> code interspersed to illustrate that the code really does behave in
>>> the ways you've claimed.
>>
>> +1, very true. I think doctest excel in almost every way above
>> UnitTests. I don't understand the popularity of UnitTests, except
>> perhaps for GUI testing which doctest can't handle. I think people
>> just aren't very *imaginative* about how to create good doctests that
>> are *also* good documentation.
>>
>
With enhanced doctests solution in mind ...
> Doc tests have lots of problems for unit testing.
>
> * Every line is a test with *all* output part of the test - in unit tests
> you only assert the specific details you're interested in
custom output checkers
> * Unordered types are a pain with doctest unless you jump through hoops
( custom output checkers + doctest runner ) | (dutest __tc__ global var)
> * Tool support for editing within doctests is *generally* worse
this is true , let's do it !
> * A failure on one line doesn't halt execution, so you can get many many
> reported errors from a single failure
it should if REPORT_ONLY_FIRST_FAILURE option [1]_ is set .
> * Try adding diagnostic prints and then running your doctests!
I have ... dutest suites for my Trac plugins do so . However logging
information is outputted to /path/to/trac/env/log/trac.log ... so a
tail -f is always handy .
> * Tools support in terms of test discovery and running individual tests is
> not as smooth
dutest offers two options since years ago MultiTestLoader combines
multiple test loaders to *load* different kinds of tests at once from
a module , whereas a package loader performs test discovery . These
loader objects are composable , so if an instance of MultiTestLoader
is supplied in to the package test loader then multiple types of tests
are loaded out of modules all over across the package hierarchy .
Indeed , in +10 years of Python development I've never used
unittest(2) discovery, and even recently implemented the one that's
used in Apache™ Bloodhound test suite . Unfortunately I've had no much
time to spend on improving all this support in dutest et al.
> * Typing >>> and ... all the time is really annoying
... I have faith ... there should be something like this for vim ... I
have faith ... ;)
> * Doctests practically beg you to write your code first and then copy and
> paste terminal sessions - they're the enemy of TDD
Of course , not , all the opposite . If the approach is understood
correctly then the first thing test author will do is to write the
code «expected» to get something done . When everything is ok with API
code style then write the code . Many problems in the API and
inconsistencies are thus detected early .
> * Failure messages are not over helpful and you lose the benefit of some of
> the new asserts (and their diagnostic output) in unittest
(custom ouput checkers) | ( dutest __tc__ variable )
> * Tests with non-linear code paths (branches) are more painful to express in
> doctests
>
that's a fact , not just branches , but also exceptions
Beyond this ...
My really short answer is that I do not agree with this . Like I just
said in previous messages with enhanced support like the one offered
by dutest (i.e. __tc__ global var bound to an instance of
unittest.TestCase) it's possible to invoke each and every unittest
assertion method . So this may be seen all the other way round
«unittest machinery is already used without even declaring a single
test class» ... and so on ...
... so , in concept , there is no real benefit in using unittest over
doctest *if* doctest module is eventually upgraded .
[...]
>
> However doctests absolutely rock for testing documentation / docstring
> examples.
>
FWIW , +1
[...]
.. [1] doctest.REPORT_ONLY_FIRST_FAILURE
(http://docs.python.org/2/library/doctest.html#doctest.REPORT_ONLY_FIRST_FAILURE)
--
Regards,
Olemis.
Apache™ Bloodhound contributor
http://issues.apache.org/bloodhound
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
