On Mon, Mar 28, 2011 at 5:27 PM, Martin Aspeli <optilude+li...@gmail.com> wrote: > On 28 March 2011 15:45, Tres Seaver <tsea...@palladion.com> wrote: >> The vast majority of the doctest testcases in zope.* packages fall into >> this category: poor isolation, lots of edge cases which would obscure >> any real narrative docs, of which there are almost none. I believe the >> conflict is intrinsic, here, and not just an accident of careless / >> naive implementation: exercising all the preconditions of the contract >> of the function-under-test makes for really poor documentation, but is >> essential to good testing. > > Agree. Here's what I've found and learned the hard way: > > doctests are sometimes easier / more fun to write the *first* time I > write tests. It's easy to just start writing, and the narrative helps > me think about what I'm doing.
Yup. > Plus, it feels like I'm saving time on > writing docs. And it doesn't because tests != docs. I've found that docs need to be written from an entirely different point of view. > They are much worse all subsequent times. Maintenance becomes a soup. I've only found this to be the case if all tests are put in the same document. Keep in mind that unit tests can be a maintenance disaster too. I speak from experience. Some key factors in maintainability: - Is the intent clear. Unit tests don't do anything to make intent clear. Doctests don't force you to make intent clear. - Coupling between different tests. Arguably doctests encourage this by making it easy to glom many unrelated tests into the same document, but this can and does happen with unit tests as well, especially if a lot of setup is needed. > Quick tests for edge cases feel like they obscure the narrative, Yup, they do! > so I may forgo them. Which is a mistake. You should create separate tests. I typically put large tests, dealing with main use cases where there is a definite flow of activity in '.test' files. I do these in separate files because they're easier to write that way. I use a '.test' suffix to avoid the pretense that these are documentation. I put edge-case tests in small docstrings in testing modules. I'm not really religious about using doctests for this, but I find small edge-case doctests easier to read than traditional unit tests. It's possible that I'd like py.test tests as much. > Refactoring is painful. It all depends on how well the tests are written and this applies equally no matter what format is used. > Tool support is poorer, True ... > And people find the docs > underwhelming. tests != docs This was a lesson learned. > If I'm doing it wrong, I'd like to see it done "right". You might check out the bobo and zc.ngi docs and tests and let me know what you think. > Manuel is kind > of cool, but I'm not sure it really addresses the issue. It's a better > doctest, not a better unittest. You may be missing the point. When writing docs, the primary goal is to guide the user to understanding as effectively as possible. The focus is on the user and their goals. A secondary, but important goal is making sure what you say in documentation is true. Manuel helps with the second goal without compromising the first goal. When writing tests, the focus is on the software and assuring that it is working correctly. Manuel can help here too by making some things easier to express and by being a better doctests, but the main goal of manuel is to enable writing good documentation that is still executable. > One of the main objections to unittest(2) seems to be that it's > "Javaesque". Yes, JUnit was the first library to solidify many of the > current, cross-language testing patterns. We shouldn't be so > NIH-stricken that we can't see the benefit of being consistent with > patterns used in most other languages and test frameworks these days. It's not about NIH, it's about ceremony. Java and it's culture are all about ceremony that just get in the way. py.test and (modern uses of) doctest are in many ways a reaction to that ceremony. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )