Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 5/20/13, Mark Janssen dreamingforw...@gmail.com wrote: * Doctests practically beg you to write your code first and then copy and paste terminal sessions - they're the enemy of TDD Of course , not , all the opposite . If the approach is understood correctly then the first thing test author will do is to write the code «expected» to get something done . When everything is ok with API code style then write the code . Many problems in the API and inconsistencies are thus detected early . Now all we need is a test() built-in, a companion to help() and we have the primo platform for doctest-code-test cycle for TDD and agile development. «test() built-in» , interesting observation ... at least to me setup.py test is more than enough in real-life , and I guess many people really involved in building APIs for sure will notice that in real life it's not as simple as «doctest-code-test» ; in the same way that TDD is not always exactly like what is read in the books . However writing doctests first for APIs could definitely be helpful to think in advance in terms of the clients , especially when there are some aspects looking a bit fuzzy . Nevertheless , what is really needed , like I've been saying (elsewhere) since years ago , is a better doctest module . The API in stdlib does not offer the means to really benefit of its potential (= that does not mean it's bad , it might be better ;) . Above I was talking about testing libraries defining APIs . In the meantime following the approach sketched above , it's been possible (at least to me) to develop tested documented RESTful + RPC APIs with relatively little effort . Besides , the differences between RPC and functions due to subtle technological implementation details may be erased . Using the approach I've sketched in previous messages it's also possible to run the very same doctests for APIs that are meant to work transparently locally or hosted online (e.g. pastebins ... or other services in the cloud) . The only thing needed is to use the right implementation of doctests setUp / tearDown e.g. switching from Python functions to ServerProxy , or REST , or ... ... so , yes , it's proven to be useful in practice ... -- Regards, Olemis. Apache™ Bloodhound contributor http://issues.apache.org/bloodhound Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Mon, May 20, 2013 at 1:51 AM, Gregory P. Smith g...@krypto.org wrote: On May 19, 2013 4:31 PM, Benjamin Peterson benja...@python.org wrote: 2013/5/19 Gregory P. Smith g...@krypto.org: Idea: I don't believe anybody has written a fixer for lib2to3 that applies fixers to doctests. That'd be an interesting project for someone. 2to3 can operate on doctests, though it doesn't do anything different to them than it does to normal sourcecode. Oh cool. I didn't realize that already existed! It won't change any output, though, which still means that they tend to break. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Sat, 18 May 2013 23:41:59 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. There are other reasons to hate doctest, such as the obnoxious error reporting. Having to wade through ten pages of output to find what went wrong is no fun. Also the difficulty of editing them. For some reason, my editor doesn't offer me facilities to edit interactive prompt session snippets. All in all, I try hard to ignore any doctest present in the Python test suite. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 07:22 PM, Mark Janssen wrote: On Sun, May 19, 2013 at 1:13 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? I think it's rather pointless to do bitwise operations on python enums. We're not that close to the machine. What, nobody uses Python to do networking? How abaout driving the GPIO on a RaspberryPI? Using the bitwise operators to compbine named flag values seems like a pretty natural fit to me (if you don't care about the specific values, you don't need IntEnum anyway). Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGaEDwACgkQ+gerLs4ltQ5eXACfTrmegJsYDvbuwrbr5zyjwWV+ jMUAoIHQBi/qkm+MClGeh/ZwWOUGCMFm =4ey/ -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 20/05/13 20:45, Antoine Pitrou wrote: On Sat, 18 May 2013 23:41:59 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. There are other reasons to hate doctest, such as the obnoxious error reporting. Having to wade through ten pages of output to find what went wrong is no fun. Ten pages of broken unit tests are no picnic either. If you have ten pages of failures, then it doesn't matter *what* testing framework you use, you're going to have a bad time. But personally, I find doc test error reports perfectly clear and readable, and not overly verbose. File test.py, line 4, in __main__ Failed example: len(abcd) Expected: 24 Got: 4 That's even simpler than a traceback. Also the difficulty of editing them. For some reason, my editor doesn't offer me facilities to edit interactive prompt session snippets. Your text editor doesn't allow you to edit text? Even Notepad allows that! Seriously, what editor are you using that doesn't allow you to edit pasted snippets? -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Mon, 20 May 2013 23:32:10 +1000 Steven D'Aprano st...@pearwood.info wrote: On 20/05/13 20:45, Antoine Pitrou wrote: On Sat, 18 May 2013 23:41:59 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. There are other reasons to hate doctest, such as the obnoxious error reporting. Having to wade through ten pages of output to find what went wrong is no fun. Ten pages of broken unit tests are no picnic either. You didn't understand the objection. I'm talking about *one* broken doctest in a sea of non-broken ones. For some reason doctest (or its unittest driver) insists on either displaying everything, or nothing. It doesn't only print the errors and leave the rest silent. Also the difficulty of editing them. For some reason, my editor doesn't offer me facilities to edit interactive prompt session snippets. Your text editor doesn't allow you to edit text? Even Notepad allows that! Seriously, what editor are you using that doesn't allow you to edit pasted snippets? I don't know if you're intentionally being stupid. Of course I can edit them *by hand*. But I'll have to re-create by hand the various artifacts of an interpreter session, e.g. the prompts. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Mon, 20 May 2013 12:45:57 +0200, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 18 May 2013 23:41:59 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. There are other reasons to hate doctest, such as the obnoxious error reporting. Having to wade through ten pages of output to find what went wrong is no fun. That's why I added the 'failfast' option to doctest. Also the difficulty of editing them. For some reason, my editor doesn't offer me facilities to edit interactive prompt session snippets. I don't have much problem with lacking tailored facilities for this in vim. I suppose that is a matter of personal style. I *would* like to teach it the proper indentation, but I haven't been bothered enough yet to do it. (After all, weren't you the one who told me the lack of tab key indentation at the interactive prompt after you enabled completion by default wasn't an issue because one could just use space to indent? :) --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Mon, 20 May 2013 09:37:32 -0400
R. David Murray rdmur...@bitdance.com wrote:
On Mon, 20 May 2013 12:45:57 +0200, Antoine Pitrou solip...@pitrou.net
wrote:
On Sat, 18 May 2013 23:41:59 -0700
Raymond Hettinger raymond.hettin...@gmail.com wrote:
We should continue to encourage users to make thorough unit tests
and to leave doctests for documentation. That said, it should be
recognized that some testing is better than no testing. And doctests
may be attractive in that regard because it is almost effortless to
cut-and-paste a snippet from the interactive prompt. That isn't a
best practice, but it isn't a worst practice either.
There are other reasons to hate doctest, such as the obnoxious
error reporting. Having to wade through ten pages of output to find
what went wrong is no fun.
That's why I added the 'failfast' option to doctest.
I didn't know that. Is it propagated by regrtest? I never use doctest
standalone.
Also the difficulty of editing them. For some reason, my editor doesn't
offer me facilities to edit interactive prompt session snippets.
I don't have much problem with lacking tailored facilities for this
in vim. I suppose that is a matter of personal style. I *would* like to
teach it the proper indentation, but I haven't been bothered enough yet
to do it. (After all, weren't you the one who told me the lack of tab
key indentation at the interactive prompt after you enabled completion
by default wasn't an issue because one could just use space to indent? :)
An interpreter prompt session is throwaway, so you can pretty much
indent as you like (which may not be very pretty in a tests file).
Besides, I was thinking about the prompts (' ' and '... '), not the
indentation itself.
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Mon, 20 May 2013 15:57:35 +0200, Antoine Pitrou solip...@pitrou.net wrote:
On Mon, 20 May 2013 09:37:32 -0400
R. David Murray rdmur...@bitdance.com wrote:
On Mon, 20 May 2013 12:45:57 +0200, Antoine Pitrou solip...@pitrou.net
wrote:
On Sat, 18 May 2013 23:41:59 -0700
Raymond Hettinger raymond.hettin...@gmail.com wrote:
We should continue to encourage users to make thorough unit tests
and to leave doctests for documentation. That said, it should be
recognized that some testing is better than no testing. And doctests
may be attractive in that regard because it is almost effortless to
cut-and-paste a snippet from the interactive prompt. That isn't a
best practice, but it isn't a worst practice either.
There are other reasons to hate doctest, such as the obnoxious
error reporting. Having to wade through ten pages of output to find
what went wrong is no fun.
That's why I added the 'failfast' option to doctest.
I didn't know that. Is it propagated by regrtest? I never use doctest
standalone.
I don't think so. That's a good idea, though.
Also the difficulty of editing them. For some reason, my editor doesn't
offer me facilities to edit interactive prompt session snippets.
I don't have much problem with lacking tailored facilities for this
in vim. I suppose that is a matter of personal style. I *would* like to
teach it the proper indentation, but I haven't been bothered enough yet
to do it. (After all, weren't you the one who told me the lack of tab
key indentation at the interactive prompt after you enabled completion
by default wasn't an issue because one could just use space to indent? :)
An interpreter prompt session is throwaway, so you can pretty much
indent as you like (which may not be very pretty in a tests file).
Besides, I was thinking about the prompts (' ' and '... '), not the
indentation itself.
True. I don't find typing or ... very burdensome, though. Less
even than fixing the alignment after hitting tab :)
--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On May 18, 2013, at 11:41 PM, Raymond Hettinger wrote: I'm hoping that core developers don't get caught-up in the doctests are bad meme. Thanks for your message Raymond. I know that doctests are controversial, but I do firmly believe that when used correctly, they have value and should not be broken without careful consideration. You make excellent points about Python 3 adoption and the canary-like nature of doctests. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On May 19, 2013, at 07:28 PM, Tim Peters wrote: But more than just one ;-) Another great use has nothing to do with docstrings: using an entire file as a doctest. This encourages writing lots of text explaining what you're doing,. with snippets of code interspersed to illustrate that the code really does behave in the ways you've claimed. Agreed! I love separate-file doctests, and the marriage of doctests and reST/Sphinx is just fantastic. It's a pleasure to write usage documentation that contains code samples that are guaranteed to work, because they pass their doctest. (I personally don't like long-winded docstring doctests because they are harder to edit and distract from the code, but YMMV.) Several years ago, I spent some time experimenting with using doctest for *everything*. I deliberately wanted to go that extreme in order to better explore where doctests are good and where they're not so good. A general rule of thumb I came up with is that reST-style doctests are great for explanations involving mostly good-path usage of a library, or IOW this is how you're supposed to use this API, and see it works!. IME, doctests are not so good at testing all the failure modes, odd corner cases, and the perhaps less-common good-path use cases. Fortunately, we have another useful tool for testing that stuff wink. I'd rather encourage users to turn their brains on when writing doctest files - and when writing unit tests. I've lost count of how many times I've seen a unit test fail, then stared helplessly at the unit test code just trying to figure out what the author thought they were doing. A lot of comments in the test code could have helped that, but - outside of doctest-based tests - there's typically very little explanatory text in testing code. +1 A rule-of-thumb I use is what I call the FORTH rule[1]. If you should be able to understand what your own test is trying to accomplish a week later, otherwise you're not writing very good tests. ;) -Barry [1] or PERL rule maybe, depending on the unit of time. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On May 19, 2013, at 04:27 PM, Gregory P. Smith wrote: Idea: I don't believe anybody has written a fixer for lib2to3 that applies fixers to doctests. That'd be an interesting project for someone. I'm not sure that's true. I don't use 2to3 anymore if I can help it, but I'm pretty sure you can 2to3 your doctests too. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 20/05/13 23:38, Antoine Pitrou wrote: On Mon, 20 May 2013 23:32:10 +1000 Steven D'Aprano st...@pearwood.info wrote: On 20/05/13 20:45, Antoine Pitrou wrote: On Sat, 18 May 2013 23:41:59 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. There are other reasons to hate doctest, such as the obnoxious error reporting. Having to wade through ten pages of output to find what went wrong is no fun. Ten pages of broken unit tests are no picnic either. You didn't understand the objection. I'm talking about *one* broken doctest in a sea of non-broken ones. For some reason doctest (or its unittest driver) insists on either displaying everything, or nothing. It doesn't only print the errors and leave the rest silent. It sounds like you are inadvertently calling doctest with the verbose option. It is not standard behaviour to display everything or nothing. Here is the output from 1 failing test out of 112, with absolutely nothing edited. [steve@ando ~]$ python test.py ** File test.py, line 224, in __main__ Failed example: len(abcd) Expected: 24 Got: 4 ** 1 items had failures: 1 of 112 in __main__ ***Test Failed*** 1 failures. If I had any criticism of doctest, it would be that by default it prints nothing at all if all tests pass. I hate that, ever since I had a bunch of doctests that for about a week I thought were passing when in fact they weren't running at all. So now I always write something like this: if __name__ == '__main__': import doctest failed, tried = doctest.testmod() if failed == 0: print(Successfully ran %d tests % tried) -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Tue, 21 May 2013 02:00:32 +1000 Steven D'Aprano st...@pearwood.info wrote: On 20/05/13 23:38, Antoine Pitrou wrote: On Mon, 20 May 2013 23:32:10 +1000 Steven D'Aprano st...@pearwood.info wrote: On 20/05/13 20:45, Antoine Pitrou wrote: On Sat, 18 May 2013 23:41:59 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. There are other reasons to hate doctest, such as the obnoxious error reporting. Having to wade through ten pages of output to find what went wrong is no fun. Ten pages of broken unit tests are no picnic either. You didn't understand the objection. I'm talking about *one* broken doctest in a sea of non-broken ones. For some reason doctest (or its unittest driver) insists on either displaying everything, or nothing. It doesn't only print the errors and leave the rest silent. It sounds like you are inadvertently calling doctest with the verbose option. It is not standard behaviour to display everything or nothing. Well, I never run doctest directly, I use regrtest (there are some doctests in the standard library). So perhaps the blame lies on regrtest or on the unittest adapter, my bad. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
Hi ! :) I'll be replying some individual messages in this thread in spite of putting my replies in the right context . Sorry if I repeat something , or this makes the thread hard to read . Indeed , IMHO this is a subject suitable to discuss in TiP ML . On 5/19/13, Gregory P. Smith g...@krypto.org wrote: On Sat, May 18, 2013 at 11:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On May 14, 2013, at 9:39 AM, Gregory P. Smith g...@krypto.org wrote: Bad: doctests. I'm hoping that core developers don't get caught-up in the doctests are bad meme. So long as doctests insist on comparing the repr of things being the number one practice that people use when writing them there is no other position I can hold on the matter. reprs are not stable and never have been. ordering changes, hashes change, ids change, pointer values change, wording and presentation of things change. none of those side effect behaviors were ever part of the public API to be depended on. «Bad doctests» slogan is not positive because the subliminal message for new users is «there's something wrong with that ... let's better not use it» . IMHO that's not true ; doctest is an incredibly awesome testing framework for delta assertions and there is nothing wrong with the philosophy behind that module and its intent . This surfaces an issue I've noticed years ago wrt doctest module (so, yes , it's obvious there's an issue ;) . The way I see it this is more about the fact that module frontend does not offer the means to benefit from all the possibilities of doctest classes in the backend (e.g. output checkers , doctest runners, ...) That one can write doctests that don't depend on such things as the repr doesn't ultimately matter because the easiest thing to do, as encouraged by examples that are pasted from an interactive interpreter session into docs, is to have the interactive interpreter show the repr and not add code to check things in a accurate-for-testing manner that would otherwise make the documentation harder for a human to read. This is something that could be easily mitigated by a custom output checker . In the end , in docs there is no difference between output messages like 'Spam object at 0xb7cf5d70' or 'Spam object at 0x1' (i.e. some deterministic label like computed hex number or anything else ...) . You might also avoid printing repr(s) Instead, we should be clear about their primary purpose which is to test the examples given in docstrings. In many cases, there is a great deal of benefit to docstrings that have worked-out examples (see the docstrings in the decimal module for example). In such cases it is also worthwhile to make sure those examples continue to match reality. Doctests are a vehicle for such assurance. In other words, doctests have a perfectly legitimate use case. I really do applaud the goal of keeping examples in documentation up to date. But doctest as it is today is the wrong approach to that. A repr mismatch does not mean the example is out of date. ... and I confess I never use doctest «as it is today» in stdlib . So , you are right . We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. Not quite, they at least tested something (yay!) but it is uncomfortably close to a worst practice. I disagree . IMO what is a bad practice is to spread the rumor that «doctests are evil» rather than saying «doctest module has limitations» It means someone else needs to come understand the body of code containing this doctest when they make an unrelated change that triggered a behavior change as a side effect that the doctested code may or may not actually depend on but does not actually declare its intent one way or another for the purposes of being a readable example rather than accurate test. I see no problem in keeping both these aspects . bikeshed colors: If doctest were never called a test but instead were called docchecker to not imply any testing aspect No way ! ( IMHO ) I just wrote dutest [1]_ framework , built on top of doctest and unittest , that does the following (among other things) : 1. Implements unittest loaders for doctests 2. Allows for customizing output checkers , doctest runners , ... anything you might find in the backend * For instance , replacing default test runner and output checkers might be useful to write delta assertions for command-line scripts 3. Tightly integrated with unittest (e.g. custom TestSuite(s) ...) 4. Access to unittest test case in special __tc__ variable , so all known assertion methods are handy ootb 5. Encapsulate
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
-- Forwarded message -- From: Olemis Lang ole...@gmail.com Date: Mon, 20 May 2013 11:33:42 -0500 Subject: Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum] To: Antoine Pitrou solip...@pitrou.net On 5/20/13, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 18 May 2013 23:41:59 -0700 Raymond Hettinger raymond.hettin...@gmail.com wrote: We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. There are other reasons to hate doctest, such as the obnoxious error reporting. Having to wade through ten pages of output to find what went wrong is no fun. +1 FWIW , while using dutest [1]_ each interactive example will be a test case and therefore the match for that particular assertion will be reported using the usual unittest output format .. [1] dutest (https://pypi.python.org/pypi/dutest) -- Regards, Olemis. Apache™ Bloodhound contributor http://issues.apache.org/bloodhound Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
I'm hoping that core developers don't get caught-up in the doctests are bad meme. Instead, we should be clear about their primary purpose which is to test the examples given in docstrings. In other words, doctests have a perfectly legitimate use case. But more than just one ;-) Another great use has nothing to do with docstrings: using an entire file as a doctest. This encourages writing lots of text explaining what you're doing,. with snippets of code interspersed to illustrate that the code really does behave in the ways you've claimed. +1, very true. I think doctest excel in almost every way above UnitTests. I don't understand the popularity of UnitTests, except perhaps for GUI testing which doctest can't handle. I think people just aren't very *imaginative* about how to create good doctests that are *also* good documentation. That serves two very good purposes in one. How can you beat that? The issues of teardown and setup are fixable and even more beautifully solved with doctests -- just use the lexical scoping of the program to determine the execution environment for the doctests. picking-your-poison-ly y'rs - tim Cheers, Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 5/19/13, Steven D'Aprano st...@pearwood.info wrote: On 20/05/13 09:27, Gregory P. Smith wrote: On Sat, May 18, 2013 at 11:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On May 14, 2013, at 9:39 AM, Gregory P. Smith g...@krypto.org wrote: Bad: doctests. I'm hoping that core developers don't get caught-up in the doctests are bad meme. So long as doctests insist on comparing the repr of things being the number one practice that people use when writing them there is no other position I can hold on the matter. reprs are not stable and never have been. I think this *massively* exaggerates the problem with doc tests. I agree , and it is a negative influence for beginners . I make heavy use of them, and have no problem writing doc tests that work in code running over multiple versions, including from 2.4 through 3.3. Objects that I write myself, I control the repr and can make it as stable as I wish. Many built-in types also have stable reprs. The repr for small ints is not going to change, the repr for floats like 0.5, 0.25, 0.125 etc. are stable and predictable, lists and tuples and strings all have stable well-defined reprs. Dicts are a conspicuous counter-example, but there are trivial work-arounds. +1 Doc tests are not limited to a simple-minded compare the object's repr. Yes You can write as much, or as little, scaffolding around the test as you need. If the scaffolding becomes too large, that's a sign that the test doesn't belong in documentation and should be moved out, perhaps into a unit test, or perhaps into a separate literate testing document that can be as big as necessary without overwhelming the doc string. There is an alternate approach related to a feature of dutest [1]_ I mentioned in a previous message (i.e. doctests setUp and tearDown methods) . The main reason to desire to leave long doctests scaffolding code out (e.g. loading a Trac environment, or setting up a separate Python virtual environment , subversion repository , ... as part of -unit, functional, ...- test setup ) is to focus on SUT / API details , avoid repetition of some steps , and keep tests readable . This code is moved to underlying unittest setUp method and it's still possible to write readable doctests for the particular feature of the SUT . In general there's a need to find a balance to decide what should be «hidden» in doctests fixture methods and what should be written in doctests . Based on my experience there's no benefit in using unittest over doctests unittests : - are unreadable - require knowledge of XUnit , etc ... - Writing complex assertions might be hard and tedious doctests: - are extremely readable - anybody familiar with the SUT could write tests - especially for modules that are meant to be used by persons who are not (professional / skilled) software developers encapsulating the use of a testing framework is a plus ; your test suite is «talking in users language» (/me not sure about stdlib ...) ordering changes, hashes change, ids change, pointer values change, wording and presentation of things change. none of those side effect behaviors were ever part of the public API to be depended on. Then don't write doctests that depend on those things. It really is that simple. There's no rule that says doctests have to test the entire API. Doctests in docstrings are *documentation first*, so you write tests that make good documentation. ... but someone could do so , if it wasn't by the current limitations of doctest frontend . ;) The fact that things that are not stable parts of the API can be tested is independent of the framework you use to do the testing. If I, as an ignorant and foolish developer, wrote a unit test like this: class MyDumbTest(unittest.TestCase): def testSpamRepr(self): x = Spam(arg) self.assertEquals(repr(x), Spam object at 0x123ab) we shouldn't conclude that unit tests are bad, but that MyDumbTest is bad and needs to be fixed. +1 [...] And that's great, it really is, I'm not being sarcastic. But unit testing is not in competition to doc testing, they are complimentary, not alternatives. If you're not using both, then you're probably missing out on something. +1 PS: ... and well , this would be my last message about dutest and how it improves upon what's offered by doctest module ... Summarizing : «Bad doctests» is not a cool statement .. [1] dutest @ PyPI (https://pypi.python.org/pypi/dutest) -- Regards, Olemis. Apache™ Bloodhound contributor http://issues.apache.org/bloodhound Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 20 May 2013, at 18:26, Mark Janssen dreamingforw...@gmail.com wrote: I'm hoping that core developers don't get caught-up in the doctests are bad meme. Instead, we should be clear about their primary purpose which is to test the examples given in docstrings. In other words, doctests have a perfectly legitimate use case. But more than just one ;-) Another great use has nothing to do with docstrings: using an entire file as a doctest. This encourages writing lots of text explaining what you're doing,. with snippets of code interspersed to illustrate that the code really does behave in the ways you've claimed. +1, very true. I think doctest excel in almost every way above UnitTests. I don't understand the popularity of UnitTests, except perhaps for GUI testing which doctest can't handle. I think people just aren't very *imaginative* about how to create good doctests that are *also* good documentation. Doc tests have lots of problems for unit testing. * Every line is a test with *all* output part of the test - in unit tests you only assert the specific details you're interested in * Unordered types are a pain with doctest unless you jump through hoops * Tool support for editing within doctests is *generally* worse * A failure on one line doesn't halt execution, so you can get many many reported errors from a single failure * Try adding diagnostic prints and then running your doctests! * Tools support in terms of test discovery and running individual tests is not as smooth * Typing and ... all the time is really annoying * Doctests practically beg you to write your code first and then copy and paste terminal sessions - they're the enemy of TDD * Failure messages are not over helpful and you lose the benefit of some of the new asserts (and their diagnostic output) in unittest * Tests with non-linear code paths (branches) are more painful to express in doctests and so on... However doctests absolutely rock for testing documentation / docstring examples. So I'm with Raymond on this one. All the best, Michael That serves two very good purposes in one. How can you beat that? The issues of teardown and setup are fixable and even more beautifully solved with doctests -- just use the lexical scoping of the program to determine the execution environment for the doctests. picking-your-poison-ly y'rs - tim Cheers, Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
I don't think a python-dev discussion about the value of doctests is going to change minds one way or the other, but I just *had* to respond to this one point: On May 20, 2013, at 11:26 PM, Michael Foord wrote: * Doctests practically beg you to write your code first and then copy and * paste terminal sessions - they're the enemy of TDD In a sense, they're your best friend too. Countless times, when I'm designing an API, writing the documentation first helps clarify how I want the library to work, or where I need to think about the API more deeply. In much the same way that TDD is ideal when you know what you're aiming for, when you *don't* exactly know, it's a huge benefit to write the documentation first. Doing so will bring into stark contrast what needs improvement in your API. The fact that you can then test much of this documentation as you go, brings the win of TDD to your documentation. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 5/19/2013 9:08 PM, Ethan Furman wrote: On 05/19/2013 05:24 PM, Nick Coghlan wrote: This is the point I was trying to make: once you use IntEnum (as you would in any case where you need bitwise operators), Enum gets out of the way for everything other than __str__, __repr__, and one other slot (that escapes me for the moment...). __getnewargs__ and __new__ But if you do math, the result is no longer an Enum of any type. And thus completely loses the debugging benefits of having a nice __repr__. IntEnum isn't useful for bitfields. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
Hi ! ... sorry , I could not avoid to reply this message ... On 5/20/13, Michael Foord fuzzy...@voidspace.org.uk wrote: On 20 May 2013, at 18:26, Mark Janssen dreamingforw...@gmail.com wrote: I'm hoping that core developers don't get caught-up in the doctests are bad meme. Instead, we should be clear about their primary purpose which is to test the examples given in docstrings. In other words, doctests have a perfectly legitimate use case. But more than just one ;-) Another great use has nothing to do with docstrings: using an entire file as a doctest. This encourages writing lots of text explaining what you're doing,. with snippets of code interspersed to illustrate that the code really does behave in the ways you've claimed. +1, very true. I think doctest excel in almost every way above UnitTests. I don't understand the popularity of UnitTests, except perhaps for GUI testing which doctest can't handle. I think people just aren't very *imaginative* about how to create good doctests that are *also* good documentation. With enhanced doctests solution in mind ... Doc tests have lots of problems for unit testing. * Every line is a test with *all* output part of the test - in unit tests you only assert the specific details you're interested in custom output checkers * Unordered types are a pain with doctest unless you jump through hoops ( custom output checkers + doctest runner ) | (dutest __tc__ global var) * Tool support for editing within doctests is *generally* worse this is true , let's do it ! * A failure on one line doesn't halt execution, so you can get many many reported errors from a single failure it should if REPORT_ONLY_FIRST_FAILURE option [1]_ is set . * Try adding diagnostic prints and then running your doctests! I have ... dutest suites for my Trac plugins do so . However logging information is outputted to /path/to/trac/env/log/trac.log ... so a tail -f is always handy . * Tools support in terms of test discovery and running individual tests is not as smooth dutest offers two options since years ago MultiTestLoader combines multiple test loaders to *load* different kinds of tests at once from a module , whereas a package loader performs test discovery . These loader objects are composable , so if an instance of MultiTestLoader is supplied in to the package test loader then multiple types of tests are loaded out of modules all over across the package hierarchy . Indeed , in +10 years of Python development I've never used unittest(2) discovery, and even recently implemented the one that's used in Apache™ Bloodhound test suite . Unfortunately I've had no much time to spend on improving all this support in dutest et al. * Typing and ... all the time is really annoying ... I have faith ... there should be something like this for vim ... I have faith ... ;) * Doctests practically beg you to write your code first and then copy and paste terminal sessions - they're the enemy of TDD Of course , not , all the opposite . If the approach is understood correctly then the first thing test author will do is to write the code «expected» to get something done . When everything is ok with API code style then write the code . Many problems in the API and inconsistencies are thus detected early . * Failure messages are not over helpful and you lose the benefit of some of the new asserts (and their diagnostic output) in unittest (custom ouput checkers) | ( dutest __tc__ variable ) * Tests with non-linear code paths (branches) are more painful to express in doctests that's a fact , not just branches , but also exceptions Beyond this ... My really short answer is that I do not agree with this . Like I just said in previous messages with enhanced support like the one offered by dutest (i.e. __tc__ global var bound to an instance of unittest.TestCase) it's possible to invoke each and every unittest assertion method . So this may be seen all the other way round «unittest machinery is already used without even declaring a single test class» ... and so on ... ... so , in concept , there is no real benefit in using unittest over doctest *if* doctest module is eventually upgraded . [...] However doctests absolutely rock for testing documentation / docstring examples. FWIW , +1 [...] .. [1] doctest.REPORT_ONLY_FIRST_FAILURE (http://docs.python.org/2/library/doctest.html#doctest.REPORT_ONLY_FIRST_FAILURE) -- Regards, Olemis. Apache™ Bloodhound contributor http://issues.apache.org/bloodhound Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 5/20/13, Olemis Lang ole...@gmail.com wrote: [...] On 5/20/13, Michael Foord fuzzy...@voidspace.org.uk wrote: [...] * Tool support for editing within doctests is *generally* worse this is true , let's do it ! [...] * Typing and ... all the time is really annoying ... I have faith ... there should be something like this for vim ... I have faith ... ;) FWIW ... an option could be to combine auto-completion (in the end that's yet another indentation ;) to this http://architects.dzone.com/articles/real-time-doctest-checking-vim ... and I could better enjoy my vim + python development experience ;) -- Regards, Olemis. Apache™ Bloodhound contributor http://issues.apache.org/bloodhound Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
* Doctests practically beg you to write your code first and then copy and paste terminal sessions - they're the enemy of TDD Of course , not , all the opposite . If the approach is understood correctly then the first thing test author will do is to write the code «expected» to get something done . When everything is ok with API code style then write the code . Many problems in the API and inconsistencies are thus detected early . Now all we need is a test() built-in, a companion to help() and we have the primo platform for doctest-code-test cycle for TDD and agile development. --Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On May 14, 2013, at 9:39 AM, Gregory P. Smith g...@krypto.org wrote: Bad: doctests. I'm hoping that core developers don't get caught-up in the doctests are bad meme. Instead, we should be clear about their primary purpose which is to test the examples given in docstrings. In many cases, there is a great deal of benefit to docstrings that have worked-out examples (see the docstrings in the decimal module for example). In such cases it is also worthwhile to make sure those examples continue to match reality. Doctests are a vehicle for such assurance. In other words, doctests have a perfectly legitimate use case. We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. Another meme that I hope dispel is the notion that the core developers are free to break user code (such as doctests) if they believe the users aren't coding in accordance with best practices. Our goal is to improve their lives with our modifications, not to make their lives more difficult. Currently, we face an adoption problem with Python 3. At PyCon, an audience of nearly 2500 people said they had tried Python 3 but weren't planning to convert to it in production code. All of the coredevs are working to make Python 3 more attractive than Python 2, but we also have to be careful to not introduce obstacles to conversion. Breaking tests makes it much harder to convert (especially because people need to rely on their tests to see if the conversion was successful). Raymond P.S. Breaking doctests should also be seen as a canary in a coal mine. When they break, it also means that printed examples are out of date, that code parsers may break, that diffs start being different, that programs that feed into other programs (perhaps via pipes and filters) may be changing their interface, etc. Occasionally, we make need to break such things but there should be a compelling offsetting benefit (i.e. evaluating thoughtfully whether I'm trying to help you by making your constant integers have a nicer repr is worth Sorry, I broke your tests, made your published examples out of date, and slowed down your code. -- in some modules it will be worth it, but in others we should value stability over micro-improvments). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Sun, May 19, 2013 at 4:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: nicer repr is worth Sorry, I broke your tests, made your published examples out of date, and slowed down your code. While the first two considerations are always potentially applicable when using enums, the latter should only be true for code that uses str() and repr() a lot. For other operations, int-based enums shouldn't add any more overhead than namedtuple does for tuples. I agree with basically everything you said, but I don't want enums are slower than normal integers to become a meme - there really shouldn't be a speed difference for any arithmetic operations when using IntEnum. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
Anyway, if you're doing arithmetic on enums you're doing it wrong. — Sent from Mailbox On Sun, May 19, 2013 at 4:55 AM, Nick Coghlan ncogh...@gmail.com wrote: On Sun, May 19, 2013 at 4:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: nicer repr is worth Sorry, I broke your tests, made your published examples out of date, and slowed down your code. While the first two considerations are always potentially applicable when using enums, the latter should only be true for code that uses str() and repr() a lot. For other operations, int-based enums shouldn't add any more overhead than namedtuple does for tuples. I agree with basically everything you said, but I don't want enums are slower than normal integers to become a meme - there really shouldn't be a speed difference for any arithmetic operations when using IntEnum. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Sat, May 18, 2013 at 11:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On May 14, 2013, at 9:39 AM, Gregory P. Smith g...@krypto.org wrote: Bad: doctests. I'm hoping that core developers don't get caught-up in the doctests are bad meme. Don't doctests intended for CPython not work on Jython, Pypy, IronPython... I've been avoiding them for this reason. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGZMoAACgkQ+gerLs4ltQ79qwCgq6FWTl6ZDIDctBg69In47YB2 +FkAnj5cEyw1szQ8GCl6aQ9+aGKcwp3y =d/xt -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 12:14 PM, Dan Stromberg wrote: On Sat, May 18, 2013 at 11:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On May 14, 2013, at 9:39 AM, Gregory P. Smith g...@krypto.org wrote: Bad: doctests. I'm hoping that core developers don't get caught-up in the doctests are bad meme. Don't doctests intended for CPython not work on Jython, Pypy, IronPython... I've been avoiding them for this reason. Naive doctests depend a lot on repr, and hence tend to break even between minor releases of CPython. Folks who use a lot of them apply a great deal of elbow grease to working around that problem, e.g. through renoormalizing the output: https://github.com/zopefoundation/zope.testing/blob/master/src/zope/testing/renormalizing.txt Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGZM1YACgkQ+gerLs4ltQ6zRACgx266WAzy1RDX0vOm7fThXzi5 zX4AoNyZFGBOML2XR4ZOecXwzG6XaHW+ =yGon -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 5/19/2013 4:13 PM, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? Those are logic, not arithmetic as usually understood. (The fact that one can do each with the other is beside the point.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 20 May 2013 06:25, Terry Jan Reedy tjre...@udel.edu wrote: On 5/19/2013 4:13 PM, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? Those are logic, not arithmetic as usually understood. (The fact that one can do each with the other is beside the point.) I consider those to be binary arithmetic, but it's a fair point. The word I really wanted was comparison anyway, since the main intended uses of enums are as flags, lookup keys and marker values. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Sun, May 19, 2013 at 1:13 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? I think it's rather pointless to do bitwise operations on python enums. We're not that close to the machine. MarkJ Tacoma, Washington ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Sat, May 18, 2013 at 11:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On May 14, 2013, at 9:39 AM, Gregory P. Smith g...@krypto.org wrote: Bad: doctests. I'm hoping that core developers don't get caught-up in the doctests are bad meme. So long as doctests insist on comparing the repr of things being the number one practice that people use when writing them there is no other position I can hold on the matter. reprs are not stable and never have been. ordering changes, hashes change, ids change, pointer values change, wording and presentation of things change. none of those side effect behaviors were ever part of the public API to be depended on. That one can write doctests that don't depend on such things as the repr doesn't ultimately matter because the easiest thing to do, as encouraged by examples that are pasted from an interactive interpreter session into docs, is to have the interactive interpreter show the repr and not add code to check things in a accurate-for-testing manner that would otherwise make the documentation harder for a human to read. Instead, we should be clear about their primary purpose which is to test the examples given in docstrings. In many cases, there is a great deal of benefit to docstrings that have worked-out examples (see the docstrings in the decimal module for example). In such cases it is also worthwhile to make sure those examples continue to match reality. Doctests are a vehicle for such assurance. In other words, doctests have a perfectly legitimate use case. I really do applaud the goal of keeping examples in documentation up to date. But doctest as it is today is the wrong approach to that. A repr mismatch does not mean the example is out of date. We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. That said, it should be recognized that some testing is better than no testing. And doctests may be attractive in that regard because it is almost effortless to cut-and-paste a snippet from the interactive prompt. That isn't a best practice, but it isn't a worst practice either. Not quite, they at least tested something (yay!) but it is uncomfortably close to a worst practice. It means someone else needs to come understand the body of code containing this doctest when they make an unrelated change that triggered a behavior change as a side effect that the doctested code may or may not actually depend on but does not actually declare its intent one way or another for the purposes of being a readable example rather than accurate test. bikeshed colors: If doctest were never called a test but instead were called docchecker to not imply any testing aspect that might've helped (too late? the cat's out of the bag). Or if it never compared anything but simply ran the example code to generate and update the doc examples from the statements with the current actual results of execution instead of doing string comparisons... (ie: more of an documentation example keep up to date tool) Another meme that I hope dispel is the notion that the core developers are free to break user code (such as doctests) if they believe the users aren't coding in accordance with best practices. Our goal is to improve their lives with our modifications, not to make their lives more difficult. Educating users how to apply best practices and making that easier for them every step of the way is a primary goal. Occasionally we'll have to do something annoying in the process but we do try to limit that. In my earlier message I suggested that someone improve doctest to not do dumb string comparisons of reprs. I still think that is a good goal if doctest is going to continue to be promoted. It would help alleviate many of the issues with doctests and bring them more in line with the issues many people's regular unittests have. As Tres already showed in an example, individual doctest using projects jump through hoops to do some of that today; centralizing saner repr comparisons for less false failures as an actual doctest feature just makes sense. Successful example: We added a bunch of new comparison methods to unittest in 2.7 that make it much easier to write tests that don't depend on implementation details such as ordering. Many users prefer to use those new features; even with older Python's via unittest2 on pypi. It doesn't mean users always write good tests, but a higher percentage of tests written are more future proof than they were before because it became easier. Currently, we face an adoption problem with Python 3. At PyCon, an audience of nearly 2500 people said they had tried Python 3 but weren't planning to convert to it in production code. All of the coredevs are working to make Python 3 more attractive than Python 2, but we also have to be careful to not introduce obstacles to conversion. Breaking tests makes it much harder to convert (especially because people
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
2013/5/19 Gregory P. Smith g...@krypto.org: Idea: I don't believe anybody has written a fixer for lib2to3 that applies fixers to doctests. That'd be an interesting project for someone. 2to3 can operate on doctests, though it doesn't do anything different to them than it does to normal sourcecode. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On May 19, 2013 4:31 PM, Benjamin Peterson benja...@python.org wrote: 2013/5/19 Gregory P. Smith g...@krypto.org: Idea: I don't believe anybody has written a fixer for lib2to3 that applies fixers to doctests. That'd be an interesting project for someone. 2to3 can operate on doctests, though it doesn't do anything different to them than it does to normal sourcecode. Oh cool. I didn't realize that already existed! -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 5/19/2013 7:22 PM, Mark Janssen wrote: On Sun, May 19, 2013 at 1:13 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? I think it's rather pointless to do bitwise operations on python enums. We're not that close to the machine. It makes sense if the enums represent bit-oriented values that will be used close to the machine. Python is used in many disciplines. --Ned. MarkJ Tacoma, Washington ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Sun, 19 May 2013 20:04:03 -0400 Ned Batchelder n...@nedbatchelder.com wrote: On 5/19/2013 7:22 PM, Mark Janssen wrote: On Sun, May 19, 2013 at 1:13 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? I think it's rather pointless to do bitwise operations on python enums. We're not that close to the machine. It makes sense if the enums represent bit-oriented values that will be used close to the machine. Python is used in many disciplines. Then it's up to the library writer to not use enums in that case. (assuming the performance of bitwise operations is critical here, which I doubt) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On Mon, May 20, 2013 at 10:14 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sun, 19 May 2013 20:04:03 -0400 Ned Batchelder n...@nedbatchelder.com wrote: On 5/19/2013 7:22 PM, Mark Janssen wrote: On Sun, May 19, 2013 at 1:13 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/19/2013 10:48 AM, Guido van Rossum wrote: Anyway, if you're doing arithmetic on enums you're doing it wrong. Hmm, bitwise operations, even? I think it's rather pointless to do bitwise operations on python enums. We're not that close to the machine. It makes sense if the enums represent bit-oriented values that will be used close to the machine. Python is used in many disciplines. Then it's up to the library writer to not use enums in that case. (assuming the performance of bitwise operations is critical here, which I doubt) This is the point I was trying to make: once you use IntEnum (as you would in any case where you need bitwise operators), Enum gets out of the way for everything other than __str__, __repr__, and one other slot (that escapes me for the moment...). The metaclass does extra work at definition time so there shouldn't be any runtime overhead - the slots should be inherited directly from the non-Enum parent. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
[Raymond Hettinger] I'm hoping that core developers don't get caught-up in the doctests are bad meme. Instead, we should be clear about their primary purpose which is to test the examples given in docstrings. I disagree. In many cases, there is a great deal of benefit to docstrings that have worked-out examples (see the docstrings in the decimal module for example). In such cases it is also worthwhile to make sure those examples continue to match reality. Doctests are a vehicle for such assurance. That's representative of how doctest was developed: to help me in keeping some well-defined mathematical functions working as intended. It still excels in that particular area (a few examples to illustrate normal cases, and a few more to illustrate well-defined end and error cases - and that's all there _is_ to be tested). In other words, doctests have a perfectly legitimate use case. But more than just one ;-) Another great use has nothing to do with docstrings: using an entire file as a doctest. This encourages writing lots of text explaining what you're doing,. with snippets of code interspersed to illustrate that the code really does behave in the ways you've claimed. We should continue to encourage users to make thorough unit tests and to leave doctests for documentation. I'd rather encourage users to turn their brains on when writing doctest files - and when writing unit tests. I've lost count of how many times I've seen a unit test fail, then stared helplessly at the unit test code just trying to figure out what the author thought they were doing. A lot of comments in the test code could have helped that, but - outside of doctest-based tests - there's typically very little explanatory text in testing code. picking-your-poison-ly y'rs - tim ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 20/05/13 09:27, Gregory P. Smith wrote: On Sat, May 18, 2013 at 11:41 PM, Raymond Hettinger raymond.hettin...@gmail.com wrote: On May 14, 2013, at 9:39 AM, Gregory P. Smith g...@krypto.org wrote: Bad: doctests. I'm hoping that core developers don't get caught-up in the doctests are bad meme. So long as doctests insist on comparing the repr of things being the number one practice that people use when writing them there is no other position I can hold on the matter. reprs are not stable and never have been. I think this *massively* exaggerates the problem with doc tests. I make heavy use of them, and have no problem writing doc tests that work in code running over multiple versions, including from 2.4 through 3.3. Objects that I write myself, I control the repr and can make it as stable as I wish. Many built-in types also have stable reprs. The repr for small ints is not going to change, the repr for floats like 0.5, 0.25, 0.125 etc. are stable and predictable, lists and tuples and strings all have stable well-defined reprs. Dicts are a conspicuous counter-example, but there are trivial work-arounds. Doc tests are not limited to a simple-minded compare the object's repr. You can write as much, or as little, scaffolding around the test as you need. If the scaffolding becomes too large, that's a sign that the test doesn't belong in documentation and should be moved out, perhaps into a unit test, or perhaps into a separate literate testing document that can be as big as necessary without overwhelming the doc string. ordering changes, hashes change, ids change, pointer values change, wording and presentation of things change. none of those side effect behaviors were ever part of the public API to be depended on. Then don't write doctests that depend on those things. It really is that simple. There's no rule that says doctests have to test the entire API. Doctests in docstrings are *documentation first*, so you write tests that make good documentation. The fact that things that are not stable parts of the API can be tested is independent of the framework you use to do the testing. If I, as an ignorant and foolish developer, wrote a unit test like this: class MyDumbTest(unittest.TestCase): def testSpamRepr(self): x = Spam(arg) self.assertEquals(repr(x), Spam object at 0x123ab) we shouldn't conclude that unit tests are bad, but that MyDumbTest is bad and needs to be fixed. Perhaps the fix is to re-write the test to care less about the exact repr. (Doctest's ellipsis directive is excellent for that.) Perhaps the fix is to give the Spam object a stable repr that doesn't suck. Or perhaps the fix is to just say, this doesn't need to be a test at all. (And doctest has a directive for that too.) They are all good solutions to the problem of unit testing things that aren't part of the API, and they are also good solutions to the same problem when it comes to doctests. [...] I really do applaud the goal of keeping examples in documentation up to date. But doctest as it is today is the wrong approach to that. A repr mismatch does not mean the example is out of date. No, it means that either the test was buggy, or the test has failed. I must admit that I don't understand what you think happens with doc testing in practice. You give the impression that there are masses of doc tests being written that look like this: x = Spam(arg) print(x) Spam object at 0xb7cf5d70 and therefore the use of doc tests are bad because it leads to broken tests. But I don't understand why you think that nobody has noticed that this test will have failed right from the start, and will have fixed it immediately. I suppose it is possible that some people write doc tests but never run them, not even once, but that's no different from those who write unit tests but never run them. They're hardly representative of the average developer, who either doesn't write tests at all, or who both writes and runs them and will notice if they fail. [...] In my earlier message I suggested that someone improve doctest to not do dumb string comparisons of reprs. I still think that is a good goal if doctest is going to continue to be promoted. It would help alleviate many of the issues with doctests and bring them more in line with the issues many people's regular unittests have. As Tres already showed in an example, individual doctest using projects jump through hoops to do some of that today; centralizing saner repr comparisons for less false failures as an actual doctest feature just makes sense. If a test needs to jump through hoops to work, then it doesn't belong as a test in the doc string. It should be a unit test, or possibly a separate test file that can be as big and complicated as needed. If you want to keep it as an example, but not actually run it, doctest has a skip directive. There's no need to complicate doctest by making it smarter (and therefore more
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
Gregory P. Smith writes: I really do applaud the goal of keeping examples in documentation up to date. But doctest as it is today is the wrong approach to that. A repr mismatch does not mean the example is out of date. Of course it does. The user sees something in the doc that's different from what his interpreter tells him. That may not bother a long-time user of the module, or one who hangs out on python-commits (uh-huh, uh-huh, yeah, right), but it worries new ones, and it should. What else may have changed? is what they should be thinking. Also, there are many cases where the output of a function is defined by some external protocol: XML, JSON, RFC , etc. Here doctests are very valuable. There are lots of testing applications where doctests suck. There are lots of testing applications where doctests are pretty much the optimal balance between ease of creation and ease of maintenance. I wouldn't be surprised if there are applications (RAD?) where *creating* tests as doctests and *converting* to a more precisely-specified framework in maintenance is best practice. Maybe somebody (not me, I do far too little testing even with doctests :-( ) should write an informational PEP. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Purpose of Doctests [Was: Best practices for Enum]
On 05/19/2013 05:24 PM, Nick Coghlan wrote: This is the point I was trying to make: once you use IntEnum (as you would in any case where you need bitwise operators), Enum gets out of the way for everything other than __str__, __repr__, and one other slot (that escapes me for the moment...). __getnewargs__ and __new__ But if you do math, the result is no longer an Enum of any type. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
