On Tue, 29 Aug 2017 12:25:45 +1000, Chris Angelico wrote: > On Tue, Aug 29, 2017 at 11:53 AM, Steve D'Aprano > <steve+pyt...@pearwood.info> wrote: >> (1) Disable doctesting for that example, and treat it as just >> documentation: >> >> def my_thing(): >> """blah blah blah >> >> >>> my_thing() #doctest:+SKIP >> 4 >> >> """ > > For a lot of functions, this completely destroys the value of > doctesting.
"The" value? Doc tests have two values: documentation (as examples of use) and as tests. Disabling the test aspect leaves the value as documentation untouched, and arguably is the least-worst result. You can always write a unit test suite to perform more detailed, complicated tests. Doc tests are rarely exhaustive, so you need unit tests as well. >> (2) Monkey-patch the random module for testing. This is probably the >> worst idea ever, but it's an idea :-) >> >> That makes for a fragile test and poor documentation. > > This highlights the inherent weakness of doctests. For proper unit > testing, I would definitely recommend this. Maybe a hybrid of 1 and 2 > could be organized... hmm. Doc tests should be seen as *documentation first* and tests second. The main roll of the tests is to prove that the documented examples still do what you say they do. It makes for a horrible and uninformative help() experience to have detailed, complex, exhaustive doc tests exercising every little corner case of your function. That should go in your unit tests. Possibly relevant: the unittest module has functionality to automatically extract and run your library's doctests, treating them as unit tests. So you can already do both. >> (3) Write your functions to take an optional source of randomness, and >> then in your doctests set them: >> >> def my_thing(randint=None): >> """blah blah blah >> >> >>> my_thing(randint=lambda a,b: 4) >> 4 >> >> """ >> if randint is None: >> from random import randint >> ... > > Unless that would be useful for other reasons, not something I like > doing. Having code in your core that exists solely (or even primarily) > to make testing easier seems like doing things backwards. I see your point, and I don't completely disagree. I'm on the fence about this one. But testing is important, and we often write code to make testing easier, e.g. pulling out a complex but short bit of code into its own function so we can test it, using dependency injection, etc. Why shouldn't we add hooks to enable testing? Not every function needs such a hook, but some do. See, for example, "Enemies of Test Driven Development": https://jasonmbaker.wordpress.com/2009/01/08/enemies-of-test-driven- development-part-i-encapsulation/ In Python, we have the best of both worlds: we can flag a method as private, and *still* test it! So in a sense, Python's very design has been created specifically to allow testing. For a dissenting view, "Are Private Methods a Code Smell?": http://carlosschults.net/en/are-private-methods-a-code-smell/ >> (4) Write your doctests to test the most general properties of the >> returned results: >> >> >> def my_thing(randint=None): >> """blah blah blah >> >> >>> num = my_thing() >> >>> isinstance(num, int) and 0 <= my_thing() <= 6 >> True >> >> """ > > This is what I'd probably do, tbh. Sometimes that's sufficient. Sometimes its not. It depends on the function. For example, imagine a function that returns a randomly selected prime number. The larger the prime, the less likely it is to be selected, but there's no upper limit. So you write: >>> num = my_thing() >>> isinstance(num, int) and 2 <= num True Not very informative as documentation, and a lousy test too. > None of the options really appeal though. Personally, I'd probably > either go with #4, or maybe something like this: > > def roll(sequence): > """Roll a set of dice > > >>> from test_mymodule import * # ensure stable RNG > >>> roll("d12 + 2d6 + 3") > You roll d12: 8 You roll 2d6: 1, 6, totalling 7. > You add a bonus of 3 For d12 + 2d6 + 3, you total: 18 > """ > > and bury all the monkey-patching into test_mymodule. Wait... are you saying that importing test_mymodule monkey-patches the current library? And doesn't un-patch it afterwards? That's horrible. Or are you saying that test_module has its own version of roll(), and so you're using *that* version instead of the one in the library? That's horrible too. I think that once you are talking about monkey-patching things in order to test them, you should give up on doc tests and use unittest instead. At least then you get nice setUp and tearDown methods that you can use. > It can have its own > implementations of randint and whatever else you use. That way, at least > there's only one line that does the messing around. I still don't like > it though - so quite honestly, I'm most likely to go the route of "don't > actually use doctests". Are you saying don't use doctests for *this* problem, or don't use them *at all*? -- Steven D'Aprano “You are deluded if you think software engineers who can't write operating systems or applications without security holes, can write virtualization layers without security holes.” —Theo de Raadt -- https://mail.python.org/mailman/listinfo/python-list
