On Sun, Dec 08, 2013 at 11:22:37AM +0100, Rafael Knuth wrote:
> Hey there,
>
> I struggle to understand what unit testing specifically means in
> practice and how to actually write unit tests for my code (my gut is
> telling me that it's a fairly important concept to understand).
In practice, unit testing is a way to check that your code does what you
expect it should do. For every function and method (or at least, as many
of them as you can), you should test that it does what you expect.
"What you expect" means the following:
- when given good input, does the function return the correct result?
- when given bad input, does the function fail the way it is supposed to
fail? (e.g. raise an exception, return an error code).
Also, the unit test module is good for writing "regression tests". Every
time you discover a bug in your code, before you fix the bug, you should
write a test that demonstrates the existence of that bug. Then, if later
changes to your program bring the bug back (this is called a
regression), the test will fail and you will discover the regression
straight away.
A simple example: suppose you have a method, Dog.bark(n) which is
supposed to return "bark!" repeated some number of times, and if n is
zero or negative it should return the empty string. Normally I put my
unit tests in a separate file. So here's some unit tests for the
Dog.bark method:
import unittest
from myprogram import Dog
class DogTest(unittest.TestCase):
def test_bark_with_positive_argument(self):
dog = Dog()
self.assertEqual(dog.bark(1), "bark!")
self.assertEqual(dog.bark(2), "bark! bark!")
self.assertEqual(dog.bark(5), "bark! bark! bark! bark! bark!")
def test_bark_with_zero_argument(self):
dog = Dog()
self.assertEqual(dog.bark(0), "")
def test_bark_with_negative_argument(self):
dog = Dog()
for i in (-1, -2, -3):
self.assertEqual(dog.bark(i), "")
def test_walk(self):
# test the walk method
# and so on
if __name__ == '__main__':
# Only run this part when running this file as a script.
unittest.main()
(Alas, good tests often end up with tediously long names. Fortunately
you don't have to read the test names very often.)
The tests all pass! Looks good. But here we see the problem with
testing: tests can only show the existence of bugs, they cannot prove
that there are no bugs at all. Oh well. At some point, some poor unlucky
fellow discovers a bug in the Dog class, and reports it to you:
Problem: Dog.bark() returns wrong result
dog = Dog()
dog.bark(-17)
Expected result: ""
Actual result: "meow!"
A fact of life -- you can never (well, hardly ever) test *every*
possible result from your functions. Occasionally there will be
surprises like this. Nevermind, this is the job of the program: fix the
bugs as they are discovered.
So we add an extra test to the unit tests. This one is going to test for
regressions, so we don't just modify the existing negative argument
test, but we make sure this is a specific, individual test.
class DogTest(unittest.TestCase):
[...other methods remain the same]
def test_bark_with_negative_argument(self):
dog = Dog()
samples = [1, 2, 3, 4, 5] + random.sample(range(6, 10000), 20)
for i in samples:
self.assertEqual(dog.bark(-i), "")
def test_regression_bug_id_92(self):
dog = Dog()
self.assertEqual(dog.bark(-17), "")
Notice that I've made the negative_argument case a lot more vigorous at
testing the method. It's a matter of personal judgement how many cases
you should check, but given that we've missed one bug, that's a good
sign that we didn't check enough. So now instead of testing just a small
handful, I test the first five negative values plus another 20 randomly
choosen ones.
I also add a specific regression test to ensure that this bug can never
happen again. In this example I've put the Bug Report ID in the test
name, but that's not compulsary. The important thing is that you have a
test for the bug.
If I run the DogTest now, test_regression_bug_id_92 fails, because I
haven't fixed the bug. This proves that the test works as expected.
Now I fix the bug, re-run the DogTest, and hopefully everything passes.
If so, I can be reasonably sure that there are no obvious bugs in the
parts of the code I've actually tested.
[...]
> Also, I went through the "Beginning Test-Driven Development in Python"
> http://net.tutsplus.com/tutorials/python-tutorials/test-driven-development-in-python/
> but I have to admit I still don't fully understand how unit tests work
> in practice and how to write my own unit tests.
How unit tests work -- the unittest module is a big, complicated package
that defines a whole lot of classes and methods. The idea is that the
module defines a class that understands how to perform testing. It knows
how to run "assertSomething" methods, or if you prefer,
"failIfNotSomething" methods. It knows how to identify the test classes,
how to identify the test methods inside those classes, how to run the
tests, collect the results, display them to you, and report the final
result of whether they all passed or some failed. There is a *lot* of
smarts built into the unittest module.
To use unittest, of course you have to import it. Then you run the unit
test main function:
unittest.main()
What this does is:
- identify the module you are running in;
- search that module for classes that inherit from TestCase (and
possibly a few others, but I always use TestCase);
- start collecting test results;
- for each test class, look for methods that start with "test";
- run those tests, and check whether they pass or fail;
- draw a pretty status diagram, showing a dot . for each passing
tests, F for failing tests, and E for errors;
- if there are any failing tests or errors, print up a report
showing them.
That's a lot of work, but it's all done for you by the unittest package.
You just have to write the tests and call unittest.main().
You can read the source code to unittest:
http://hg.python.org/cpython/file/ad2cd599f1cf/Lib/unittest
but I warn you that it is a big, advanced package, and not especially
the easiest to read. A lot of it is copied from a similar Java testing
framework. Don't try to understand the whole thing at once, take it in
little pieces.
> So, what I am
> specifically searching for is a very simple code sample which I can
> take apart and iterate through each component, and I was wondering if
> you are aware of resources that might be helpful?
ActiveState Python recipes. If you don't mind me linking to my own work:
http://code.activestate.com/recipes/users/4172944/
Raymond Hettinger's recipes are always worth learning from, although
they are often quite advanced:
http://code.activestate.com/recipes/users/178123/
If you can afford it, I recommend you buy the cookbook:
http://shop.oreilly.com/product/9780596001674.do
although I'm not sure if that has been updated to Python 3 yet.
> My understanding of unit testing is that I have to embed my code into
> a test and then I have to define conditions under which my code is
> supposed to fail and pass. Is that assumption correct?
That's pretty much it.
--
Steven
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
