On Wednesday, April 03, 2013 11:36:54 Walter Bright wrote: > On 4/3/2013 10:58 AM, Jonathan M Davis wrote: > > If you push for the lines of unit testing code to be kept to a minimum, I > > don't see how you can possibly expect stuff to be thoroughly tested. > > My idea of perfection would be 100% coverage with zero redundancy in the > unittests. > > In my experience with testing, the technique of "quantity has a quality all > its own" style of testing does not produce adequate test coverage - it just > simply takes a lot of time to run (which makes it less useful, as one then > tends to avoid running them).
Well, determining what's actually redundant isn't always easy. If a test is clearly redundant, then it makes sense to remove it, but if you're not careful with that (especially if you're basing your tests off of what the current code looks like), then it can be easy to remove tests which were basically unnecessary with the current implementation but which would have caught bugs when the code was refactored. So, while in principle, I agree that having zero redundancy would be good, in practice, I don't think that it's that straightforward. I also don't think that code coverage means much beyond the fact that if you don't have 100% (minus the lines of code that can never be hit - e.g. assert(0);), then obviously some stuff isn't tested properly. You need to hit all of the corner cases and whatnot which may not work correctly yet or which may get broken when refactoring, and often, 100% test coverage doesn't get you there, much as it's an important milestone. Certainly, I agree that having the minimal tests required to test everything that needs testing should be the goal, but figuring out which tests are and aren't really needed is a bit of art. Personally, I do tend to err on the side of over-testing rather than under-testing though, as that does a better job of ensuring that the code is correct. Actually, I'd argue that in perfect world, you'd test absolutely every possible input to make sure that it had the correct output, but that's obviously impossible in all but the most simplistic code, and actually attempting that approach just leads to unit tests which take too long to run. - Jonathan M Davis
