> "my philosophy is to test as little as possible to reach a given level of > confidence" > > So the question is, what is the level of confidence you are aiming at? Code > coverage in general, and mutation testing in particular, gives you a tool by > which you can gauge the level of confidence you can have in a test suite, > but doesn't force you to aim at any particular level.
I'd guess that mutation testing will find problems in test suites that otherwise give you reasonable confidence because the test suites are tailored for regressions that the project's developers are likely to make, so you'd have to have 100% coverage and very strict tests to pass mutation testing. Perhaps I'm being overly pessimistic and you do optimise for common mistakes, e.g., passing null/returning null, depending on mutable values in equals, etc. > I can't definitively state what Kent meant by his answer back in 2008, or > what's going through his head now. What I can say is that he's interested > enough in coverage and mutation testing to have given PIT a try. Excellent. > Fundamentally a unit test only ever tests that your code "does what it > does". I'd have thought that tests should test that your code does what it should do. I mean, it's obvious that it does what it does, why test that? Sure, regressions, but if you end up with 100 test failures because you deliberately change the behaviour of something will you spot that one test that fails for a 'valid' reason instead of just failing because your code changed, or will you just fix all tests by updating them for the new behaviour? > Static analysis could only give you this if you provided it with some sort > of specification of what the program should do . . . kind of like a unit > test. You can also call it a type signature, although that's a bit of a stretch outside of dependently-typed languages. -- You received this message because you are subscribed to the Google Groups "Java Posse" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.
