On Friday, 20 October 2017 at 14:04:25 UTC, Mike Parker wrote:


I'm somewhat late to this party.... but anyway, here is my two cents on the way Unit testing needs to be tweaked.

One of the values of Unit Testing is Defect Localization.

ie. In a well designed unit test suite, tell me which test failed, I will tell you, to within a few lines, where the bug is.

However in the presence of setup and teardown failures, we lose that.

Ideally we should differentiate between failures that occur during setup and teardown, versus exceptions occurring in the behaviour under test, or assertions on the validity of the result.

ie. Failures under setup and teardown are not failures of the behaviour under test. The only thing we can say about the behaviour under test in these cases is, that it “hasn’t been tested”.

Hopefully the behaviour that failed during the setup and teardown is explicitly tested elsewhere.

ie. We should stop at the first test that fails at a point other than setup and teardown, as this is likely to be the cause, for the cascade of failures in setup and teardown of other tests.

Reply via email to