On Friday, 20 October 2017 at 14:04:25 UTC, Mike Parker wrote:
I'm somewhat late to this party.... but anyway, here is my two
cents on the way Unit testing needs to be tweaked.
One of the values of Unit Testing is Defect Localization.
ie. In a well designed unit test suite, tell me which test
failed, I will tell you, to within a few lines, where the bug is.
However in the presence of setup and teardown failures, we lose
Ideally we should differentiate between failures that occur
during setup and teardown, versus exceptions occurring in the
behaviour under test, or assertions on the validity of the result.
ie. Failures under setup and teardown are not failures of the
behaviour under test. The only thing we can say about the
behaviour under test in these cases is, that it “hasn’t been
Hopefully the behaviour that failed during the setup and teardown
is explicitly tested elsewhere.
ie. We should stop at the first test that fails at a point other
than setup and teardown, as this is likely to be the cause, for
the cascade of failures in setup and teardown of other tests.