Am Montag, den 13.11.2006, 12:58 +0100 schrieb Stefano Bagnara: > > But intended-to-fail failing tests obscures all > > oops-something-bad-happened failing tests. > > Right. This is the BIG problem of failing tests.
Okay, but I don't think that not writing, not executing or inverting them is better. > >> I propose to accept failing tests in the codebase. Nightly build should > >> not fail just because tests are failing. Instead we could provide a > >> testreport in the same folder. > >> Of course tests ran from ant should not be interrupted by the first > >> failure. (haltonfailure="no") > > > > +1 > > Imho the nighly build should not be published if it does not pass tests. > Non passing tests should be only there by mistake, so the mistake could > have compromised the quality of the code unintentionally. Everyone knows that nightly build is potentially unstable. I don't think that the bug added by mistake, that brings a test to fail, is more dangerous than the bug added by mistake that noone notices. I find it more useful to encourage people submitting bug-exposing failing tests and make everyone aware of these known bugs. It's a matter of transparency. > >> Well, of course this brings some administrative overhead. Maybe we need > >> a solution to sort our tests using TestSuites. > >> We could separate the tests known to fail to be warned at once when a > >> change brings additional tests to fail. > > > > -1 > > I don't understand why we should complicate things so much. > > If after a change an assertTrue fail and is known to fail because the > test is wrong then it should be changed to assertFalse and a JIRA issue > added if the test need more attention or have to be completed. Of course a test should be fixed if it is wrong. I'm not talking about executing invalid tests. That's another story. I mean tests that expose known bugs that won't be fixed in a short time frame. Joachim --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
