I agree here: Disabling the test and having an issue keeps the build green but bears the danger of forgetting about it ...
Regards Felix Am 02.06.2013 um 16:04 schrieb Eric Norman: > Personally, I'm not a big fan of hiding flaky/failing tests since it tends > to remove some of the motivation to stabilize/fix them in a timely manner. > > That's my 2 cents. > > Regards, > Eric > > On Fri, May 31, 2013 at 12:14 PM, Robert Munteanu <romb...@apache.org>wrote: > >> Hi, >> >> It seems that the ErrorHandlingTest fails sporadically when run inside a >> full maven build. I've tried locating the root cause for a couple of >> hours but failed. For this test, and for future flaky/failing tests, I >> suggest that we >> >> 1. Create an issue for the failing test >> 2. Disable the test and mark it with the issue key >> 3. Re-enable the test when it is stable/passing ( which may be >> considerably later than step 2) >> 4. Close the issue after the test is re-enabled >> >> This has the advantage of keeping the build green and making it easier >> to find regressions since a failing or unstable build will actually mean >> something. >> >> What do you think? >> >> Robert >> >>
