On 21.02.17 15:32, Alex Parvulescu wrote:
Hi, If in this context b) actually means 'fix the tests to be more lenient' I agree this should be the way to go.
Yes and this is not necessarily bad as I presume that some of the time-outs might well be overly tight (at least in some environments).
However I think the failing tests are not being given enough priority currently and if people aren't able to carve out the time for investigation, then it means we're stuck with very unreliable builds and this can hide more problems down the line. I would disable the tests that have failed more that a few times, on Jenkins only with the goal of returning to a consistent green build again.
+1. Please have a look at the CIHelper class and its usage for disabling tests on Jenkins only.
Michael
alex On Tue, Feb 21, 2017 at 2:09 PM, Thomas Mueller <[email protected]> wrote:Hi, I assume with (b) you mean: change tests to use loops, combined with very high timeouts. Example: Before: save(); Thread.sleep(1000); assertTrue(abc()); After: save(); for(int i=0; !abc() && i<600; i++) { Thread.sleep(100); } assertTrue(abc()); The additional benefit of this logic is that on a fast machine, the test is faster (only 100 ms sleep instead of 1 second). Disadvantage: additional complexity, as you wrote (could be avoided with Java 8 lambda expressions). Regards, Thomas On 21/02/17 13:49, "Michael Dürig" <[email protected]> wrote:Hi, I assume that at least some of the tests that sporadically fail on the Apache Jenkins fail because of timing issues. To address this we could either a) skip these tests on Jenkins, b) increase the time-out, c) apply platform dependent time-outs. I would prefer b). I presume that there is no impact on the build time unless the build fails anyway because it is running into one of these time-outs. If this is not acceptable we could go for b) and provision platform dependent time-outs through the CIHelpers class. I somewhat dislike the additional complexity though. As last resort we can still do a). WDYT? Michal
