Hi,
> Flakey Test Problems:
> a) Flakey tests create so much noise that people no longer pay
> attention to the automated reporting via email.
> b) When running unit tests manually before a commit (i.e. "ant test")
> a flakey test can fail.
>
> Solutions:
> We cloud fix (a) by marking as flakey and having a new target
> "non-flakey" that is run by the jenkins jobs that are currently run
> continuously.
We have a solution for this already: Mark all those tests with @AwaitsFix or
@BadApple
By default those aren't executed in Jenkins runs and also not for developers,
but devs can enable/disable them using -Dtests.awaitsfix=true and
-Dtests.badapples=true:
[help] # Test groups. ----------------------------------------------------
[help] #
[help] # test groups can be enabled or disabled (true/false). Default
[help] # value provided below in [brackets].
[help]
[help] ant -Dtests.nightly=[false] - nightly test group (@Nightly)
[help] ant -Dtests.weekly=[false] - weekly tests (@Weekly)
[help] ant -Dtests.awaitsfix=[false] - known issue (@AwaitsFix)
[help] ant -Dtests.slow=[true] - slow tests (@Slow)
We can of course also make a weekly jenkins jobs that enables those tests on
Jenkins only weekly (like nightly stuff). We have "tests.badapples" and
"tests.awaitsfix" - I don't know what's the difference between both.
So we have 2 options to classify tests, let's choose one and apply it to all
Flakey tests!
Uwe
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
