Actually, I see we already discussed this in a PR and agreed to use
"Broken":
https://github.com/apache/incubator-brooklyn/pull/1006
I'll thus switch to:
@Test(groups={"Broken"})
We must ensure that the test's javadoc is always updated to include the
error message and that it's non-deterministic. Otherwise we risk someone
coming along later, saying "works for me" and re-enabling it without
sufficient investigation.
Aled
On 24/05/2016 09:54, Aled Sage wrote:
Hi all,
TL;DR: let's disable non-deterministic failing unit tests while they
are being fixed.
---
We have a small number of non-deterministic test failures that are
causing false-negatives when jenkins builds master or builds a PR branch.
Clearly we should fix these tests (either re-writing them to avoid
non-determinism, or fixing whatever underlying bug it is exposing).
However, I think we need a pragmatic short-term solution to get our
jenkins build being reliable+believable asap. We've been living with
the occasional jenkins build failure for far too long!
I've created:
* https://issues.apache.org/jira/browse/BROOKLYN-272
* https://github.com/apache/brooklyn-server/pull/147
The issue tracks the failing tests, and the PR disables them, marking
them in testng as:
@Test(groups={"WIP", "Non-deterministic-failure"})
Thoughts?
Aled
