Hi Thomas,
Not sure I follow. We aren't saying commenting out the tests. We are
proposing to use a test group to mark them as not included in the tests
run by `mvn clean install`. It achieves that based on some logic in the
pom that excludes various groups from different profiles. If you run it
in your IDE, it would still be run (because we're not using
`enabled=false`).
Is your suggestion to use enabled=false motivated by a desire to disable
it in the IDE as well? Or something else?
Aled
On 24/05/2016 10:04, Thomas Bouron wrote:
Hi Aled.
+1 for the marking and disabling those tests. However, I would use the
`enabled=false` parameter[1] of the `@Test` annotation instead of commented
them out.
[1] http://www.mkyong.com/unittest/testng-tutorial-3-ignore-test/
On Tue, 24 May 2016 at 09:54 Aled Sage <[email protected]> wrote:
Hi all,
TL;DR: let's disable non-deterministic failing unit tests while they are
being fixed.
---
We have a small number of non-deterministic test failures that are
causing false-negatives when jenkins builds master or builds a PR branch.
Clearly we should fix these tests (either re-writing them to avoid
non-determinism, or fixing whatever underlying bug it is exposing).
However, I think we need a pragmatic short-term solution to get our
jenkins build being reliable+believable asap. We've been living with the
occasional jenkins build failure for far too long!
I've created:
* https://issues.apache.org/jira/browse/BROOKLYN-272
* https://github.com/apache/brooklyn-server/pull/147
The issue tracks the failing tests, and the PR disables them, marking
them in testng as:
@Test(groups={"WIP", "Non-deterministic-failure"})
Thoughts?
Aled
--
Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
http://www.cloudsoftcorp.com/
Github: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron
