Aled Sage created BROOKLYN-272:
----------------------------------
Summary: Non-deterministic test failures in jenkins
Key: BROOKLYN-272
URL: https://issues.apache.org/jira/browse/BROOKLYN-272
Project: Brooklyn
Issue Type: Bug
Reporter: Aled Sage
There are several non-deterministic tests that fail occasionally in the apache
jenkins build. We should fix those tests: either the underlying bug if there is
one, or the way to the test has been written to make it more deterministic.
Having build failures unrelated to the changes being made in a PR (and on
master) is extremely disruptive for development. We must avoid that wherever
possible.
I therefore suggest we disable these non-deterministic failing tests
immediately, and use this jira issue to track fixing them. We can annotate the
tests with {{@Test(groups={"WIP", "Non-deterministic-failure"})} to make them
easy to find. (note the "Non-deterministic-failure" group is not used by any
profiles, it's just there for searching purposes).
The tests I have identified from recent build failures are:
* {{BasicStartableTest.testTransitionsThroughLifecycles}} - see
https://issues.apache.org/jira/browse/BROOKLYN-256
* {{AbstractGeoDnsServiceTest.testFiltersForRunningEntities}}, failing with:
{noformat}
org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: failed
succeeds-eventually, 69 attempts, 30003ms elapsed: AssertionError:
val={R48yHLqg=<address-ignored>, eBHFc4Qd=<address-ignored>}
at org.apache.brooklyn.test.Asserts.fail(Asserts.java:721)
at org.apache.brooklyn.test.Asserts.assertTrue(Asserts.java:703)
at
org.apache.brooklyn.core.entity.EntityAsserts$2.run(EntityAsserts.java:92)
at
org.apache.brooklyn.test.Asserts$RunnableAdapter.call(Asserts.java:1277)
at
org.apache.brooklyn.test.Asserts.succeedsEventually(Asserts.java:930)
at
org.apache.brooklyn.test.Asserts.succeedsEventually(Asserts.java:854)
at
org.apache.brooklyn.core.entity.EntityAsserts.assertAttributeEventually(EntityAsserts.java:89)
at
org.apache.brooklyn.core.entity.EntityAsserts.assertAttributeEventually(EntityAsserts.java:84)
at
org.apache.brooklyn.entity.dns.AbstractGeoDnsServiceTest.testFiltersForRunningEntities(AbstractGeoDnsServiceTest.java:263)
{noformat}
* {{LoadBalancingPolicyConcurrencyTest.testConcurrentlyAddContainers}}, failing
with:
{noformat}
org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: failed
succeeds-eventually, 29 attempts, 10002ms elapsed: AssertionError:
actual=[18.0, 21.0, 19.0, 0.0, 20.0, 0.0, 20.0, 36.0, 19.0, 0.0, 19.0, 21.0,
21.0, 19.0, 21.0, 36.0, 21.0, 19.0, 18.0, 36.0]; expected=[20.0, 20.0, 20.0,
20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0,
20.0, 20.0, 20.0, 20.0] expected [20.0] but found [0.0]
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.failNotEquals(Assert.java:494)
at org.testng.Assert.assertEquals(Assert.java:207)
at
org.apache.brooklyn.policy.loadbalancing.AbstractLoadBalancingPolicyTest.assertWorkrates(AbstractLoadBalancingPolicyTest.java:122)
at
org.apache.brooklyn.policy.loadbalancing.AbstractLoadBalancingPolicyTest$2.run(AbstractLoadBalancingPolicyTest.java:138)
at
org.apache.brooklyn.test.Asserts$RunnableAdapter.call(Asserts.java:1277)
at
org.apache.brooklyn.test.Asserts.succeedsEventually(Asserts.java:930)
at
org.apache.brooklyn.test.Asserts.succeedsEventually(Asserts.java:854)
at
org.apache.brooklyn.policy.loadbalancing.AbstractLoadBalancingPolicyTest.assertWorkratesEventually(AbstractLoadBalancingPolicyTest.java:136)
at
org.apache.brooklyn.policy.loadbalancing.LoadBalancingPolicyConcurrencyTest.testConcurrentlyAddContainers(LoadBalancingPolicyConcurrencyTest.java:101)
{noformat}
* {{PollerTest.testFeedContinuesWhenPollerThrows}}, failing with:
{noformat}
testFeedContinuesWhenPollerThrows(org.apache.brooklyn.core.feed.PollerTest)
Time elapsed: 3.556 sec <<< FAILURE!
org.apache.brooklyn.util.exceptions.PropagatedRuntimeException: failed
succeeds-eventually, 4 attempts, 105ms elapsed: AssertionError:
entity=FeedExceptionEntityImpl{id=dYG0SKbP}; attribute=Sensor: flag
(java.lang.Boolean) expected [false] but found [true]
at org.apache.brooklyn.test.Asserts.fail(Asserts.java:721)
at org.apache.brooklyn.test.Asserts.failNotEquals(Asserts.java:114)
at org.apache.brooklyn.test.Asserts.assertEquals(Asserts.java:436)
at
org.apache.brooklyn.core.entity.EntityAsserts.assertAttributeEquals(EntityAsserts.java:54)
at
org.apache.brooklyn.core.entity.EntityAsserts$1.run(EntityAsserts.java:70)
at
org.apache.brooklyn.test.Asserts$RunnableAdapter.call(Asserts.java:1277)
at org.apache.brooklyn.test.Asserts.succeedsEventually(Asserts.java:930)
at org.apache.brooklyn.test.Asserts.succeedsEventually(Asserts.java:854)
at
org.apache.brooklyn.core.entity.EntityAsserts.assertAttributeEqualsEventually(EntityAsserts.java:67)
at
org.apache.brooklyn.core.feed.PollerTest.testFeedContinuesWhenPollerThrows(PollerTest.java:65)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
