Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/35/
7 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
FAILED:
org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
Error Message:
Timeout while trying to assert number of documents @ source_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @
source_collection
at
__randomizedtesting.SeedInfo.seed([414FC989A1A4EA57:92469997E43776C0]:0)
at
org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:282)
at
org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates(CdcrReplicationHandlerTest.java:233)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: expected:<200> but was:<210>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:273)
... 42 more
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth
investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties
were stopped - this is worth investigating!
at
__randomizedtesting.SeedInfo.seed([414FC989A1A4EA57:C91BF6530F5887AF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
6 threads leaked from SUITE scope at
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1)
Thread[id=110522, name=StoppableCommitThread-EventThread, state=WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
2) Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 3) Thread[id=110523,
name=zkCallback-29781-thread-1, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 4) Thread[id=110472,
name=StoppableCommitThread, state=WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
java.lang.Object.wait(Native Method) at
java.lang.Object.wait(Object.java:502) at
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406) at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210) at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at
org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
at
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
at
org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1227)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:524)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:544)
at
org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:49)
5) Thread[id=110521, name=StoppableCommitThread-SendThread(127.0.0.1:33460),
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method) at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
6) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE
scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=110522, name=StoppableCommitThread-EventThread, state=WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
2) Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=110523, name=zkCallback-29781-thread-1, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=110472, name=StoppableCommitThread, state=WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210)
at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at
org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
at
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
at
org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1227)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:524)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:544)
at
org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:49)
5) Thread[id=110521, name=StoppableCommitThread-SendThread(127.0.0.1:33460),
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
6) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([414FC989A1A4EA57]:0)
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1)
Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2) Thread[id=110523,
name=zkCallback-29781-thread-1, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 3) Thread[id=110521,
name=StoppableCommitThread-SendThread(127.0.0.1:33460), state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
java.lang.Thread.sleep(Native Method) at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
4) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
1) Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=110523, name=zkCallback-29781-thread-1, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=110521, name=StoppableCommitThread-SendThread(127.0.0.1:33460),
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
4) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING,
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([414FC989A1A4EA57]:0)
FAILED:
org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<20000> but was:<10100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync
expected:<20000> but was:<10100>
at
__randomizedtesting.SeedInfo.seed([414FC989A1A4EA57:950A82D046F259AC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at
org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10514 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> avg 29, 2017 12:03:46 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out:
org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=11,
name=SUITE-TestGeo3DPoint-seed#[98D7CDCF196793D5], state=RUNNABLE,
group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native
Method)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 3) Thread[id=12,
name=TEST-TestGeo3DPoint.testRandomBig-seed#[98D7CDCF196793D5], state=RUNNABLE,
group=TGRP-TestGeo3DPoint]
[junit4] 2> at
org.apache.lucene.spatial3d.geom.Plane.findIntersectionBounds(Plane.java:1053)
[junit4] 2> at
org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1214)
[junit4] 2> at
org.apache.lucene.spatial3d.geom.XYZBounds.addIntersection(XYZBounds.java:262)
[junit4] 2> at
org.apache.lucene.spatial3d.geom.GeoStandardPath$PathSegment.getBounds(GeoStandardPath.java:872)
[junit4] 2> at
org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:336)
[junit4] 2> at
org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at
org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at
org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon,
state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at
com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint
-Dtests.method=testRandomBig -Dtests.seed=98D7CDCF196793D5 -Dtests.multiplier=2
-Dtests.nightly=true -Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
-Dtests.locale=sr-Latn-BA -Dtests.timezone=US/Eastern -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 6654s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because
suite timeout was reached.
[junit4] > at
__randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
[junit4] 2> avg 29, 2017 12:03:46 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked
thread(s).
[junit4] 2> avg 29, 2017 12:04:06 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at
org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12,
name=TEST-TestGeo3DPoint.testRandomBig-seed#[98D7CDCF196793D5], state=RUNNABLE,
group=TGRP-TestGeo3DPoint]
[junit4] 2> at
org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at
org.apache.lucene.spatial3d.geom.GeoStandardPath$SegmentEndpoint.getBounds(GeoStandardPath.java:609)
[junit4] 2> at
org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:339)
[junit4] 2> at
org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at
org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at
org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> avg 29, 2017 12:04:06 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12,
name=TEST-TestGeo3DPoint.testRandomBig-seed#[98D7CDCF196793D5], state=RUNNABLE,
group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_98D7CDCF196793D5-001
[junit4] 2> Aug 29, 2017 4:04:06 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
{id=PostingsFormat(name=LuceneFixedGap)},
docValues:{id=DocValuesFormat(name=Lucene70),
point=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1018,
maxMBSortInHeap=6.416690269127653, sim=RandomSimilarity(queryNorm=true): {},
locale=sr-Latn-BA, timezone=US/Eastern
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation
1.8.0_144 (64-bit)/cpus=4,threads=1,free=39527016,total=389021696
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint
-Dtests.seed=98D7CDCF196793D5 -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
-Dtests.locale=sr-Latn-BA -Dtests.timezone=US/Eastern -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>=
7200000 msec).
[junit4] > at
__randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
[junit4] Completed [14/15 (1!)] on J1 in 7221.96s, 11 tests, 2 errors <<<
FAILURES!
[...truncated 1947 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> Creating dataDir:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CdcrBootstrapTest_414FC989A1A4EA57-001/init-core-data-001
[junit4] 2> 188486 WARN
(SUITE-CdcrBootstrapTest-seed#[414FC989A1A4EA57]-worker) [ ]
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 188586 ERROR (Thread-74) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 188917 ERROR (jetty-launcher-126-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 1> Target zkHost = 127.0.0.1:42371/solr
[junit4] 2> 191207 ERROR (Thread-84) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 191454 ERROR (jetty-launcher-138-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 1> Adding 100 docs with commit=true, numDocs=100
[junit4] 1> Adding 100 docs with commit=true, numDocs=200
[junit4] 1> Adding 100 docs with commit=true, numDocs=300
[junit4] 1> Adding 100 docs with commit=true, numDocs=400
[junit4] 1> Adding 100 docs with commit=true, numDocs=500
[junit4] 1> Adding 100 docs with commit=true, numDocs=600
[junit4] 1> Adding 100 docs with commit=true, numDocs=700
[junit4] 1> Adding 100 docs with commit=true, numDocs=800
[junit4] 1> Adding 100 docs with commit=true, numDocs=900
[junit4] 1> Adding 100 docs with commit=true, numDocs=1000
[junit4] 1> Adding 100 docs with commit=true, numDocs=1100
[junit4] 1> Adding 100 docs with commit=true, numDocs=1200
[junit4] 1> Adding 100 docs with commit=true, numDocs=1300
[junit4] 1> Adding 100 docs with commit=true, numDocs=1400
[junit4] 1> Adding 100 docs with commit=true, numDocs=1500
[junit4] 1> Adding 100 docs with commit=true, numDocs=1600
[junit4] 1> Adding 100 docs with commit=true, numDocs=1700
[junit4] 1> Adding 100 docs with commit=true, numDocs=1800
[junit4] 1> Adding 100 docs with commit=true, numDocs=1900
[junit4] 1> Adding 100 docs with commit=true, numDocs=2000
[junit4] 1> Adding 100 docs with commit=true, numDocs=2100
[junit4] 1> Adding 100 docs with commit=true, numDocs=2200
[junit4] 1> Adding 100 docs with commit=true, numDocs=2300
[junit4] 1> Adding 100 docs with commit=true, numDocs=2400
[junit4] 1> Adding 100 docs with commit=true, numDocs=2500
[junit4] 1> Adding 100 docs with commit=true, numDocs=2600
[junit4] 1> Adding 100 docs with commit=true, numDocs=2700
[junit4] 1> Adding 100 docs with commit=true, numDocs=2800
[junit4] 1> Adding 100 docs with commit=true, numDocs=2900
[junit4] 1> Adding 100 docs with commit=true, numDocs=3000
[junit4] 1> Adding 100 docs with commit=true, numDocs=3100
[junit4] 1> Adding 100 docs with commit=true, numDocs=3200
[junit4] 1> Adding 100 docs with commit=true, numDocs=3300
[junit4] 1> Adding 100 docs with commit=true, numDocs=3400
[junit4] 1> Adding 100 docs with commit=true, numDocs=3500
[junit4] 1> Adding 100 docs with commit=true, numDocs=3600
[junit4] 1> Adding 100 docs with commit=true, numDocs=3700
[junit4] 1> Adding 100 docs with commit=true, numDocs=3800
[junit4] 1> Adding 100 docs with commit=true, numDocs=3900
[junit4] 1> Adding 100 docs with commit=true, numDocs=4000
[junit4] 1> Adding 100 docs with commit=true, numDocs=4100
[junit4] 1> Adding 100 docs with commit=true, numDocs=4200
[junit4] 1> Adding 100 docs with commit=true, numDocs=4300
[junit4] 1> Adding 100 docs with commit=true, numDocs=4400
[junit4] 1> Adding 100 docs with commit=true, numDocs=4500
[junit4] 1> Adding 100 docs with commit=true, numDocs=4600
[junit4] 1> Adding 100 docs with commit=true, numDocs=4700
[junit4] 1> Adding 100 docs with commit=true, numDocs=4800
[junit4] 1> Adding 100 docs with commit=true, numDocs=4900
[junit4] 1> Adding 100 docs with commit=true, numDocs=5000
[junit4] 1> Adding 100 docs with commit=true, numDocs=5100
[junit4] 1> Adding 100 docs with commit=true, numDocs=5200
[junit4] 1> Adding 100 docs with commit=true, numDocs=5300
[junit4] 1> Adding 100 docs with commit=true, numDocs=5400
[junit4] 1> Adding 100 docs with commit=true, numDocs=5500
[junit4] 1> Adding 100 docs with commit=true, numDocs=5600
[junit4] 1> Adding 100 docs with commit=true, numDocs=5700
[junit4] 1> Adding 100 docs with commit=true, numDocs=5800
[junit4] 1> Adding 100 docs with commit=true, numDocs=5900
[junit4] 1> Adding 100 docs with commit=true, numDocs=6000
[junit4] 1> Adding 100 docs with commit=true, numDocs=6100
[junit4] 1> Adding 100 docs with commit=true, numDocs=6200
[junit4] 1> Adding 100 docs with commit=true, numDocs=6300
[junit4] 1> Adding 100 docs with commit=true, numDocs=6400
[junit4] 1> Adding 100 docs with commit=true, numDocs=6500
[junit4] 1> Adding 100 docs with commit=true, numDocs=6600
[junit4] 1> Adding 100 docs with commit=true, numDocs=6700
[junit4] 1> Adding 100 docs with commit=true, numDocs=6800
[junit4] 1> Adding 100 docs with commit=true, numDocs=6900
[junit4] 1> Adding 100 docs with commit=true, numDocs=7000
[junit4] 1> Adding 100 docs with commit=true, numDocs=7100
[junit4] 1> Adding 100 docs with commit=true, numDocs=7200
[junit4] 1> Adding 100 docs with commit=true, numDocs=7300
[junit4] 1> Adding 100 docs with commit=true, numDocs=7400
[junit4] 1> Adding 100 docs with commit=true, numDocs=7500
[junit4] 1> Adding 100 docs with commit=true, numDocs=7600
[junit4] 1> Adding 100 docs with commit=true, numDocs=7700
[junit4] 1> Adding 100 docs with commit=true, numDocs=7800
[junit4] 1> Adding 100 docs with commit=true, numDocs=7900
[junit4] 1> Adding 100 docs with commit=true, numDocs=8000
[junit4] 1> Adding 100 docs with commit=true, numDocs=8100
[junit4] 1> Adding 100 docs with commit=true, numDocs=8200
[junit4] 1> Adding 100 docs with commit=true, numDocs=8300
[junit4] 1> Adding 100 docs with commit=true, numDocs=8400
[junit4] 1> Adding 100 docs with commit=true, numDocs=8500
[junit4] 1> Adding 100 docs with commit=true, numDocs=8600
[junit4] 1> Adding 100 docs with commit=true, numDocs=8700
[junit4] 1> Adding 100 docs with commit=true, numDocs=8800
[junit4] 1> Adding 100 docs with commit=true, numDocs=8900
[junit4] 1> Adding 100 docs with commit=true, numDocs=9000
[junit4] 1> Adding 100 docs with commit=true, numDocs=9100
[junit4] 1> Adding 100 docs with commit=true, numDocs=9200
[junit4] 1> Adding 100 docs with commit=true, numDocs=9300
[junit4] 1> Adding 100 docs with commit=true, numDocs=9400
[junit4] 1> Adding 100 docs with commit=true, numDocs=9500
[junit4] 1> Adding 100 docs with commit=true, numDocs=9600
[junit4] 1> Adding 100 docs with commit=true, numDocs=9700
[junit4] 1> Adding 100 docs with commit=true, numDocs=9800
[junit4] 1> Adding 100 docs with commit=true, numDocs=9900
[junit4] 1> Adding 100 docs with commit=true, numDocs=10000
[junit4] 1> Cdcr queue response:
{responseHeader={status=0,QTime=31},queues={127.0.0.1:42371/solr={cdcr-target={queueSize=10100,lastTimestamp=}}},tlogTotalSize=494631,tlogTotalCount=100,updateLogSynchronizer=stopped}
[junit4] 2> 271427 ERROR
(updateExecutor-130-thread-1-processing-n:127.0.0.1:37463_solr
x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
[n:127.0.0.1:37463_solr c:cdcr-target s:shard1 r:core_node2
x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation
failed
[junit4] 2> java.util.concurrent.ExecutionException:
java.lang.AssertionError
[junit4] 2> at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at
java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:646)
[junit4] 2> at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.AssertionError
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:789)
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:716)
[junit4] 2> at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 2> 272557 WARN
(cdcr-bootstrap-status-157-thread-1-processing-n:127.0.0.1:40002_solr
x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2)
[n:127.0.0.1:40002_solr c:cdcr-source s:shard1 r:core_node2
x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Cancelling
waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 1> Adding 100 docs with commit=true, numDocs=10100
[junit4] 1> Adding 100 docs with commit=true, numDocs=10200
[junit4] 1> Adding 100 docs with commit=true, numDocs=10300
[junit4] 1> Adding 100 docs with commit=true, numDocs=10400
[junit4] 1> Adding 100 docs with commit=true, numDocs=10500
[junit4] 1> Adding 100 docs with commit=true, numDocs=10600
[junit4] 1> Adding 100 docs with commit=true, numDocs=10700
[junit4] 1> Adding 100 docs with commit=true, numDocs=10800
[junit4] 1> Adding 100 docs with commit=true, numDocs=10900
[junit4] 1> Adding 100 docs with commit=true, numDocs=11000
[junit4] 2> 287133 WARN
(cdcr-bootstrap-status-160-thread-1-processing-n:127.0.0.1:40002_solr
x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2)
[n:127.0.0.1:40002_solr c:cdcr-source s:shard1 r:core_node2
x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Cancelling
waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 2> 287377 WARN
(zkCallback-145-thread-6-processing-n:127.0.0.1:40002_solr)
[n:127.0.0.1:40002_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered,
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 287394 ERROR
(TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[414FC989A1A4EA57])
[ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so
ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 287671 WARN (Thread-84) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/collections/cdcr-source/state.json
[junit4] 2> 4 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2> 3 /solr/collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> 288078 WARN
(zkCallback-133-thread-3-processing-n:127.0.0.1:37463_solr)
[n:127.0.0.1:37463_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered,
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 288144 ERROR
(TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[414FC989A1A4EA57])
[ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so
ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 288277 WARN (Thread-74) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3
/solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 4 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 288392 ERROR (Thread-110) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 288510 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x15e2c487f9d0000, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 288575 ERROR (jetty-launcher-161-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 1> Target zkHost = 127.0.0.1:41453/solr
[junit4] 2> 290546 ERROR (Thread-120) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 290725 ERROR (jetty-launcher-173-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 1> Adding 100 docs with commit=true, numDocs=100
[junit4] 1> Adding 100 docs with commit=true, numDocs=200
[junit4] 1> Adding 100 docs with commit=true, numDocs=300
[junit4] 1> Adding 100 docs with commit=true, numDocs=400
[junit4] 1> Adding 100 docs with commit=true, numDocs=500
[junit4] 1> Adding 100 docs with commit=true, numDocs=600
[junit4] 1> Adding 100 docs with commit=true, numDocs=700
[junit4] 1> Adding 100 docs with commit=true, numDocs=800
[junit4] 1> Adding 100 docs with commit=true, numDocs=900
[junit4] 1> Adding 100 docs with commit=true, numDocs=1000
[junit4] 1> Adding 100 docs with commit=true, numDocs=1100
[junit4] 1> Adding 100 docs with commit=true, numDocs=1200
[junit4] 1> Adding 100 docs with commit=true, numDocs=1300
[junit4] 1> Adding 100 docs with commit=true, numDocs=1400
[junit4] 1> Adding 100 docs with commit=true, numDocs=1500
[junit4] 1> Adding 100 docs with commit=true, numDocs=1600
[junit4] 1> Adding 100 docs with commit=true, numDocs=1700
[junit4] 1> Adding 100 docs with commit=true, numDocs=1800
[junit4] 1> Adding 100 docs with commit=true, numDocs=1900
[junit4] 1> Adding 100 docs with commit=true, numDocs=2000
[junit4] 1> Adding 100 docs with commit=true, numDocs=2100
[junit4] 1> Adding 100 docs with commit=true, numDocs=2200
[junit4] 1> Adding 100 docs with commit=true, numDocs=2300
[junit4] 1> Adding 100 docs with commit=true, numDocs=2400
[junit4] 1> Adding 100 docs with commit=true, numDocs=2500
[junit4] 1> Adding 100 docs with commit=true, numDocs=2600
[junit4] 1> Adding 100 docs with commit=true, numDocs=2700
[junit4] 1> Adding 100 docs with commit=true, numDocs=2800
[junit4] 1> Adding 100 docs with commit=true, numDocs=2900
[junit4] 1> Adding 100 docs with commit=true, numDocs=3000
[junit4] 1> Adding 100 docs with commit=true, numDocs=3100
[junit4] 1> Adding 100 docs with commit=true, numDocs=3200
[junit4] 1> Adding 100 docs with commit=true, numDocs=3300
[junit4] 1> Adding 100 docs with commit=true, numDocs=3400
[junit4] 1> Adding 100 docs with commit=true, numDocs=3500
[junit4] 1> Adding 100 docs with commit=true, numDocs=3600
[junit4] 1> Adding 100 docs with commit=true, numDocs=3700
[junit4] 1> Adding 100 docs with commit=true, numDocs=3800
[junit4] 1> Adding 100 docs with commit=true, numDocs=3900
[junit4] 1> Adding 100 docs with commit=true, numDocs=4000
[junit4] 1> Adding 100 docs with commit=true, numDocs=4100
[junit4] 1> Adding 100 docs with commit=true, numDocs=4200
[junit4] 1> Adding 100 docs with commit=true, numDocs=4300
[junit4] 1> Adding 100 docs with commit=true, numDocs=4400
[junit4] 1> Adding 100 docs with commit=true, numDocs=4500
[junit4] 1> Adding 100 docs with commit=true, numDocs=4600
[junit4] 1> Adding 100 docs with commit=true, numDocs=4700
[junit4] 1> Adding 100 docs with commit=true, numDocs=4800
[junit4] 1> Adding 100 docs with commit=true, numDocs=4900
[junit4] 1> Adding 100 docs with commit=true, numDocs=5000
[junit4] 1> Adding 100 docs with commit=true, numDocs=5100
[junit4] 1> Adding 100 docs with commit=true, numDocs=5200
[junit4] 1> Adding 100 docs with commit=true, numDocs=5300
[junit4] 1> Adding 100 docs with commit=true, numDocs=5400
[junit4] 1> Adding 100 docs with commit=true, numDocs=5500
[junit4] 1> Adding 100 docs with commit=true, numDocs=5600
[junit4] 1> Adding 100 docs with commit=true, numDocs=5700
[junit4] 1> Adding 100 docs with commit=true, numDocs=5800
[junit4] 1> Adding 100 docs with commit=true, numDocs=5900
[junit4] 1> Adding 100 docs with commit=true, numDocs=6000
[junit4] 1> Adding 100 docs with commit=true, numDocs=6100
[junit4] 1> Adding 100 docs with commit=true, numDocs=6200
[junit4] 1> Adding 100 docs with commit=true, numDocs=6300
[junit4] 1> Adding 100 docs with commit=true, numDocs=6400
[junit4] 1> Adding 100 docs with commit=true, numDocs=6500
[junit4] 1> Adding 100 docs with commit=true, numDocs=6600
[junit4] 1> Adding 100 docs with commit=true, numDocs=6700
[junit4] 1> Adding 100 docs with commit=true, numDocs=6800
[junit4] 1> Adding 100 docs with commit=true, numDocs=6900
[junit4] 1> Adding 100 docs with commit=true, numDocs=7000
[junit4] 1> Adding 100 docs with commit=true, numDocs=7100
[junit4] 1> Adding 100 docs with commit=true, numDocs=7200
[junit4] 1> Adding 100 docs with commit=true, numDocs=7300
[junit4] 1> Adding 100 docs with commit=true, numDocs=7400
[junit4] 1> Adding 100 docs with commit=true, numDocs=7500
[junit4] 1> Adding 100 docs with commit=true, numDocs=7600
[junit4] 1> Adding 100 docs with commit=true, numDocs=7700
[junit4] 1> Adding 100 docs with commit=true, numDocs=7800
[junit4] 1> Adding 100 docs with commit=true, numDocs=7900
[junit4] 1> Adding 100 docs with commit=true, numDocs=8000
[junit4] 1> Adding 100 docs with commit=true, numDocs=8100
[junit4] 1> Adding 100 docs with commit=true, numDocs=8200
[junit4] 1> Adding 100 docs with commit=true, numDocs=8300
[junit4] 1> Adding 100 docs with commit=true, numDocs=8400
[junit4] 1> Adding 100 docs with commit=true, numDocs=8500
[junit4] 1> Adding 100 docs with commit=true, numDocs=8600
[junit4] 1> Adding 100 docs with commit=true, numDocs=8700
[junit4] 1> Adding 100 docs with commit=true, numDocs=8800
[junit4] 1> Adding 100 docs with commit=true, numDocs=8900
[junit4] 1> Adding 100 docs with commit=true, numDocs=9000
[junit4] 1> Adding 100 docs with commit=true, numDocs=9100
[junit4] 1> Adding 100 docs with commit=true, numDocs=9200
[junit4] 1> Adding 100 docs with commit=true, numDocs=9300
[junit4] 1> Adding 100 docs with commit=true, numDocs=9400
[junit4] 1> Adding 100 docs with commit=true, numDocs=9500
[junit4] 1> Adding 100 docs with commit=true, numDocs=9600
[junit4] 1> Adding 100 docs with commit=true, numDocs=9700
[junit4] 1> Adding 100 docs with commit=true, numDocs=9800
[junit4] 1> Adding 100 docs with commit=true, numDocs=9900
[junit4] 1> Adding 100 docs with commit=true, numDocs=10000
[junit4] 2> 336021 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x15e2c4888370008, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 337288 WARN
(zkCallback-180-thread-1-processing-n:127.0.0.1:45076_solr)
[n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered,
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 337338 ERROR
(TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57])
[ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging
may be missing or incomplete.
[junit4] 1> -
[junit4] 1> replica:core_node2 rstate:down live:true
[junit4] 1> -
[junit4] 1> replica:core_node2 rstate:active live:true
[junit4] 1> no one is recoverying
[junit4] 2> 340547 ERROR
(TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57])
[ ] o.a.s.c.s.i.CloudSolrClient Request to collection cdcr-source failed
due to (510) org.apache.solr.common.SolrException: Could not find a healthy
node to handle the request., retry? 0
[junit4] 2> 340547 WARN
(TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57])
[ ] o.a.s.c.s.i.CloudSolrClient Re-trying request to collection(s)
cdcr-source after stale state error from server.
[junit4] 1> Cdcr queue response:
{responseHeader={status=0,QTime=183},queues={127.0.0.1:41453/solr={cdcr-target={queueSize=-1927282669,lastTimestamp=}}},tlogTotalSize=4973,tlogTotalCount=1,updateLogSynchronizer=stopped}
[junit4] 2> 347300 WARN
(cdcr-bootstrap-status-199-thread-1-processing-n:127.0.0.1:36989_solr
x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2)
[n:127.0.0.1:36989_solr c:cdcr-source s:shard1 r:core_node2
x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Cancelling
waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 2> 347642 ERROR
(updateExecutor-165-thread-1-processing-n:127.0.0.1:34947_solr
x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
[n:127.0.0.1:34947_solr c:cdcr-target s:shard1 r:core_node2
x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation
failed
[junit4] 2> java.util.concurrent.ExecutionException:
java.lang.AssertionError
[junit4] 2> at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at
java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:646)
[junit4] 2> at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.AssertionError
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:789)
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:716)
[junit4] 2> at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 2> 348925 WARN
(zkCallback-193-thread-3-processing-n:127.0.0.1:36989_solr)
[n:127.0.0.1:36989_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered,
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 349140 ERROR
(TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57])
[ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so
ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 349329 WARN (Thread-120) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/configs/cdcr-source
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/collections/cdcr-source/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 351051 WARN
(zkCallback-168-thread-2-processing-n:127.0.0.1:34947_solr)
[n:127.0.0.1:34947_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered,
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 351075 ERROR
(TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57])
[ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so
ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 351171 WARN (Thread-110) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 2
/solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 351219 ERROR (Thread-153) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 351457 ERROR (jetty-launcher-202-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 1> Target zkHost = 127.0.0.1:44782/solr
[junit4] 2> 353008 ERROR (Thread-163) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 353123 ERROR (jetty-launcher-214-thread-1) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 1> Adding 100 docs with commit=true, numDocs=100
[junit4] 1> Adding 100 docs with commit=true, numDocs=200
[junit4] 1> Adding 100 docs with commit=true, numDocs=300
[junit4] 1> Adding 100 docs with commit=true, numDocs=400
[junit4] 1> Adding 100 docs with commit=true, numDocs=500
[junit4] 1> Adding 100 docs with commit=true, numDocs=600
[junit4] 1> Adding 100 docs with commit=true, numDocs=700
[junit4] 1> Adding 100 docs with commit=true, numDocs=800
[junit4] 1> Adding 100 docs with commit=true, numDocs=900
[junit4] 1> Adding 100 docs with commit=true, numDocs=1000
[junit4] 1> Adding 100 docs with commit=true, numDocs=1100
[junit4] 1> Adding 100 docs with commit=true, numDocs=1200
[junit4] 1> Adding 100 docs with commit=true, numDocs=1300
[junit4] 1> Adding 100 docs with commit=true, numDocs=1400
[junit4] 1> Adding 100 docs with commit=true, numDocs=1500
[junit4] 1> Adding 100 docs with commit=true, numDocs=1600
[junit4] 1> Adding 100 docs with commit=true, numDocs=1700
[junit4] 1> Adding 100 docs with commit=true, numDocs=1800
[junit4] 1> Adding 100 docs with commit=true, numDocs=1900
[junit4] 1> Adding 100 docs with commit=true, numDocs=2000
[junit4] 1> Adding 100 docs with commit=true, numDocs=2100
[junit4] 1> Adding 100 docs with commit=true, numDocs=2200
[junit4] 1> Adding 100 docs with commit=true, numDocs=2300
[junit4] 1> Adding 100 docs with commit=true, numDocs=2400
[junit4] 1> Adding 100 docs with commit=true, numDocs=2500
[junit4] 1> Adding 100 docs with commit=true, numDocs=2600
[junit4] 1> Adding 100 docs with commit=true, numDocs=2700
[junit4] 1> Adding 100 docs with commit=true, numDocs=2800
[junit4] 1> Adding 100 docs with commit=true, numDocs=2900
[junit4] 1> Adding 100 docs with commit=true, numDocs=3000
[junit4] 1> Adding 100 docs with commit=true, numDocs=3100
[junit4] 1> Adding 100 docs with commit=true, numDocs=3200
[junit4] 1> Adding 100 docs with commit=true, numDocs=3300
[junit4] 1> Adding 100 docs with commit=true, numDocs=3400
[junit4] 1> Adding 100 docs with commit=true, numDocs=3500
[junit4] 1> Adding 100 docs with commit=true, numDocs=3600
[junit4] 1> Adding 100 docs with commit=true, numDocs=3700
[junit4] 1> Adding 100 docs with commit=true, numDocs=3800
[junit4] 1> Adding 100 docs with commit=true, numDocs=3900
[junit4] 1> Adding 100 docs with commit=true, numDocs=4000
[junit4] 1> Adding 100 docs with commit=true, numDocs=4100
[junit4] 1> Adding 100 docs with commit=true, numDocs=4200
[junit4] 1> Adding 100 docs with commit=true, numDocs=4300
[junit4] 1> Adding 100 docs with commit=true, numDocs=4400
[junit4] 1> Adding 100 docs with commit=true, numDocs=4500
[junit4] 1> Adding 100 docs with commit=true, numDocs=4600
[junit4] 1> Adding 100 docs with commit=true, numDocs=4700
[junit4] 1> Adding 100 docs with commit=true, numDocs=4800
[junit4] 1> Adding 100 docs with commit=true, numDocs=4900
[junit4] 1> Adding 100 docs with commit=true, numDocs=5000
[junit4] 1> Adding 100 docs with commit=true, numDocs=5100
[junit4] 1> Adding 100 docs with commit=true, numDocs=5200
[junit4] 1> Adding 100 docs with commit=true, numDocs=5300
[junit4] 1> Adding 100 docs with commit=true, numDocs=5400
[junit4] 1> Adding 100 docs with commit=true, numDocs=5500
[junit4] 1> Adding 100 docs with commit=true, numDocs=5600
[junit4] 1> Adding 100 docs with commit=true, numDocs=5700
[junit4] 1> Adding 100 docs with commit=true, numDocs=5800
[junit4] 1> Adding 100 docs with commit=true, numDocs=5900
[junit4] 1> Adding 100 docs with commit=true, numDocs=6000
[junit4] 1> Adding 100 docs with commit=true, numDocs=6100
[junit4] 1> Adding 100 docs with commit=true, numDocs=6200
[junit4] 1> Adding 100 docs with commit=true, numDocs=6300
[junit4] 1> Adding 100 docs with commit=true, numDocs=6400
[junit4] 1> Adding 100 docs with commit=true, numDocs=6500
[junit4] 1> Adding 100 docs with commit=true, numDocs=6600
[junit4] 1> Adding 100 docs with commit=true, numDocs=6700
[junit4] 1> Adding 100 docs with commit=true, numDocs=6800
[junit4] 1> Adding 100 docs with commit=true, numDocs=6900
[junit4] 1> Adding 100 docs with commit=true, numDocs=7000
[junit4] 1> Adding 100 docs with commit=true, numDocs=7100
[junit4] 1> Adding 100 docs with commit=true, numDocs=7200
[junit4] 1> Adding 100 docs with commit=true, numDocs=7300
[junit4] 1> Adding 100 docs with commit=true, numDocs=7400
[junit4] 1> Adding 100 docs with commit=true, numDocs=7500
[junit4] 1> Adding 100 docs with commit=true, numDocs=7600
[junit4] 1> Adding 100 docs with commit=true, numDocs=7700
[junit4] 1> Adding 100 docs with commit=true, numDocs=7800
[junit4] 1> Adding 100 docs with commit=true, numDocs=7900
[junit4] 1> Adding 100 docs with commit=true, numDocs=8000
[junit4] 1> Adding 100 docs with commit=true, numDocs=8100
[junit4] 1> Adding 100 docs with commit=true, numDocs=8200
[junit4] 1> Adding 100 docs with commit=true, numDocs=8300
[junit4] 1> Adding 100 docs with commit=true, numDocs=8400
[junit4] 1> Adding 100 docs with commit=true, numDocs=8500
[junit4] 1> Adding 100 docs with commit=true, numDocs=8600
[junit4] 1> Adding 100 docs with commit=true, numDocs=8700
[junit4] 1> Adding 100 docs with commit=true, numDocs=8800
[junit4] 1> Adding 100 docs with commit=true, numDocs=8900
[junit4] 1> Adding 100 docs with commit=true, numDocs=9000
[junit4] 1> Adding 100 docs with commit=true, numDocs=9100
[junit4] 1> Adding 100 docs with commit=true, numDocs=9200
[junit4] 1> Adding 100 docs with commit=true, numDocs=9300
[junit4] 1> Adding 100 docs with commit=true, numDocs=9400
[junit4] 1> Adding 100 docs with commit=true, numDocs=9500
[junit4] 1> Adding 100 docs with commit=true, numDocs=9600
[junit4] 1> Adding 100 docs with commit=true, numDocs=9700
[junit4] 1> Adding 100 docs with commit=true, numDocs=9800
[junit4] 1> Adding 100 docs with commit=true, numDocs=9900
[junit4] 1> Adding 100 docs with commit=true, numDocs=10000
[junit4] 1> Adding 100 docs with commit=true, numDocs=10100
[junit4] 2> 406330 WARN
(cdcr-bootstrap-status-233-thread-1-processing-n:127.0.0.1:38859_solr
x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2)
[n:127.0.0.1:38859_solr c:cdcr-source s:shard1 r:core_node2
x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Bootstrap
process was not found on target collection: cdcr-target shard: shard1, leader:
https://127.0.0.1:40181/solr/cdcr-target_shard1_replica_n1/
[junit4] 1> Adding 100 docs with commit=true, numDocs=10200
[junit4] 1> Adding 100 docs with commit=true, numDocs=10300
[junit4] 1> Adding 100 docs with commit=true, numDocs=10400
[junit4] 1> Adding 100 docs with commit=true, numDocs=10500
[junit4] 1> Adding 100 docs with commit=true, numDocs=10600
[junit4] 1> Adding 100 docs with commit=true, numDocs=10700
[junit4] 1> Adding 100 docs with commit=true, numDocs=10800
[junit4] 2> 411504 ERROR
(updateExecutor-206-thread-1-processing-n:127.0.0.1:40181_solr
x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
[n:127.0.0.1:40181_solr c:cdcr-target s:shard1 r:core_node2
x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation
failed
[junit4] 2> java.util.concurrent.ExecutionException:
java.lang.AssertionError
[junit4] 2> at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at
java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:646)
[junit4] 2> at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.AssertionError
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:789)
[junit4] 2> at
org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:716)
[junit4] 2> at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 1> Adding 100 docs with commit=true, numDocs=10900
[junit4] 1> Adding 100 docs with commit=true, numDocs=11000
[junit4] 1> Adding 100 docs with commit=true, numDocs=11100
[junit4] 1> Adding 100 docs with commit=true, numDocs=11200
[junit4] 1> Adding 100 docs with commit=true, numDocs=11300
[junit4] 1> Adding 100 docs with commit=true, numDocs=11400
[junit4] 1> Adding 100 docs with commit=true, numDocs=11500
[junit4] 1> Adding 100 docs with commit=true, numDocs=11600
[junit4] 1> Adding 100 docs with commit=true, numDocs=11700
[junit4] 1> Adding 100 docs with commit=true, numDocs=11800
[junit4] 1> Adding 100 docs with commit=true, numDocs=11900
[junit4] 1> Adding 100 docs with commit=true, numDocs=12000
[junit4] 1> Adding 100 docs with commit=true, numDocs=12100
[junit4] 1> Adding 100 docs with commit=true, numDocs=12200
[junit4] 1> Adding 100 docs with commit=true, numDocs=12300
[junit4] 1> Adding 100 docs with commit=true, numDocs=12400
[junit4] 1> Adding 100 docs with commit=true, numDocs=12500
[junit4] 1> Adding 100 docs with commit=true, numDocs=12600
[junit4] 1> Adding 100 docs with commit=true, numDocs=12700
[junit4] 1> Adding 100 docs with commit=true, numDocs=12800
[junit4] 1> Adding 100 docs with commit=true, numDocs=12900
[junit4] 1> Adding 100 docs with commit=true, numDocs=13000
[junit4] 1> Adding 100 docs with commit=true, numDocs=13100
[junit4] 1> Adding 100 docs with commit=true, numDocs=13200
[junit4] 1> Adding 100 docs with commit=true, numDocs=13300
[junit4] 1> Adding 100 docs with commit=true, numDocs=13400
[junit4] 1> Adding 100 docs with commit=true, numDocs=13500
[junit4] 1> Adding 100 docs with commit=true, numDocs=13600
[junit4] 1> Adding 100 docs with commit=true, numDocs=13700
[junit4] 1> Adding 100 docs with commit=true, numDocs=13800
[junit4] 1> Adding 100 docs with commit=true, numDocs=13900
[junit4] 1> Adding 100 docs with commit=true, numDocs=14000
[junit4] 1> Adding 100 docs with commit=true, numDocs=14100
[junit4] 1> Adding 100 docs with commit=true, numDocs=14200
[junit4] 1> Adding 100 docs with commit=true, numDocs=14300
[junit4] 1> Adding 100 docs with commit=true, numDocs=14400
[junit4] 1> Adding 100 docs with commit=true, numDocs=14500
[junit4] 1> Adding 100 docs with commit=true, numDocs=14600
[junit4] 1> Adding 100 docs with commit=true, numDocs=14700
[junit4]
[...truncated too long message...]
andler @ source_collection:shard1
[junit4] 2> 4170425 INFO
(zkCallback-837-thread-1-processing-n:127.0.0.1:34461_) [n:127.0.0.1:34461_
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 4170481 INFO
(zkCallback-837-thread-1-processing-n:127.0.0.1:34461_) [n:127.0.0.1:34461_
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/source_collection/state.json] for collection
[source_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4171925 WARN
(zkCallback-898-thread-1-processing-n:127.0.0.1:36483_) [n:127.0.0.1:36483_
] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to
ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4171926 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.e.j.s.AbstractConnector Stopped
ServerConnector@5251415a{HTTP/1.1,[http/1.1]}{127.0.0.1:36483}
[junit4] 2> 4171926 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.e.j.s.h.ContextHandler Stopped
o.e.j.s.ServletContextHandler@59251a6d{/,null,UNAVAILABLE}
[junit4] 2> 4171927 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 34461
[junit4] 2> 4171927 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2023714256
[junit4] 2> 4171927 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.node, tag=null
[junit4] 2> 4171927 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@6179532d: rootName = null,
domain = solr.node, service url = null, agent id = null] for registry solr.node
/ com.codahale.metrics.MetricRegistry@2feb5fc5
[junit4] 2> 4171944 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.jvm, tag=null
[junit4] 2> 4171944 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@29125df7: rootName = null,
domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm /
com.codahale.metrics.MetricRegistry@5c00bd90
[junit4] 2> 4171945 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.jetty, tag=null
[junit4] 2> 4171945 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@796462c2: rootName = null,
domain = solr.jetty, service url = null, agent id = null] for registry
solr.jetty / com.codahale.metrics.MetricRegistry@2c259142
[junit4] 2> 4171946 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.cluster, tag=null
[junit4] 2> 4171948 INFO (coreCloseExecutor-1866-thread-1)
[n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3
x:source_collection_shard1_replica_n1] o.a.s.c.SolrCore
[source_collection_shard1_replica_n1] CLOSING SolrCore
org.apache.solr.core.SolrCore@e205d92
[junit4] 2> 4171948 INFO (coreCloseExecutor-1866-thread-1)
[n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3
x:source_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.core.source_collection.shard1.replica_n1,
tag=237002130
[junit4] 2> 4171948 INFO (coreCloseExecutor-1866-thread-1)
[n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3
x:source_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing
reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4a3a2b40: rootName
= null, domain = solr.core.source_collection.shard1.replica_n1, service url =
null, agent id = null] for registry
solr.core.source_collection.shard1.replica_n1 /
com.codahale.metrics.MetricRegistry@611a09a4
[junit4] 2> 4171982 INFO (coreCloseExecutor-1866-thread-1)
[n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3
x:source_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.collection.source_collection.shard1.leader,
tag=237002130
[junit4] 2> 4171982 INFO (coreCloseExecutor-1866-thread-1)
[n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3
x:source_collection_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Solr core is
being closed - shutting down CDCR handler @ source_collection:shard1
[junit4] 2> 4171984 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.c.Overseer Overseer
(id=98565184670859271-127.0.0.1:34461_-n_0000000001) closing
[junit4] 2> 4171997 INFO
(OverseerStateUpdate-98565184670859271-127.0.0.1:34461_-n_0000000001)
[n:127.0.0.1:34461_ ] o.a.s.c.Overseer Overseer Loop exiting :
127.0.0.1:34461_
[junit4] 2> 4171999 WARN
(zkCallback-837-thread-1-processing-n:127.0.0.1:34461_) [n:127.0.0.1:34461_
] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to
ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4172000 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.e.j.s.AbstractConnector Stopped
ServerConnector@46bd95a1{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4172000 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.e.j.s.h.ContextHandler Stopped
o.e.j.s.ServletContextHandler@106d5432{/,null,UNAVAILABLE}
[junit4] 2> 4172000 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.SolrTestCaseJ4 ###Ending testPartialReplication
[junit4] 2> 4172000 ERROR
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so
ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4172001 INFO
(TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57])
[ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42587 42587
[junit4] 2> 4172121 INFO (Thread-123659) [ ] o.a.s.c.ZkTestServer
connecting to 127.0.0.1:42587 42587
[junit4] 2> 4172121 WARN (Thread-123659) [ ] o.a.s.c.ZkTestServer
Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2> 4
/solr/collections/source_collection/leaders/shard1/leader
[junit4] 2> 3 /solr/security.json
[junit4] 2> 2
/solr/collections/tmp_collection/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 4
/solr/collections/source_collection/cdcr/state/buffer
[junit4] 2> 4
/solr/collections/source_collection/cdcr/state/process
[junit4] 2> 3 /solr/collections/source_collection/state.json
[junit4] 2> 2 /solr/collections/tmp_collection/state.json
[junit4] 2> 2
/solr/collections/source_collection/leader_elect/shard1/election/98565184670859271-core_node3-n_0000000000
[junit4] 2> 2
/solr/collections/tmp_collection/cdcr/state/buffer
[junit4] 2> 2
/solr/collections/tmp_collection/cdcr/state/process
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/collections
[junit4] 2> 21 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationHandlerTest_414FC989A1A4EA57-001
[junit4] 2> Aug 29, 2017 5:43:03 AM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked
thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene70,
sim=RandomSimilarity(queryNorm=true): {}, locale=en,
timezone=America/Argentina/Rio_Gallegos
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation
1.8.0_144 (64-bit)/cpus=4,threads=1,free=220602664,total=523763712
[junit4] 2> NOTE: All tests run in this JVM: [MergeStrategyTest,
HdfsBasicDistributedZkTest, ZkStateReaderTest, JavabinLoaderTest,
TestHashPartitioner, TestFiltering, ScriptEngineTest,
CloneFieldUpdateProcessorFactoryTest, TestObjectReleaseTracker,
TestSubQueryTransformer, AlternateDirectoryTest,
SchemaVersionSpecificBehaviorTest, TestManagedSchema,
TestDelegationWithHadoopAuth, DocValuesMultiTest, TestCoreContainer,
TestSerializedLuceneMatchVersion, TestConfigSets, MoveReplicaHDFSTest,
SimpleCollectionCreateDeleteTest, DisMaxRequestHandlerTest,
CleanupOldIndexTest, ConnectionReuseTest, CollectionsAPIDistributedZkTest,
BlockJoinFacetRandomTest, TestStandardQParsers, TestSchemaResource,
BasicFunctionalityTest, TestCoreAdminApis, CdcrReplicationHandlerTest]
[junit4] Completed [639/731 (3!)] on J1 in 144.57s, 5 tests, 1 failure <<<
FAILURES!
[...truncated 46471 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
