Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/252/
4 tests failed.
FAILED:
org.apache.solr.cloud.LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout
Error Message:
Could not start container:7000. Reason:class
org.apache.solr.common.SolrException Error CREATEing SolrCore
'collection1_shard1': Unable to create core [collection1_shard1] Caused by:
Timed out getting shard id for core: collection1_shard1
Stack Trace:
java.lang.AssertionError: Could not start container:7000. Reason:class
org.apache.solr.common.SolrException Error CREATEing SolrCore
'collection1_shard1': Unable to create core [collection1_shard1] Caused by:
Timed out getting shard id for core: collection1_shard1
at
__randomizedtesting.SeedInfo.seed([36E56268232D3CA4:9FD2C7CB0408B44F]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.solr.cloud.LeaderElectionIntegrationTest.setUp(LeaderElectionIntegrationTest.java:102)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:905)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionIntegrationTest
Error Message:
11 threads leaked from SUITE scope at
org.apache.solr.cloud.LeaderElectionIntegrationTest: 1) Thread[id=6435,
name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[36E56268232D3CA4]-EventThread,
state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
2) Thread[id=6446, name=zkCallback-1213-thread-6, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 3) Thread[id=6434,
name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[36E56268232D3CA4]-SendThread(127.0.0.1:53048),
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at
java.lang.Thread.sleep(Native Method) at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 4)
Thread[id=6444, name=zkCallback-1213-thread-4, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 5) Thread[id=6447,
name=zkCallback-1213-thread-7, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 6) Thread[id=6438,
name=OverseerCollectionConfigSetProcessor-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=TIMED_WAITING, group=Overseer collection creation process.] at
java.lang.Thread.sleep(Native Method) at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
at
org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:328)
at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:167)
at java.lang.Thread.run(Thread.java:745) 7) Thread[id=6440,
name=Thread-2184, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
at java.lang.Object.wait(Native Method) at
java.lang.Object.wait(Object.java:502) at
org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 8)
Thread[id=6445, name=zkCallback-1213-thread-5, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 9) Thread[id=6439,
name=OverseerHdfsCoreFailoverThread-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at
java.lang.Thread.sleep(Native Method) at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745) 10) Thread[id=6443,
name=zkCallback-1213-thread-3, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 11) Thread[id=6437,
name=OverseerStateUpdate-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=TIMED_WAITING, group=Overseer state updater.] at
java.lang.Thread.sleep(Native Method) at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
at
org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:309)
at
org.apache.solr.cloud.DistributedQueue.firstChild(DistributedQueue.java:288)
at
org.apache.solr.cloud.DistributedQueue.firstElement(DistributedQueue.java:357)
at
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164)
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:142)
at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196)
at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 11 threads leaked from
SUITE scope at org.apache.solr.cloud.LeaderElectionIntegrationTest:
1) Thread[id=6435,
name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[36E56268232D3CA4]-EventThread,
state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
2) Thread[id=6446, name=zkCallback-1213-thread-6, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=6434,
name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[36E56268232D3CA4]-SendThread(127.0.0.1:53048),
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
4) Thread[id=6444, name=zkCallback-1213-thread-4, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=6447, name=zkCallback-1213-thread-7, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
6) Thread[id=6438,
name=OverseerCollectionConfigSetProcessor-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=TIMED_WAITING, group=Overseer collection creation process.]
at java.lang.Thread.sleep(Native Method)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
at
org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:328)
at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:167)
at java.lang.Thread.run(Thread.java:745)
7) Thread[id=6440, name=Thread-2184, state=WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253)
8) Thread[id=6445, name=zkCallback-1213-thread-5, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
9) Thread[id=6439,
name=OverseerHdfsCoreFailoverThread-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
10) Thread[id=6443, name=zkCallback-1213-thread-3, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
11) Thread[id=6437,
name=OverseerStateUpdate-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=TIMED_WAITING, group=Overseer state updater.]
at java.lang.Thread.sleep(Native Method)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
at
org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:309)
at
org.apache.solr.cloud.DistributedQueue.firstChild(DistributedQueue.java:288)
at
org.apache.solr.cloud.DistributedQueue.firstElement(DistributedQueue.java:357)
at
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164)
at
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:142)
at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([36E56268232D3CA4]:0)
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionIntegrationTest
Error Message:
There are still zombie threads that couldn't be terminated: 1)
Thread[id=6446, name=zkCallback-1213-thread-6, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 2) Thread[id=6434,
name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[36E56268232D3CA4]-SendThread(127.0.0.1:53048),
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at
java.lang.Thread.sleep(Native Method) at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
3) Thread[id=6444, name=zkCallback-1213-thread-4, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 4) Thread[id=6447,
name=zkCallback-1213-thread-7, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 5) Thread[id=6440,
name=Thread-2184, state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
at java.lang.Object.wait(Native Method) at
java.lang.Object.wait(Object.java:502) at
org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 6)
Thread[id=6445, name=zkCallback-1213-thread-5, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) 7) Thread[id=6439,
name=OverseerHdfsCoreFailoverThread-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at
java.lang.Throwable.fillInStackTrace(Native Method) at
java.lang.Throwable.fillInStackTrace(Throwable.java:783) at
java.lang.Throwable.<init>(Throwable.java:265) at
java.lang.Exception.<init>(Exception.java:66) at
java.lang.InterruptedException.<init>(InterruptedException.java:67) at
java.lang.Thread.sleep(Native Method) at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745) 8) Thread[id=6443,
name=zkCallback-1213-thread-3, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest] at
sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
1) Thread[id=6446, name=zkCallback-1213-thread-6, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=6434,
name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[36E56268232D3CA4]-SendThread(127.0.0.1:53048),
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
at java.lang.Thread.sleep(Native Method)
at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
3) Thread[id=6444, name=zkCallback-1213-thread-4, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=6447, name=zkCallback-1213-thread-7, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=6440, name=Thread-2184, state=WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253)
6) Thread[id=6445, name=zkCallback-1213-thread-5, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
7) Thread[id=6439,
name=OverseerHdfsCoreFailoverThread-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
at java.lang.Throwable.<init>(Throwable.java:265)
at java.lang.Exception.<init>(Exception.java:66)
at java.lang.InterruptedException.<init>(InterruptedException.java:67)
at java.lang.Thread.sleep(Native Method)
at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
8) Thread[id=6443, name=zkCallback-1213-thread-3, state=TIMED_WAITING,
group=TGRP-LeaderElectionIntegrationTest]
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([36E56268232D3CA4]:0)
FAILED: org.apache.solr.cloud.TestCloudBackupRestore.test
Error Message:
Error from server at http://127.0.0.1:54829/solr: Backup directory already
exists:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-002/mytestbackup
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://127.0.0.1:54829/solr: Backup directory already exists:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-002/mytestbackup
at
__randomizedtesting.SeedInfo.seed([36E56268232D3CA4:BEB15DB28DD1515C]:0)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:404)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:357)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1228)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:998)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:934)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at
org.apache.solr.cloud.TestCloudBackupRestore.testBackupAndRestore(TestCloudBackupRestore.java:153)
at
org.apache.solr.cloud.TestCloudBackupRestore.test(TestCloudBackupRestore.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11209 lines...]
[junit4] Suite: org.apache.solr.cloud.TestCloudBackupRestore
[junit4] 2> Creating dataDir:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/init-core-data-001
[junit4] 2> 580366 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 580367 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 580382 INFO (Thread-1658) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 580383 INFO (Thread-1658) [ ] o.a.s.c.ZkTestServer
Starting server
[junit4] 2> 580482 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.ZkTestServer start zk server on port:44314
[junit4] 2> 580483 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 580485 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 580492 INFO (zkCallback-567-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@7ca347a4
name:ZooKeeperConnection Watcher:127.0.0.1:44314 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 580492 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 580492 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 580493 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
[junit4] 2> 580535 INFO (jetty-launcher-566-thread-1) [ ]
o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 580535 INFO (jetty-launcher-566-thread-2) [ ]
o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 580536 INFO (jetty-launcher-566-thread-1) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@3f12ace1{/solr,null,AVAILABLE}
[junit4] 2> 580537 INFO (jetty-launcher-566-thread-1) [ ]
o.e.j.s.ServerConnector Started
ServerConnector@111be574{HTTP/1.1,[http/1.1]}{127.0.0.1:49276}
[junit4] 2> 580537 INFO (jetty-launcher-566-thread-1) [ ]
o.e.j.s.Server Started @584547ms
[junit4] 2> 580537 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
hostPort=49276}
[junit4] 2> 580538 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 580538 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node1'
[junit4] 2> 580538 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 580538 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
system property or JNDI)
[junit4] 2> 580538 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 580542 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 580570 INFO (jetty-launcher-566-thread-2) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@5d8e0b65{/solr,null,AVAILABLE}
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.e.j.s.ServerConnector Started
ServerConnector@597fc99a{HTTP/1.1,[http/1.1]}{127.0.0.1:54829}
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.e.j.s.Server Started @584580ms
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
hostPort=54829}
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2'
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
system property or JNDI)
[junit4] 2> 580571 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 580588 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 580631 INFO (zkCallback-569-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@3ef32960
name:ZooKeeperConnection Watcher:127.0.0.1:44314/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 580632 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 580632 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 580649 INFO (zkCallback-568-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@9372e1d name:ZooKeeperConnection
Watcher:127.0.0.1:44314/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 580649 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 580649 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 580649 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 580650 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 580655 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x155190f6cb50001, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 580655 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.CorePropertiesLocator Config-defined core root directory:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2/.
[junit4] 2> 580655 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.CoreContainer New CoreContainer 1807218367
[junit4] 2> 580655 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.CoreContainer Loading cores into CoreContainer
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2]
[junit4] 2> 580659 WARN (jetty-launcher-566-thread-2) [ ]
o.a.s.c.CoreContainer Couldn't add files from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2/lib
to classpath:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2/lib
[junit4] 2> 580659 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections :
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime :
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 580660 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 580660 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 580660 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.l.LogWatcher Registering Log Listener [Log4j
(org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 580660 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44314/solr
[junit4] 2> 580661 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 580661 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 580678 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 580682 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.CorePropertiesLocator Config-defined core root directory:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node1/.
[junit4] 2> 580682 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.CoreContainer New CoreContainer 1066668542
[junit4] 2> 580682 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.CoreContainer Loading cores into CoreContainer
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node1]
[junit4] 2> 580682 WARN (jetty-launcher-566-thread-1) [ ]
o.a.s.c.CoreContainer Couldn't add files from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node1/lib
to classpath:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node1/lib
[junit4] 2> 580682 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections :
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime :
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 580683 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 580683 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 580683 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.l.LogWatcher Registering Log Listener [Log4j
(org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 580683 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44314/solr
[junit4] 2> 580683 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 580683 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 580698 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 580719 INFO (zkCallback-575-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@3add7555
name:ZooKeeperConnection Watcher:127.0.0.1:44314 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 580722 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 580722 INFO (jetty-launcher-566-thread-1) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 580756 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.ZkController Added new OnReconnect
listener org.apache.solr.cloud.ZkController$$Lambda$27/1109178346@335fa9d7
[junit4] 2> 580765 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ConnectionManager Waiting for client to
connect to ZooKeeper
[junit4] 2> 580765 INFO (zkCallback-572-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@d497aca name:ZooKeeperConnection
Watcher:127.0.0.1:44314 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 580766 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 580766 INFO (jetty-launcher-566-thread-2) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 580791 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.ZkController Added new OnReconnect
listener org.apache.solr.cloud.ZkController$$Lambda$27/1109178346@3f44c341
[junit4] 2> 580803 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ConnectionManager Waiting for client to
connect to ZooKeeper
[junit4] 2> 580804 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@6d69bc5b
name:ZooKeeperConnection Watcher:127.0.0.1:44314/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 580804 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ConnectionManager Client is connected to
ZooKeeper
[junit4] 2> 580808 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 580813 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-queue-work
[junit4] 2> 580815 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-running
[junit4] 2> 580816 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-completed
[junit4] 2> 580816 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-failure
[junit4] 2> 580817 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 580818 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 580820 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 580823 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@20d0209e
name:ZooKeeperConnection Watcher:127.0.0.1:44314/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 580824 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 580826 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ConnectionManager Client is connected to
ZooKeeper
[junit4] 2> 580827 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 580828 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader Updating cluster state
from ZooKeeper...
[junit4] 2> 580828 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster
properties
[junit4] 2> 580828 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (0)
[junit4] 2> 580829 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader Updating cluster state
from ZooKeeper...
[junit4] 2> 580830 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster
properties
[junit4] 2> 580830 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (0)
[junit4] 2> 580830 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 580831 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/election
[junit4] 2> 580834 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/election
[junit4] 2> 580837 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 580838 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 580838 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.LeaderElector Joined leadership election
with path:
/overseer_elect/election/96010521097338885-127.0.0.1:49276_solr-n_0000000000
[junit4] 2> 580838 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.LeaderElector Joined leadership election
with path:
/overseer_elect/election/96010521097338886-127.0.0.1:54829_solr-n_0000000001
[junit4] 2> 580838 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerElectionContext I am going to be
the leader 127.0.0.1:49276_solr
[junit4] 2> 580838 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/leader
[junit4] 2> 580839 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.LeaderElector Watching path
/overseer_elect/election/96010521097338885-127.0.0.1:49276_solr-n_0000000000 to
know if I could be the leader
[junit4] 2> 580839 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:54829_solr
[junit4] 2> 580839 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.SolrZkClient makePath:
/live_nodes/127.0.0.1:54829_solr
[junit4] 2> 580840 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer Overseer
(id=96010521097338885-127.0.0.1:49276_solr-n_0000000000) starting
[junit4] 2> 580840 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader A live node change:
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes],
has occurred - updating... (live nodes size: [0])
[junit4] 2> 580853 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (1)
[junit4] 2> 580854 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.CoreContainer Security conf doesn't exist.
Skipping setup for authorization module.
[junit4] 2> 580854 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.CoreContainer No authentication plugin
used.
[junit4] 2> 580855 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.CorePropertiesLocator Looking for core
definitions underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2/.
[junit4] 2> 580855 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
definitions
[junit4] 2> 580858 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.s.SolrDispatchFilter
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0
[junit4] 2> 580858 INFO (jetty-launcher-566-thread-2)
[n:127.0.0.1:54829_solr ] o.a.s.s.SolrDispatchFilter
SolrDispatchFilter.init() done
[junit4] 2> 580859 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/overseer/queue-work
[junit4] 2> 580861 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 580862 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 580863 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader A live node change:
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes],
has occurred - updating... (live nodes size: [0])
[junit4] 2> 580863 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (1) -> (1)
[junit4] 2> 580864 INFO
(OverseerCollectionConfigSetProcessor-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerTaskProcessor Process current
queue of overseer operations
[junit4] 2> 580871 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (0) -> (1)
[junit4] 2> 580871 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.ZkController Publish
node=127.0.0.1:49276_solr as DOWN
[junit4] 2> 580872 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:49276_solr
[junit4] 2> 580872 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/live_nodes/127.0.0.1:49276_solr
[junit4] 2> 580873 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer processMessage: queueSize: 1,
message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:49276_solr"} current state version: 0
[junit4] 2> 580873 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for
node: 127.0.0.1:49276_solr
[junit4] 2> 580873 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader A live node change:
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes],
has occurred - updating... (live nodes size: [1])
[junit4] 2> 580873 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader A live node change:
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes],
has occurred - updating... (live nodes size: [1])
[junit4] 2> 580874 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (1) -> (2)
[junit4] 2> 580877 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from
ZooKeeper... (1) -> (2)
[junit4] 2> 580897 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.CoreContainer Security conf doesn't exist.
Skipping setup for authorization module.
[junit4] 2> 580897 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.CoreContainer No authentication plugin
used.
[junit4] 2> 580897 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.CorePropertiesLocator Looking for core
definitions underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node1/.
[junit4] 2> 580898 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
definitions
[junit4] 2> 580898 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.s.SolrDispatchFilter
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0
[junit4] 2> 580898 INFO (jetty-launcher-566-thread-1)
[n:127.0.0.1:49276_solr ] o.a.s.s.SolrDispatchFilter
SolrDispatchFilter.init() done
[junit4] 2> 580898 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 580899 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 580903 INFO (zkCallback-580-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@26e6f734
name:ZooKeeperConnection Watcher:127.0.0.1:44314 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 580904 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 580904 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 580905 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.s.i.CloudSolrClient Final constructed zkHost string:
127.0.0.1:44314/solr
[junit4] 2> 580905 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 580905 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 580907 INFO (zkCallback-583-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@5926062b
name:ZooKeeperConnection Watcher:127.0.0.1:44314/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 580907 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 580907 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 580907 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 580907 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 580908 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 580909 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
[junit4] 2> 580910 INFO
(SUITE-TestCloudBackupRestore-seed#[36E56268232D3CA4]-worker) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 580923 INFO
(TEST-TestCloudBackupRestore.test-seed#[36E56268232D3CA4]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 580925 INFO (qtp1960046155-3968) [n:127.0.0.1:49276_solr ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params
replicationFactor=2&property.customKey=customValue&collection.configName=conf1&maxShardsPerNode=2&autoAddReplicas=true&name=backuprestore&action=CREATE&numShards=2&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 580927 INFO
(OverseerCollectionConfigSetProcessor-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerTaskProcessor Overseer Collection
Message Handler: Get the message
id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "name":"backuprestore",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"true",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "property.customKey":"customValue",
[junit4] 2> "operation":"create"}
[junit4] 2> 580927 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler
OverseerCollectionMessageHandler.processMessage : create , {
[junit4] 2> "name":"backuprestore",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"true",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "property.customKey":"customValue",
[junit4] 2> "operation":"create"}
[junit4] 2> 580927 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler Create
collection backuprestore
[junit4] 2> 580928 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler creating
collections conf node /collections/backuprestore
[junit4] 2> 580928 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.SolrZkClient makePath:
/collections/backuprestore
[junit4] 2> 580929 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader A collections change:
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections],
has occurred - updating...
[junit4] 2> 580929 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader A collections change:
[WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections],
has occurred - updating...
[junit4] 2> 580929 INFO (zkCallback-583-thread-1) [ ]
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/collections], has occurred - updating...
[junit4] 2> 580936 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer processMessage: queueSize: 1,
message = {
[junit4] 2> "name":"backuprestore",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"true",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "property.customKey":"customValue",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 580936 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.ClusterStateMutator building a new
cName: backuprestore
[junit4] 2> 580937 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.ZkStateWriter going to create_collection
/collections/backuprestore/state.json
[junit4] 2> 581036 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler Creating
SolrCores for new collection backuprestore, shardNames [shard1, shard2] ,
replicationFactor : 2
[junit4] 2> 581036 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler Creating
core backuprestore_shard1_replica1 as part of shard shard1 of collection
backuprestore on 127.0.0.1:54829_solr
[junit4] 2> 581037 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler Creating
core backuprestore_shard2_replica1 as part of shard shard2 of collection
backuprestore on 127.0.0.1:54829_solr
[junit4] 2> 581037 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler Creating
core backuprestore_shard2_replica2 as part of shard shard2 of collection
backuprestore on 127.0.0.1:49276_solr
[junit4] 2> 581037 INFO
(OverseerThreadFactory-2405-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.OverseerCollectionMessageHandler Creating
core backuprestore_shard1_replica2 as part of shard shard1 of collection
backuprestore on 127.0.0.1:49276_solr
[junit4] 2> 581055 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr ]
o.a.s.h.a.CoreAdminOperation core create command
property.customKey=customValue&qt=/admin/cores&collection.configName=conf1&name=backuprestore_shard1_replica2&action=CREATE&numShards=2&collection=backuprestore&shard=shard1&wt=javabin&version=2
[junit4] 2> 581055 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr ]
o.a.s.c.CoreDescriptor Created CoreDescriptor:
{name=backuprestore_shard1_replica2, collection.configName=conf1,
config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true,
configSetProperties=configsetprops.json, collection=backuprestore,
customKey=customValue, numShards=2, dataDir=data/, shard=shard1}
[junit4] 2> 581056 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2] o.a.s.c.ZkController
publishing state=down
[junit4] 2> 581057 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer processMessage: queueSize: 1,
message = {
[junit4] 2> "core":"backuprestore_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:49276/solr";,
[junit4] 2> "node_name":"127.0.0.1:49276_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 581057 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2
message={
[junit4] 2> "core":"backuprestore_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:49276/solr";,
[junit4] 2> "node_name":"127.0.0.1:49276_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"}
[junit4] 2> 581057 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2] o.a.s.c.ZkController
Registering watch for collection backuprestore
[junit4] 2> 581058 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr ]
o.a.s.h.a.CoreAdminOperation core create command
property.customKey=customValue&qt=/admin/cores&collection.configName=conf1&name=backuprestore_shard2_replica1&action=CREATE&numShards=2&collection=backuprestore&shard=shard2&wt=javabin&version=2
[junit4] 2> 581058 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr ]
o.a.s.h.a.CoreAdminOperation core create command
property.customKey=customValue&qt=/admin/cores&collection.configName=conf1&name=backuprestore_shard1_replica1&action=CREATE&numShards=2&collection=backuprestore&shard=shard1&wt=javabin&version=2
[junit4] 2> 581058 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr ]
o.a.s.c.CoreDescriptor Created CoreDescriptor:
{name=backuprestore_shard2_replica1, collection.configName=conf1,
config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true,
configSetProperties=configsetprops.json, collection=backuprestore,
customKey=customValue, numShards=2, dataDir=data/, shard=shard2}
[junit4] 2> 581058 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr ]
o.a.s.h.a.CoreAdminOperation core create command
property.customKey=customValue&qt=/admin/cores&collection.configName=conf1&name=backuprestore_shard2_replica2&action=CREATE&numShards=2&collection=backuprestore&shard=shard2&wt=javabin&version=2
[junit4] 2> 581058 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr ]
o.a.s.c.CoreDescriptor Created CoreDescriptor:
{name=backuprestore_shard2_replica2, collection.configName=conf1,
config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true,
configSetProperties=configsetprops.json, collection=backuprestore,
customKey=customValue, numShards=2, dataDir=data/, shard=shard2}
[junit4] 2> 581059 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr ]
o.a.s.c.CoreDescriptor Created CoreDescriptor:
{name=backuprestore_shard1_replica1, collection.configName=conf1,
config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true,
configSetProperties=configsetprops.json, collection=backuprestore,
customKey=customValue, numShards=2, dataDir=data/, shard=shard1}
[junit4] 2> 581059 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1] o.a.s.c.ZkController
publishing state=down
[junit4] 2> 581059 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1] o.a.s.c.ZkController
publishing state=down
[junit4] 2> 581059 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2]
o.a.s.c.c.ZkStateReader Add data for [backuprestore] ver [0]
[junit4] 2> 581059 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1] o.a.s.c.ZkController
Registering watch for collection backuprestore
[junit4] 2> 581059 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica2] o.a.s.c.ZkController
publishing state=down
[junit4] 2> 581060 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1]
o.a.s.c.c.ZkStateReader Add data for [backuprestore] ver [0]
[junit4] 2> 581060 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1] o.a.s.c.ZkController
Registering watch for collection backuprestore
[junit4] 2> 581060 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1] o.a.s.c.ZkController
Waiting for coreNodeName for core backuprestore_shard1_replica1 in collection
backuprestore to be assigned
[junit4] 2> 581060 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer processMessage: queueSize: 2,
message = {
[junit4] 2> "core":"backuprestore_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:54829/solr";,
[junit4] 2> "node_name":"127.0.0.1:54829_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 581060 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2
message={
[junit4] 2> "core":"backuprestore_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:54829/solr";,
[junit4] 2> "node_name":"127.0.0.1:54829_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"}
[junit4] 2> 581061 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica2] o.a.s.c.ZkController
Registering watch for collection backuprestore
[junit4] 2> 581061 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica2] o.a.s.c.ZkController
Waiting for coreNodeName for core backuprestore_shard2_replica2 in collection
backuprestore to be assigned
[junit4] 2> 581061 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer processMessage: queueSize: 2,
message = {
[junit4] 2> "core":"backuprestore_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:54829/solr";,
[junit4] 2> "node_name":"127.0.0.1:54829_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 581061 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2
message={
[junit4] 2> "core":"backuprestore_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:54829/solr";,
[junit4] 2> "node_name":"127.0.0.1:54829_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"}
[junit4] 2> 581062 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.Overseer processMessage: queueSize: 1,
message = {
[junit4] 2> "core":"backuprestore_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:49276/solr";,
[junit4] 2> "node_name":"127.0.0.1:49276_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 581062 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2
message={
[junit4] 2> "core":"backuprestore_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:49276/solr";,
[junit4] 2> "node_name":"127.0.0.1:49276_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"backuprestore",
[junit4] 2> "operation":"state"}
[junit4] 2> 581067 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2] o.a.s.c.ZkController
Waiting for coreNodeName for core backuprestore_shard1_replica2 in collection
backuprestore to be assigned
[junit4] 2> 581082 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1] o.a.s.c.ZkController
Waiting for coreNodeName for core backuprestore_shard2_replica1 in collection
backuprestore to be assigned
[junit4] 2> 581163 INFO
(OverseerStateUpdate-96010521097338885-127.0.0.1:49276_solr-n_0000000000)
[n:127.0.0.1:49276_solr ] o.a.s.c.o.ZkStateWriter going to update_collection
/collections/backuprestore/state.json version: 0
[junit4] 2> 581164 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader A cluster state change:
[WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/backuprestore/state.json] for collection [backuprestore] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 581164 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader A cluster state change:
[WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/backuprestore/state.json] for collection [backuprestore] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 581164 INFO
(zkCallback-577-thread-1-processing-n:127.0.0.1:54829_solr)
[n:127.0.0.1:54829_solr ] o.a.s.c.c.ZkStateReader Updating data for
[backuprestore] from [0] to [1]
[junit4] 2> 581164 INFO
(zkCallback-576-thread-1-processing-n:127.0.0.1:49276_solr)
[n:127.0.0.1:49276_solr ] o.a.s.c.c.ZkStateReader Updating data for
[backuprestore] from [0] to [1]
[junit4] 2> 581164 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1] o.a.s.c.ZkController
waiting to find shard id in clusterstate for backuprestore_shard1_replica1
[junit4] 2> 581164 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1] o.a.s.c.ZkController
Check for collection zkNode:backuprestore
[junit4] 2> 581164 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica2] o.a.s.c.ZkController
waiting to find shard id in clusterstate for backuprestore_shard2_replica2
[junit4] 2> 581164 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1] o.a.s.c.ZkController
waiting to find shard id in clusterstate for backuprestore_shard2_replica1
[junit4] 2> 581164 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica2] o.a.s.c.ZkController
Check for collection zkNode:backuprestore
[junit4] 2> 581164 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1] o.a.s.c.ZkController
Check for collection zkNode:backuprestore
[junit4] 2> 581164 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2] o.a.s.c.ZkController
waiting to find shard id in clusterstate for backuprestore_shard1_replica2
[junit4] 2> 581164 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2] o.a.s.c.ZkController
Check for collection zkNode:backuprestore
[junit4] 2> 581166 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1] o.a.s.c.ZkController
Collection zkNode exists
[junit4] 2> 581166 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1]
o.a.s.c.c.ZkStateReader Load collection config from:
[/collections/backuprestore]
[junit4] 2> 581166 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica2] o.a.s.c.ZkController
Collection zkNode exists
[junit4] 2> 581166 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1] o.a.s.c.ZkController
Collection zkNode exists
[junit4] 2> 581166 INFO (qtp1960046155-3971) [n:127.0.0.1:49276_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica2]
o.a.s.c.c.ZkStateReader Load collection config from:
[/collections/backuprestore]
[junit4] 2> 581166 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1]
o.a.s.c.c.ZkStateReader Load collection config from:
[/collections/backuprestore]
[junit4] 2> 581166 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2] o.a.s.c.ZkController
Collection zkNode exists
[junit4] 2> 581166 INFO (qtp1960046155-3966) [n:127.0.0.1:49276_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica2]
o.a.s.c.c.ZkStateReader Load collection config from:
[/collections/backuprestore]
[junit4] 2> 581166 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1]
o.a.s.c.c.ZkStateReader path=[/collections/backuprestore] [configName]=[conf1]
specified config exists in ZooKeeper
[junit4] 2> 581167 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2/backuprestore_shard2_replica1'
[junit4] 2> 581167 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1]
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 581167 INFO (qtp1029042061-3977) [n:127.0.0.1:54829_solr
c:backuprestore s:shard2 x:backuprestore_shard2_replica1]
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
system property or JNDI)
[junit4] 2> 581167 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1]
o.a.s.c.c.ZkStateReader path=[/collections/backuprestore] [configName]=[conf1]
specified config exists in ZooKeeper
[junit4] 2> 581167 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudBackupRestore_36E56268232D3CA4-001/tempDir-001/node2/backuprestore_shard1_replica1'
[junit4] 2> 581168 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1]
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 581168 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shard1 x:backuprestore_shard1_replica1]
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find
system property or JNDI)
[junit4] 2> 581170 INFO (qtp1029042061-3980) [n:127.0.0.1:54829_solr
c:backuprestore s:shar
[...truncated too long message...]
:1127)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 11) Thread[id=6437,
name=OverseerStateUpdate-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=TIMED_WAITING, group=Overseer state updater.]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102)
[junit4] > at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
[junit4] > at
org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:309)
[junit4] > at
org.apache.solr.cloud.DistributedQueue.firstChild(DistributedQueue.java:288)
[junit4] > at
org.apache.solr.cloud.DistributedQueue.firstElement(DistributedQueue.java:357)
[junit4] > at
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164)
[junit4] > at
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:142)
[junit4] > at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at
__randomizedtesting.SeedInfo.seed([36E56268232D3CA4]:0)Throwable #2:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
[junit4] > 1) Thread[id=6446, name=zkCallback-1213-thread-6,
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=6434,
name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[36E56268232D3CA4]-SendThread(127.0.0.1:53048),
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 3) Thread[id=6444, name=zkCallback-1213-thread-4,
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=6447, name=zkCallback-1213-thread-7,
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=6440, name=Thread-2184, state=WAITING,
group=TGRP-LeaderElectionIntegrationTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Object.wait(Object.java:502)
[junit4] > at
org.apache.solr.core.CloserThread.run(CoreContainer.java:1253)
[junit4] > 6) Thread[id=6445, name=zkCallback-1213-thread-5,
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 7) Thread[id=6439,
name=OverseerHdfsCoreFailoverThread-96010539846664195-127.0.0.1:7000_solr-n_0000000000,
state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] > at java.lang.Throwable.fillInStackTrace(Native Method)
[junit4] > at
java.lang.Throwable.fillInStackTrace(Throwable.java:783)
[junit4] > at java.lang.Throwable.<init>(Throwable.java:265)
[junit4] > at java.lang.Exception.<init>(Exception.java:66)
[junit4] > at
java.lang.InterruptedException.<init>(InterruptedException.java:67)
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 8) Thread[id=6443, name=zkCallback-1213-thread-3,
state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at
__randomizedtesting.SeedInfo.seed([36E56268232D3CA4]:0)
[junit4] Completed [384/613 (2!)] on J1 in 359.67s, 2 tests, 1 failure, 2
errors <<< FAILURES!
[...truncated 776 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:740: The
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:684: The
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:59: The
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build.xml:233:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/common-build.xml:531:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/lucene/common-build.xml:1427:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/lucene/common-build.xml:984:
There were test failures: 613 suites (10 ignored), 2567 tests, 2 suite-level
errors, 1 error, 1 failure, 724 ignored (71 assumptions) [seed:
36E56268232D3CA4]
Total time: 50 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
