Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1497/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED:
org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([E851EAF348116DFC:3E8ECBA97E0C7E64]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at
org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry(TestLeaderElectionZkExpiry.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11693 lines...]
[junit4] Suite: org.apache.solr.cloud.TestLeaderElectionZkExpiry
[junit4] 2> 375845 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ]
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_E851EAF348116DFC-001/init-core-data-001
[junit4] 2> 375846 WARN
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ]
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=353 numCloses=353
[junit4] 2> 375846 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ]
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 375847 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker) [ ]
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via:
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 375855 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderElectionWithZkExpiry
[junit4] 2> 375869 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.c.SolrXmlConfig MBean server found:
com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d, but no JMX reporters were
configured - adding default JMX reporter.
[junit4] 2> 375922 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry
'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d
[junit4] 2> 375934 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry
'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d
[junit4] 2> 375935 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@730c929d
[junit4] 2> 375936 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_E851EAF348116DFC-001/testLeaderElectionWithZkExpiry-solr-001
[junit4] 2> 375937 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 375937 INFO (Thread-1101) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 375947 INFO (Thread-1101) [ ] o.a.s.c.ZkTestServer
Starting server
[junit4] 2> 375949 ERROR (Thread-1101) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 376038 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[ ] o.a.s.c.ZkTestServer start zk server on port:58040
[junit4] 2> 376261 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 376262 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to
be the leader dummy.host.com:8984_solr
[junit4] 2> 376267 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer
(id=98916024352047106-dummy.host.com:8984_solr-n_0000000000) starting
[junit4] 2> 376288 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
[junit4] 2> 376291 INFO
(zkCallback-393-thread-1-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
from ZooKeeper... (0) -> (1)
[junit4] 2> 376395 WARN
(zkCallback-393-thread-2-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Disconnected type:None path:null path: null type: None
[junit4] 2> 376397 WARN
(zkCallback-393-thread-2-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has
disconnected
[junit4] 2> 377867 WARN
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC]-SendThread(127.0.0.1:58040))
[n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to
ZooKeeper service, session 0x15f6b999e190002 has expired
[junit4] 2> 377868 WARN
(OverseerStateUpdate-98916024352047106-dummy.host.com:8984_solr-n_0000000000)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Solr cannot talk to ZK,
exiting Overseer main queue loop
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer/queue
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:320)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:320)
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:410)
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueue.firstChild(ZkDistributedQueue.java:387)
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:441)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 377868 INFO
(OverseerStateUpdate-98916024352047106-dummy.host.com:8984_solr-n_0000000000)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer Loop exiting :
dummy.host.com:8984_solr
[junit4] 2> 377868 WARN
(OverseerCollectionConfigSetProcessor-98916024352047106-dummy.host.com:8984_solr-n_0000000000)
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor Overseer cannot
talk to ZK
[junit4] 2> 377868 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Expired type:None path:null path: null type: None
[junit4] 2> 377868 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous
ZooKeeper session was expired. Attempting to reconnect to recover relationship
with ZooKeeper...
[junit4] 2> 377868 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer
(id=98916024352047106-dummy.host.com:8984_solr-n_0000000000) closing
[junit4] 2> 377875 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr
] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:332)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:301)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 377894 WARN
(zkCallback-393-thread-1-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for
/live_nodes]
[junit4] 2> 377895 ERROR
(OverseerAutoScalingTriggerThread-98916024352047106-dummy.host.com:8984_solr-n_0000000000)
[n:dummy.host.com:8984_solr ] o.a.s.c.a.ScheduledTriggers Failed to add
trigger .auto_add_replicas
[junit4] 2> org.apache.solr.common.SolrException:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /autoscaling
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:124)
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:114)
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:110)
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.<init>(ScheduledTriggers.java:419)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:161)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:218)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /autoscaling
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:296)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:296)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:475)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:97)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:78)
[junit4] 2> at
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:122)
[junit4] 2> ... 8 more
[junit4] 2> 377895 WARN
(OverseerAutoScalingTriggerThread-98916024352047106-dummy.host.com:8984_solr-n_0000000000)
[n:dummy.host.com:8984_solr ] o.a.s.c.a.OverseerTriggerThread Error
removing old nodeAdded markers
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /autoscaling/nodeAdded
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:320)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:320)
[junit4] 2> at
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:66)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 377895 WARN
(OverseerAutoScalingTriggerThread-98916024352047106-dummy.host.com:8984_solr-n_0000000000)
[n:dummy.host.com:8984_solr ] o.a.s.c.a.OverseerTriggerThread
OverseerTriggerThread has been closed, exiting.
[junit4] 2> 377896 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection
expired - starting a new one...
[junit4] 2> 377897 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with
ZooKeeper reestablished.
[junit4] 2> 377897 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session
re-connected ... refreshing core states after session expiration.
[junit4] 2> 377899 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
from ZooKeeper... (1) -> (0)
[junit4] 2> 377904 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to
be the leader dummy.host.com:8984_solr
[junit4] 2> 377905 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer
(id=98916024352047107-dummy.host.com:8984_solr-n_0000000001) starting
[junit4] 2> 378007 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Disconnected type:None path:null path: null type: None
[junit4] 2> 378007 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has
disconnected
[junit4] 2> 379307 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040))
[n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to
ZooKeeper service, session 0x15f6b999e190003 has expired
[junit4] 2> 379308 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Expired type:None path:null path: null type: None
[junit4] 2> 379308 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous
ZooKeeper session was expired. Attempting to reconnect to recover relationship
with ZooKeeper...
[junit4] 2> 379511 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Could not create Overseer node
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:357)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:357)
[junit4] 2> at
org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:771)
[junit4] 2> at
org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:644)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.<init>(Overseer.java:107)
[junit4] 2> at
org.apache.solr.cloud.Overseer.start(Overseer.java:518)
[junit4] 2> at
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:799)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 379512 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer
(id=98916024352047107-dummy.host.com:8984_solr-n_0000000001) closing
[junit4] 2> 379512 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection
expired - starting a new one...
[junit4] 2> 379512 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController
:java.lang.RuntimeException:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /overseer
[junit4] 2> at
org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:780)
[junit4] 2> at
org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:644)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.<init>(Overseer.java:107)
[junit4] 2> at
org.apache.solr.cloud.Overseer.start(Overseer.java:518)
[junit4] 2> at
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:799)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:353)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /overseer
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:357)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:357)
[junit4] 2> at
org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:771)
[junit4] 2> ... 19 more
[junit4] 2>
[junit4] 2> 379512 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.RuntimeException:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /overseer
[junit4] 2> at
org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:780)
[junit4] 2> at
org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:644)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.<init>(Overseer.java:107)
[junit4] 2> at
org.apache.solr.cloud.Overseer.start(Overseer.java:518)
[junit4] 2> at
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:799)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:353)
[junit4] 2> ... 10 more
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /overseer
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:357)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:357)
[junit4] 2> at
org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:771)
[junit4] 2> ... 19 more
[junit4] 2>
[junit4] 2> 379512 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed
[junit4] 2> 379512 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:false
[junit4] 2> 379514 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with
ZooKeeper reestablished.
[junit4] 2> 379514 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session
re-connected ... refreshing core states after session expiration.
[junit4] 2> 379618 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Disconnected type:None path:null path: null type: None
[junit4] 2> 379618 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has
disconnected
[junit4] 2> 381302 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040))
[n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to
ZooKeeper service, session 0x15f6b999e190004 has expired
[junit4] 2> 381302 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Expired type:None path:null path: null type: None
[junit4] 2> 381302 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous
ZooKeeper session was expired. Attempting to reconnect to recover relationship
with ZooKeeper...
[junit4] 2> 381303 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection
expired - starting a new one...
[junit4] 2> 381304 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with
ZooKeeper reestablished.
[junit4] 2> 381304 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session
re-connected ... refreshing core states after session expiration.
[junit4] 2> 381403 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /aliases.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:296)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:296)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:443)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 381403 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /aliases.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:296)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:296)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:443)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 381403 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed
[junit4] 2> 381404 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 381413 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Disconnected type:None path:null path: null type: None
[junit4] 2> 381413 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has
disconnected
[junit4] 2> 382709 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040))
[n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to
ZooKeeper service, session 0x15f6b999e190005 has expired
[junit4] 2> 382709 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Expired type:None path:null path: null type: None
[junit4] 2> 382709 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous
ZooKeeper session was expired. Attempting to reconnect to recover relationship
with ZooKeeper...
[junit4] 2> 382709 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection
expired - starting a new one...
[junit4] 2> 382710 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with
ZooKeeper reestablished.
[junit4] 2> 382710 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session
re-connected ... refreshing core states after session expiration.
[junit4] 2> 382719 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxnFactory Ignoring unexpected runtime exception
[junit4] 2> java.nio.channels.CancelledKeyException
[junit4] 2> at
sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at
sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:182)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 382819 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Disconnected type:None path:null path: null type: None
[junit4] 2> 382819 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has
disconnected
[junit4] 2> 384753 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040))
[n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to
ZooKeeper service, session 0x15f6b999e190006 has expired
[junit4] 2> 384753 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Expired type:None path:null path: null type: None
[junit4] 2> 384753 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous
ZooKeeper session was expired. Attempting to reconnect to recover relationship
with ZooKeeper...
[junit4] 2> 384753 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection
expired - starting a new one...
[junit4] 2> 384755 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with
ZooKeeper reestablished.
[junit4] 2> 384755 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session
re-connected ... refreshing core states after session expiration.
[junit4] 2> 384854 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /clusterstate.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 384854 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /clusterstate.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 384854 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed
[junit4] 2> 384854 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 384857 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Disconnected type:None path:null path: null type: None
[junit4] 2> 384857 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager zkClient has
disconnected
[junit4] 2> 386735 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-SendThread(127.0.0.1:58040))
[n:dummy.host.com:8984_solr ] o.a.z.ClientCnxn Unable to reconnect to
ZooKeeper service, session 0x15f6b999e190007 has expired
[junit4] 2> 386735 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2d684005 name:
ZooKeeperConnection Watcher:127.0.0.1:58040/solr got event WatchedEvent
state:Expired type:None path:null path: null type: None
[junit4] 2> 386735 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Our previous
ZooKeeper session was expired. Attempting to reconnect to recover relationship
with ZooKeeper...
[junit4] 2> 386735 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Connection
expired - starting a new one...
[junit4] 2> 386740 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connection with
ZooKeeper reestablished.
[junit4] 2> 386741 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController ZooKeeper session
re-connected ... refreshing core states after session expiration.
[junit4] 2> 386839 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /clusterstate.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 386839 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /clusterstate.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:428)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:328)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 386839 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed
[junit4] 2> 386840 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 386848 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to
be the leader dummy.host.com:8984_solr
[junit4] 2> 386850 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer
(id=98916024352047113-dummy.host.com:8984_solr-n_0000000002) starting
[junit4] 2> 386884 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer
(id=98916024352047113-dummy.host.com:8984_solr-n_0000000002) closing
[junit4] 2> 386904 ERROR
(OverseerCollectionConfigSetProcessor-98916024352047113-dummy.host.com:8984_solr-n_0000000002)
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor Unable to
prioritize overseer
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1097)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:60)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386906 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
[junit4] 2> 386909 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy
Reconnected to ZooKeeper
[junit4] 2> 386909 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 386909 INFO
(zkCallback-393-thread-3-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
from ZooKeeper... (0) -> (1)
[junit4] 2> 386910 INFO
(OverseerStateUpdate-98916024352047113-dummy.host.com:8984_solr-n_0000000002)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer Loop exiting :
dummy.host.com:8984_solr
[junit4] 2> 386911 WARN (OverseerExitThread) [n:dummy.host.com:8984_solr
] o.a.s.c.Overseer I'm exiting, but I'm still the leader
[junit4] 2> 386913 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerElectionContext I am going to
be the leader dummy.host.com:8984_solr
[junit4] 2> 386914 INFO (OverseerExitThread) [n:dummy.host.com:8984_solr
] o.a.s.c.OverseerElectionContext I am going to be the leader
dummy.host.com:8984_solr
[junit4] 2> 386915 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer
(id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) starting
[junit4] 2> 386916 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr
] o.a.s.c.LeaderElector node exists
[junit4] 2> org.apache.zookeeper.KeeperException$NodeExistsException:
KeeperErrorCode = NodeExists for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$makePath$8(SolrZkClient.java:488)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:487)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:430)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:417)
[junit4] 2> at
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:788)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
[junit4] 2> at
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1967)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:326)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386923 INFO (OverseerExitThread) [n:dummy.host.com:8984_solr
] o.a.s.c.Overseer Overseer
(id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) closing
[junit4] 2> 386923 INFO
(OverseerCollectionConfigSetProcessor-98916024352047113-dummy.host.com:8984_solr-n_0000000003)
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor According to ZK
I (id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) am no longer a
leader.
[junit4] 2> 386923 INFO
(OverseerStateUpdate-98916024352047113-dummy.host.com:8984_solr-n_0000000003)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer According to ZK I
(id=98916024352047113-dummy.host.com:8984_solr-n_0000000003) am no longer a
leader.
[junit4] 2> 386923 INFO
(OverseerStateUpdate-98916024352047113-dummy.host.com:8984_solr-n_0000000003)
[n:dummy.host.com:8984_solr ] o.a.s.c.Overseer Overseer Loop exiting :
dummy.host.com:8984_solr
[junit4] 2> 386923 ERROR
(OverseerCollectionConfigSetProcessor-98916024352047113-dummy.host.com:8984_solr-n_0000000003)
[n:dummy.host.com:8984_solr ] o.a.s.c.OverseerTaskProcessor Unable to
prioritize overseer
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1097)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
[junit4] 2> at
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:60)
[junit4] 2> at
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386923 WARN
(OverseerAutoScalingTriggerThread-98916024352047113-dummy.host.com:8984_solr-n_0000000003)
[n:dummy.host.com:8984_solr ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:332)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
[junit4] 2> at
org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:153)
[junit4] 2> at
org.apache.solr.client.solrj.cloud.autoscaling.DistribStateManager.getAutoScalingConfig(DistribStateManager.java:62)
[junit4] 2> at
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:117)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 386938 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
[junit4] 2> 386942 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkController
:org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1006)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:910)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$multi$9(SolrZkClient.java:550)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:550)
[junit4] 2> at
org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:976)
[junit4] 2> at
org.apache.solr.cloud.ZkController.access$500(ZkController.java:138)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:361)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2>
[junit4] 2> 386942 ERROR
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:413)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
[junit4] 2> at
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
[junit4] 2> at
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1006)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:910)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$multi$9(SolrZkClient.java:550)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.multi(SolrZkClient.java:550)
[junit4] 2> at
org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:976)
[junit4] 2> at
org.apache.solr.cloud.ZkController.access$500(ZkController.java:138)
[junit4] 2> at
org.apache.solr.cloud.ZkController$1.command(ZkController.java:361)
[junit4] 2> ... 5 more
[junit4] 2>
[junit4] 2> 386942 WARN
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.DefaultConnectionStrategy Reconnect
to ZooKeeper failed
[junit4] 2> 386943 INFO
(zkCallback-393-thread-7-processing-n:dummy.host.com:8984_solr-EventThread)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 386946 WARN (OverseerExitThread) [n:dummy.host.com:8984_solr
] o.a.s.c.Overseer Unable to rejoinElection
[junit4] 2> org.apache.solr.common.SolrException: Unable to rejoin election
[junit4] 2> at
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1970)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:326)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for
/overseer_elect/election/98916024352047113-dummy.host.com:8984_solr-n_
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:361)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:258)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:138)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
[junit4] 2> at
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:1967)
[junit4] 2> ... 2 more
[junit4] 2> 386946 ERROR (OverseerExitThread) [n:dummy.host.com:8984_solr
] o.a.s.c.Overseer could not read the data
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:332)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:301)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 388444 WARN
(zkCallback-393-thread-3-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for
/live_nodes]
[junit4] 2> 388444 WARN
(zkCallback-393-thread-1-processing-n:dummy.host.com:8984_solr)
[n:dummy.host.com:8984_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch
triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for
/live_nodes]
[junit4] 2> 446294 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.c.CoreContainer Shutting down
CoreContainer instance=1880262316
[junit4] 2> 446294 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.node, tag=null
[junit4] 2> 446294 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@9ca6743: rootName = null,
domain = solr.node, service url = null, agent id = null] for registry solr.node
/ com.codahale.metrics.MetricRegistry@acda2a9
[junit4] 2> 446297 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.jvm, tag=null
[junit4] 2> 446297 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@c57ce4e: rootName = null,
domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm /
com.codahale.metrics.MetricRegistry@30fb61d4
[junit4] 2> 446303 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.jetty, tag=null
[junit4] 2> 446303 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@4e19af0e: rootName = null,
domain = solr.jetty, service url = null, agent id = null] for registry
solr.jetty / com.codahale.metrics.MetricRegistry@39dbbf3
[junit4] 2> 446303 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.c.TransientSolrCoreCacheDefault
Allocating transient cache for 2147483647 transient cores
[junit4] 2> 446304 ERROR
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is
not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN
server state changes
[junit4] 2> 446305 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.c.ZkTestServer connecting to
127.0.0.1:58040 58040
[junit4] 2> 448829 INFO (Thread-1101) [ ] o.a.s.c.ZkTestServer
connecting to 127.0.0.1:58040 58040
[junit4] 2> 448830 WARN (Thread-1101) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 4 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> 448830 INFO
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[E851EAF348116DFC])
[n:dummy.host.com:8984_solr ] o.a.s.SolrTestCaseJ4 ###Ending
testLeaderElectionWithZkExpiry
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestLeaderElectionZkExpiry
-Dtests.method=testLeaderElectionWithZkExpiry -Dtests.seed=E851EAF348116DFC
-Dtests.slow=true -Dtests.locale=vi-VN -Dtests.timezone=Canada/Pacific
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 73.0s J0 |
TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at
__randomizedtesting.SeedInfo.seed([E851EAF348116DFC:3E8ECBA97E0C7E64]:0)
[junit4] > at
org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry(TestLeaderElectionZkExpiry.java:97)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 448832 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker)
[n:dummy.host.com:8984_solr ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 448832 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker)
[n:dummy.host.com:8984_solr ] o.a.s.c.CoreContainer Shutting down
CoreContainer instance=1880262316
[junit4] 2> 448832 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker)
[n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.node, tag=null
[junit4] 2> 448832 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker)
[n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.jvm, tag=null
[junit4] 2> 448832 INFO
(SUITE-TestLeaderElectionZkExpiry-seed#[E851EAF348116DFC]-worker)
[n:dummy.host.com:8984_solr ] o.a.s.m.SolrMetricManager Closing metric
reporters for registry=solr.jetty, tag=null
[junit4] 2> NOTE: leaving temporary files on disk at:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestLeaderElectionZkExpiry_E851EAF348116DFC-001
[junit4] 2> NOTE: test params are:
codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION,
chunkSize=6055, maxDocsPerChunk=9, blockSize=877),
termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION,
chunkSize=6055, blockSize=877)), sim=RandomSimilarity(queryNorm=false): {},
locale=vi-VN, timezone=Canada/Pacific
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152
(64-bit)/cpus=3,threads=1,free=111401648,total=292343808
[junit4] 2> NOTE: All tests run in this JVM:
[TestLocalFSCloudBackupRestore, AnalyticsQueryTest,
AutoAddReplicasPlanActionTest, AddReplicaTest, AutoscalingHistoryHandlerTest,
TestEmbeddedSolrServerSchemaAPI, TestJsonFacets, TestSmileRequest,
TestDistributedSearch, V2ApiIntegrationTest, SliceStateTest, DeleteNodeTest,
MoveReplicaHDFSFailoverTest, TestTolerantUpdateProcessorRandomCloud,
TestLeaderElectionZkExpiry]
[junit4] Completed [67/746 (1!)] on J0 in 73.00s, 1 test, 1 failure <<<
FAILURES!
[...truncated 49504 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
