Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3509/
Java: 64bit/jdk-11 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry

Error Message:


Stack Trace:
java.lang.AssertionError
        at 
__randomizedtesting.SeedInfo.seed([F49C4FF42EEF100B:22436EAE18F20393]:0)
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at 
org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry(TestLeaderElectionZkExpiry.java:95)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)




Build Log:
[...truncated 14326 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestLeaderElectionZkExpiry
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderElectionZkExpiry_F49C4FF42EEF100B-001/init-core-data-001
   [junit4]   2> 1313771 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1313771 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, 
clientAuth=0.0/0.0)
   [junit4]   2> 1313771 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1313773 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderElectionWithZkExpiry
   [junit4]   2> 1313776 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@4d0414f2, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1313854 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: 
WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1313867 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 
2147483647 transient cores
   [junit4]   2> 1313867 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics 
history in memory.
   [junit4]   2> 1313877 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 
'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d0414f2
   [junit4]   2> 1313882 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 
'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d0414f2
   [junit4]   2> 1313882 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d0414f2
   [junit4]   2> 1313883 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderElectionZkExpiry_F49C4FF42EEF100B-001/testLeaderElectionWithZkExpiry-solr-001
   [junit4]   2> 1313884 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1313884 INFO  (ZkTestServer Run Thread) [    ] 
o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1313885 INFO  (ZkTestServer Run Thread) [    ] 
o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1313985 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkTestServer start zk server on port:43509
   [junit4]   2> 1313985 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43509
   [junit4]   2> 1313985 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 43509
   [junit4]   2> 1313987 INFO  (zkConnectionManagerCallback-8818-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1313989 INFO  (zkConnectionManagerCallback-8820-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1313990 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1314017 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1314017 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.Overseer Overseer 
(id=72060270075838466-dummy.host.com:8984_solr-n_0000000000) starting
   [junit4]   2> 1314019 INFO  (zkConnectionManagerCallback-8829-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1314021 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 
127.0.0.1:43509/solr ready
   [junit4]   2> 1314021 INFO  
(OverseerStateUpdate-72060270075838466-dummy.host.com:8984_solr-n_0000000000) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Starting to work on the main 
queue : dummy.host.com:8984_solr
   [junit4]   2> 1314021 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1314022 INFO  
(OverseerStateUpdate-72060270075838466-dummy.host.com:8984_solr-n_0000000000) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 1314022 INFO  (zkCallback-8828-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1314023 INFO  (zkCallback-8828-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1314123 WARN  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1314123 WARN  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1315946 WARN  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B]-SendThread(localhost.localdomain:43509))
 [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 
0x100026f104a0002 has expired
   [junit4]   2> 1315946 WARN  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1315946 WARN  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. 
Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1315946 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.Overseer Overseer 
(id=72060270075838466-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 1315946 WARN  
(OverseerStateUpdate-72060270075838466-dummy.host.com:8984_solr-n_0000000000) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Solr cannot talk to ZK, 
exiting Overseer main queue loop
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer/queue
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1541) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:346)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:346) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:415)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.firstChild(ZkDistributedQueue.java:392)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:446)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:212) 
[java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1315947 INFO  
(OverseerStateUpdate-72060270075838466-dummy.host.com:8984_solr-n_0000000000) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1315948 WARN  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Error communicating with ZooKeeper
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:319)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1316047 WARN  (zkCallback-8821-thread-1) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1316047 WARN  (ScheduledTrigger-7314-thread-1) [    ] 
o.a.s.c.a.TriggerEventQueue Exception peeking queue of trigger 
.scheduled_maintenance
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /autoscaling/events/.scheduled_maintenance
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1541) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:346)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:346) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:415)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.firstChild(ZkDistributedQueue.java:392)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.firstElement(ZkDistributedQueue.java:509)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQueue.java:150) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.peekEvent(TriggerEventQueue.java:69)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:611)
 [java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
 [?:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
[?:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
[?:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1316047 ERROR 
(OverseerAutoScalingTriggerThread-72060270075838466-dummy.host.com:8984_solr-n_0000000000)
 [    ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .auto_add_replicas
   [junit4]   2> org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:119) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:115) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:111) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.<init>(ScheduledTriggers.java:566)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:220)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:108) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:127) 
~[java/:?]
   [junit4]   2>        ... 9 more
   [junit4]   2> 1316048 WARN  
(OverseerAutoScalingTriggerThread-72060270075838466-dummy.host.com:8984_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Error removing old markers
   [junit4]   2> org.apache.solr.common.AlreadyClosedException: null
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:262)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1316048 WARN  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1316049 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1316049 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1316049 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core 
states after session expiration.
   [junit4]   2> 1316050 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1316051 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.Overseer Overseer 
(id=72060270075838466-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 1316052 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.Overseer Overseer 
(id=72060270075838466-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 1316052 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.Overseer Overseer 
(id=72060270075838466-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 1316052 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.Overseer Overseer 
(id=72060270075838466-dummy.host.com:8984_solr-n_0000000000) closing
   [junit4]   2> 1316052 INFO  (zkConnectionManagerCallback-8822-thread-1) [    
] o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1316052 INFO  (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=72060270075838468-dummy.host.com:8984_solr-n_0000000001) starting
   [junit4]   2> 1316054 INFO  
(OverseerStateUpdate-72060270075838468-dummy.host.com:8984_solr-n_0000000001) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Starting to work on the main 
queue : dummy.host.com:8984_solr
   [junit4]   2> 1316054 INFO  (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Publish 
node=dummy.host.com:8984_solr as DOWN
   [junit4]   2> 1316055 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn CancelledKeyException causing close of session 
0x100026f104a0004
   [junit4]   2> 1316155 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1316155 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1317409 WARN  
(zkConnectionManagerCallback-8822-thread-1-SendThread(localhost.localdomain:43509))
 [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 
0x100026f104a0004 has expired
   [junit4]   2> 1317409 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1317409 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. 
Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1317409 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838468-dummy.host.com:8984_solr-n_0000000001) closing
   [junit4]   2> 1317409 INFO  
(OverseerStateUpdate-72060270075838468-dummy.host.com:8984_solr-n_0000000001) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1317420 WARN  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Error communicating with ZooKeeper
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:319)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1317655 WARN  (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Could not publish node as 
down: KeeperErrorCode = Session expired for /overseer/queue/qn-
   [junit4]   2> 1317655 INFO  (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1317655 ERROR 
(OverseerAutoScalingTriggerThread-72060270075838468-dummy.host.com:8984_solr-n_0000000001)
 [    ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .scheduled_maintenance
   [junit4]   2> org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.scheduled_maintenance
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:119) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:115) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:111) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.<init>(ScheduledTriggers.java:566)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:220)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.scheduled_maintenance
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:108) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:127) 
~[java/:?]
   [junit4]   2>        ... 9 more
   [junit4]   2> 1317655 ERROR (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController 
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
= Session expired for /autoscaling.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:215)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:201)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:1082)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController.access$500(ZkController.java:149)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:376)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:177)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:58)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:151)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:285)
   [junit4]   2>        at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   [junit4]   2>        at 
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
   [junit4]   2>        at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   [junit4]   2>        at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   [junit4]   2>        at java.base/java.lang.Thread.run(Thread.java:834)
   [junit4]   2> 
   [junit4]   2> 1317655 ERROR 
(OverseerAutoScalingTriggerThread-72060270075838468-dummy.host.com:8984_solr-n_0000000001)
 [    ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .auto_add_replicas
   [junit4]   2> org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:119) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:115) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:111) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.<init>(ScheduledTriggers.java:566)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:220)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:108) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:127) 
~[java/:?]
   [junit4]   2>        ... 9 more
   [junit4]   2> 1317656 ERROR (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:428)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:177)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:58)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:151)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:285)
   [junit4]   2>        at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   [junit4]   2>        at 
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
   [junit4]   2>        at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   [junit4]   2>        at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   [junit4]   2>        at java.base/java.lang.Thread.run(Thread.java:834)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:215)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getAutoScalingConfig(ZkStateReader.java:201)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController.createEphemeralLiveNode(ZkController.java:1082)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController.access$500(ZkController.java:149)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:376)
   [junit4]   2>        ... 10 more
   [junit4]   2> 
   [junit4]   2> 1317656 WARN  (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect 
to ZooKeeper failed
   [junit4]   2> 1317656 INFO  (zkConnectionManagerCallback-8822-thread-1) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient 
Connected:false
   [junit4]   2> 1317656 WARN  
(OverseerAutoScalingTriggerThread-72060270075838468-dummy.host.com:8984_solr-n_0000000001)
 [    ] o.a.s.c.a.OverseerTriggerThread Error removing old markers
   [junit4]   2> org.apache.solr.common.AlreadyClosedException: null
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:261)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1317656 WARN  
(OverseerAutoScalingTriggerThread-72060270075838468-dummy.host.com:8984_solr-n_0000000001)
 [    ] o.a.s.c.a.OverseerTriggerThread Error removing old markers
   [junit4]   2> org.apache.solr.common.AlreadyClosedException: null
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:262)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1317656 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1317657 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1317657 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1317657 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states 
after session expiration.
   [junit4]   2> 1317659 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838468-dummy.host.com:8984_solr-n_0000000001) closing
   [junit4]   2> 1317659 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838468-dummy.host.com:8984_solr-n_0000000001) closing
   [junit4]   2> 1317659 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838468-dummy.host.com:8984_solr-n_0000000001) closing
   [junit4]   2> 1317659 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838468-dummy.host.com:8984_solr-n_0000000001) closing
   [junit4]   2> 1317659 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1317660 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=72060270075838469-dummy.host.com:8984_solr-n_0000000002) starting
   [junit4]   2> 1317661 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Publish 
node=dummy.host.com:8984_solr as DOWN
   [junit4]   2> 1317661 INFO  
(OverseerStateUpdate-72060270075838469-dummy.host.com:8984_solr-n_0000000002) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Starting to work on the main 
queue : dummy.host.com:8984_solr
   [junit4]   2> 1317661 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1317662 INFO  (zkCallback-8821-thread-5) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1317662 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy 
Reconnected to ZooKeeper
   [junit4]   2> 1317662 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient 
Connected:true
   [junit4]   2> 1317763 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1317763 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1319246 WARN  
(zkConnectionManagerCallback-8822-thread-1-SendThread(localhost.localdomain:43509))
 [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 
0x100026f104a0005 has expired
   [junit4]   2> 1319246 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1319246 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. 
Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1319246 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838469-dummy.host.com:8984_solr-n_0000000002) closing
   [junit4]   2> 1319247 INFO  
(OverseerStateUpdate-72060270075838469-dummy.host.com:8984_solr-n_0000000002) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1319247 ERROR 
(OverseerCollectionConfigSetProcessor-72060270075838469-dummy.host.com:8984_solr-n_0000000002)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerTaskProcessor Unable to 
prioritize overseer 
   [junit4]   2> java.lang.InterruptedException: sleep interrupted
   [junit4]   2>        at java.lang.Thread.sleep(Native Method) ~[?:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:126) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:81)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:68)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:187) 
[java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1319247 WARN  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Error communicating with ZooKeeper
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:319)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1319263 WARN  (zkCallback-8821-thread-6) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1319263 ERROR 
(OverseerAutoScalingTriggerThread-72060270075838469-dummy.host.com:8984_solr-n_0000000002)
 [    ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .scheduled_maintenance
   [junit4]   2> org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.scheduled_maintenance
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:119) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:115) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:111) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.<init>(ScheduledTriggers.java:566)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:220)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.scheduled_maintenance
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:108) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:127) 
~[java/:?]
   [junit4]   2>        ... 9 more
   [junit4]   2> 1319264 ERROR 
(OverseerAutoScalingTriggerThread-72060270075838469-dummy.host.com:8984_solr-n_0000000002)
 [    ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .auto_add_replicas
   [junit4]   2> org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:119) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:115) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:111) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.<init>(ScheduledTriggers.java:566)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:220)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:108) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:127) 
~[java/:?]
   [junit4]   2>        ... 9 more
   [junit4]   2> 1319264 WARN  
(OverseerAutoScalingTriggerThread-72060270075838469-dummy.host.com:8984_solr-n_0000000002)
 [    ] o.a.s.c.a.OverseerTriggerThread Error removing old markers
   [junit4]   2> org.apache.solr.common.AlreadyClosedException: null
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:261)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1319264 WARN  
(OverseerAutoScalingTriggerThread-72060270075838469-dummy.host.com:8984_solr-n_0000000002)
 [    ] o.a.s.c.a.OverseerTriggerThread Error removing old markers
   [junit4]   2> org.apache.solr.common.AlreadyClosedException: null
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:262)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1319265 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1319265 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1319265 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1319265 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states 
after session expiration.
   [junit4]   2> 1319266 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1319267 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838469-dummy.host.com:8984_solr-n_0000000002) closing
   [junit4]   2> 1319267 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838469-dummy.host.com:8984_solr-n_0000000002) closing
   [junit4]   2> 1319267 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838469-dummy.host.com:8984_solr-n_0000000002) closing
   [junit4]   2> 1319267 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838469-dummy.host.com:8984_solr-n_0000000002) closing
   [junit4]   2> 1319267 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1319267 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) starting
   [junit4]   2> 1319269 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Publish 
node=dummy.host.com:8984_solr as DOWN
   [junit4]   2> 1319269 INFO  
(OverseerStateUpdate-72060270075838470-dummy.host.com:8984_solr-n_0000000003) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Starting to work on the main 
queue : dummy.host.com:8984_solr
   [junit4]   2> 1319269 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1319269 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy 
Reconnected to ZooKeeper
   [junit4]   2> 1319269 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient 
Connected:true
   [junit4]   2> 1319270 INFO  (zkCallback-8828-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1319275 INFO  (zkCallback-8821-thread-6) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1319275 INFO  (zkCallback-8828-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1319375 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1319376 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1320563 WARN  
(zkConnectionManagerCallback-8822-thread-1-SendThread(localhost.localdomain:43509))
 [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 
0x100026f104a0006 has expired
   [junit4]   2> 1320563 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1320563 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. 
Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1320563 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1320563 WARN  
(OverseerCollectionConfigSetProcessor-72060270075838470-dummy.host.com:8984_solr-n_0000000003)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerTaskProcessor Overseer cannot 
talk to ZK
   [junit4]   2> 1320563 INFO  
(OverseerStateUpdate-72060270075838470-dummy.host.com:8984_solr-n_0000000003) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1320564 WARN  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Error communicating with ZooKeeper
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:319)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1320875 WARN  (zkCallback-8821-thread-6) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1320875 WARN  (ScheduledTrigger-7326-thread-1) [    ] 
o.a.s.c.a.TriggerEventQueue Exception peeking queue of trigger 
.scheduled_maintenance
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /autoscaling/events/.scheduled_maintenance
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1541) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:346)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:346) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.fetchZkChildren(ZkDistributedQueue.java:415)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.firstChild(ZkDistributedQueue.java:392)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.firstElement(ZkDistributedQueue.java:509)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQueue.java:150) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.peekEvent(TriggerEventQueue.java:69)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:611)
 [java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
 [?:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
[?:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
[?:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1320875 ERROR 
(OverseerAutoScalingTriggerThread-72060270075838470-dummy.host.com:8984_solr-n_0000000003)
 [    ] o.a.s.c.a.ScheduledTriggers Failed to add trigger .auto_add_replicas
   [junit4]   2> org.apache.solr.common.SolrException: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:119) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:115) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:111) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueueFactory.makeQueue(ZkDistributedQueueFactory.java:36)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.TriggerEventQueue.<init>(TriggerEventQueue.java:49)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.<init>(ScheduledTriggers.java:566)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.ScheduledTriggers.add(ScheduledTriggers.java:220)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:245)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /autoscaling/events/.auto_add_replicas
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:108) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:93) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:127) 
~[java/:?]
   [junit4]   2>        ... 9 more
   [junit4]   2> 1320877 WARN  
(OverseerAutoScalingTriggerThread-72060270075838470-dummy.host.com:8984_solr-n_0000000003)
 [    ] o.a.s.c.a.OverseerTriggerThread Error removing old markers
   [junit4]   2> org.apache.solr.common.AlreadyClosedException: null
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:261)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1320877 WARN  
(OverseerAutoScalingTriggerThread-72060270075838470-dummy.host.com:8984_solr-n_0000000003)
 [    ] o.a.s.c.a.OverseerTriggerThread Error removing old markers
   [junit4]   2> org.apache.solr.common.AlreadyClosedException: null
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:74)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.ZkDistribStateManager.listData(ZkDistribStateManager.java:80)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.listTree(DistribStateManager.java:104)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.cloud.DistribStateManager.removeRecursively(DistribStateManager.java:123)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.removeMarkers(OverseerTriggerThread.java:269)
 [java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:262)
 [java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1320877 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1320878 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1320878 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1320878 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states 
after session expiration.
   [junit4]   2> 1320878 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1320983 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1320983 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1322315 WARN  
(zkConnectionManagerCallback-8822-thread-1-SendThread(localhost.localdomain:43509))
 [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 
0x100026f104a0007 has expired
   [junit4]   2> 1322315 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1322315 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. 
Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1322315 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1322315 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1322315 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1322315 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1322315 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states 
after session expiration.
   [junit4]   2> 1322424 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1322424 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1323900 WARN  
(zkConnectionManagerCallback-8822-thread-1-SendThread(localhost.localdomain:43509))
 [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 
0x100026f104a0008 has expired
   [junit4]   2> 1323900 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1323900 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. 
Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1323900 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1323900 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1323901 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1323901 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1323901 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states 
after session expiration.
   [junit4]   2> 1324001 ERROR 
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.ZkController 
:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
= Session expired for /security.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:322)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:322)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.addSecurityNodeWatcher(ZkStateReader.java:485)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:468)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:343)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:177)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:58)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:151)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:533)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:508)
   [junit4]   2> 
   [junit4]   2> 1324001 ERROR 
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper 
failed:org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:428)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:177)
   [junit4]   2>        at 
org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:58)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:151)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:533)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:508)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /security.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1111)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:322)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:322)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.addSecurityNodeWatcher(ZkStateReader.java:485)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:468)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkController$1.command(ZkController.java:343)
   [junit4]   2>        ... 5 more
   [junit4]   2> 
   [junit4]   2> 1324001 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 1324001 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 1324002 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1324002 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1324024 INFO  (zkConnectionManagerCallback-8831-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1325098 WARN  
(zkConnectionManagerCallback-8822-thread-1-SendThread(localhost.localdomain:43509))
 [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 
0x100026f104a0009 has expired
   [junit4]   2> 1325098 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@787984be name: 
ZooKeeperConnection Watcher:127.0.0.1:43509/solr got event WatchedEvent 
state:Expired type:None path:null path: null type: None
   [junit4]   2> 1325098 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. 
Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1325098 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1325098 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1325099 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1325099 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1325099 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states 
after session expiration.
   [junit4]   2> 1325504 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1325504 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1325504 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1325504 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838470-dummy.host.com:8984_solr-n_0000000003) closing
   [junit4]   2> 1325504 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1325505 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer 
(id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) starting
   [junit4]   2> 1325506 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325506 INFO  
(OverseerStateUpdate-72060270075838475-dummy.host.com:8984_solr-n_0000000004) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Starting to work on the main 
queue : dummy.host.com:8984_solr
   [junit4]   2> 1325506 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Publish 
node=dummy.host.com:8984_solr as DOWN
   [junit4]   2> 1325506 INFO  
(OverseerStateUpdate-72060270075838475-dummy.host.com:8984_solr-n_0000000004) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
dummy.host.com:8984_solr
   [junit4]   2> 1325508 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/dummy.host.com:8984_solr
   [junit4]   2> 1325508 ERROR 
(OverseerCollectionConfigSetProcessor-72060270075838475-dummy.host.com:8984_solr-n_0000000004)
 [n:dummy.host.com:8984_solr    ] o.a.s.c.OverseerTaskProcessor Unable to 
prioritize overseer 
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2>        at java.lang.Object.wait(Native Method) ~[?:?]
   [junit4]   2>        at java.lang.Object.wait(Object.java:328) ~[?:?]
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1411) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1106) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$exists$3(SolrZkClient.java:334)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:334) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:68)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:187) 
[java/:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:834) [?:?]
   [junit4]   2> 1325508 WARN  (OverseerExitThread) [    ] o.a.s.c.Overseer I 
(id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) am exiting, but 
I'm still the leader
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  (zkCallback-8828-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1325509 INFO  (zkCallback-8821-thread-5) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.DefaultConnectionStrategy 
Reconnected to ZooKeeper
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) 
[n:dummy.host.com:8984_solr    ] o.a.s.c.c.ConnectionManager zkClient 
Connected:true
   [junit4]   2> 1325509 INFO  (OverseerExitThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] o.a.s.c.Overseer 
Overseer (id=72060270075838475-dummy.host.com:8984_solr-n_0000000004) closing
   [junit4]   2> 1325509 WARN  (OverseerExitThread) [    ] 
o.a.s.c.LeaderElector Deleting duplicate registration: 
/overseer_elect/election/72060270075838475-dummy.host.com:8984_solr-n_0000000005
   [junit4]   2> 1325509 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.LeaderElector Deleting duplicate registration: 
/overseer_elect/election/72060270075838475-dummy.host.com:8984_solr-n_0000000006
   [junit4]   2> 1325509 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.LeaderElector Deleting duplicate registration: 
/overseer_elect/election/72060270075838475-dummy.host.com:8984_solr-n_0000000005
   [junit4]   2> 1325509 WARN  (OverseerExitThread) [    ] 
o.a.s.c.LeaderElector Deleting duplicate registration: 
/overseer_elect/election/72060270075838475-dummy.host.com:8984_solr-n_0000000006
   [junit4]   2> 1325509 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.LeaderElector Deleting duplicate registration: 
/overseer_elect/election/72060270075838475-dummy.host.com:8984_solr-n_0000000007
   [junit4]   2> 1325509 WARN  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.LeaderElector Deleting duplicate registration: 
/overseer_elect/election/72060270075838475-dummy.host.com:8984_solr-n_0000000007
   [junit4]   2> 1325510 INFO  (OverseerExitThread) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1325510 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1325510 INFO  
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 
dummy.host.com:8984_solr
   [junit4]   2> 1325510 INFO  (OverseerExitThread) [n:dummy.host.com:8984_solr 
   ] o.a.s.c.Overseer Overseer 
(id=72060270075838475-dummy.host.com:8984_solr-n_0000000007) starting
   [junit4]   2> 1325510 ERROR 
(zkConnectionManagerCallback-8822-thread-1-EventThread) [    ] 
o.a.s.c.LeaderElector node exists
   [junit4]   2> org.apache.zookeeper.KeeperException$NodeExistsException: 
KeeperErrorCode = NodeExists for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:122) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54) 
~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]

[...truncated too long message...]

] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, 
tag=null
   [junit4]   2> 1384033 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@3e2e79f7: rootName = null, 
domain = solr.jetty, service url = null, agent id = null] for registry 
solr.jetty / com.codahale.metrics.MetricRegistry@3ededc45
   [junit4]   2> 1384034 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 1384035 WARN  (ZkTestServer Run Thread) [    ] 
o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        9       /solr/aliases.json
   [junit4]   2>        9       /solr/clusterprops.json
   [junit4]   2>        7       /solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        9       /solr/clusterstate.json
   [junit4]   2>        4       /solr/autoscaling.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        9       /solr/collections
   [junit4]   2>        8       /solr/live_nodes
   [junit4]   2>        3       /solr/overseer/queue
   [junit4]   2>        3       /solr/overseer/queue-work
   [junit4]   2>        2       /solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 1384037 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43509
   [junit4]   2> 1384037 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 43509
   [junit4]   2> 1384037 INFO  
(TEST-TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry-seed#[F49C4FF42EEF100B])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testLeaderElectionWithZkExpiry
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestLeaderElectionZkExpiry 
-Dtests.method=testLeaderElectionWithZkExpiry -Dtests.seed=F49C4FF42EEF100B 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=dsb 
-Dtests.timezone=America/Asuncion -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 70.3s J1 | 
TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([F49C4FF42EEF100B:22436EAE18F20393]:0)
   [junit4]    >        at 
org.apache.solr.cloud.TestLeaderElectionZkExpiry.testLeaderElectionWithZkExpiry(TestLeaderElectionZkExpiry.java:95)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    >        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    >        at 
java.base/java.lang.reflect.Method.invoke(Method.java:566)
   [junit4]    >        at java.base/java.lang.Thread.run(Thread.java:834)
   [junit4]   2> 1384040 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1384040 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.c.CoreContainer Shutting down CoreContainer instance=922172301
   [junit4]   2> 1384040 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, 
tag=null
   [junit4]   2> 1384040 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, 
tag=null
   [junit4]   2> 1384040 INFO  
(SUITE-TestLeaderElectionZkExpiry-seed#[F49C4FF42EEF100B]-worker) [    ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, 
tag=null
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestLeaderElectionZkExpiry_F49C4FF42EEF100B-001
   [junit4]   2> Feb 06, 2019 2:42:05 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 5 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), 
sim=RandomSimilarity(queryNorm=true): {}, locale=dsb, timezone=America/Asuncion
   [junit4]   2> NOTE: Linux 4.15.0-45-generic amd64/Oracle Corporation 11 
(64-bit)/cpus=8,threads=1,free=245205472,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [MetricsHistoryHandlerTest, 
RestartWhileUpdatingTest, TestTestInjection, TestReversedWildcardFilterFactory, 
DeleteStatusTest, TestTrieFacet, MultiSolrCloudTestCaseTest, 
TestSolrCoreParser, UtilsToolTest, BasicFunctionalityTest, TestSimLargeCluster, 
TestManagedResourceStorage, CollectionStateFormat2Test, TestCloudPivotFacet, 
SubstringBytesRefFilterTest, TestExpandComponent, 
SuggestComponentContextFilterQueryTest, TestFuzzyAnalyzedSuggestions, 
TestLRUCache, TestIndexSearcher, TestDocTermOrdsUninvertLimit, TestSearchPerf, 
OutOfBoxZkACLAndCredentialsProvidersTest, CSVRequestHandlerTest, 
URLClassifyProcessorTest, TestManagedSchemaAPI, TestMissingGroups, 
TestCloudNestedDocsSort, TestPartialUpdateDeduplication, 
SegmentsInfoRequestHandlerTest, JsonLoaderTest, TestBinaryResponseWriter, 
MergeStrategyTest, SolrCloudExampleTest, BadComponentTest, 
BasicAuthIntegrationTest, TestTolerantSearch, TestReRankQParserPlugin, 
BooleanFieldTest, TestShortCircuitedRequests, ZkSolrClientTest, 
DistributedUpdateProcessorTest, TestCloudDeleteByQuery, 
TestCollationFieldDocValues, SimpleMLTQParserTest, TestUniqueKeyFieldResource, 
TestMacros, TestRecoveryHdfs, SharedFSAutoReplicaFailoverTest, 
TestOverriddenPrefixQueryForCustomFieldType, AdminHandlersProxyTest, 
TransactionLogTest, TestJsonFacetsWithNestedObjects, 
SpellingQueryConverterTest, TermVectorComponentTest, TestPerFieldSimilarity, 
TriggerIntegrationTest, BlockJoinFacetRandomTest, TemplateUpdateProcessorTest, 
TestBlobHandler, TestPrepRecovery, SolrGraphiteReporterTest, IndexSchemaTest, 
TestCopyFieldCollectionResource, DateMathParserTest, TestSolrDeletionPolicy1, 
TestRandomFaceting, TestCorePropertiesReload, DistributedFacetExistsSmallTest, 
TestPolicyCloud, HttpPartitionOnCommitTest, LeaderTragicEventTest, 
ResponseHeaderTest, SpatialHeatmapFacetsTest, LukeRequestHandlerTest, 
DistributedQueryElevationComponentTest, DirectSolrSpellCheckerTest, 
TestSolrXml, TestSubQueryTransformerCrossCore, TestIntervalFaceting, 
DistribJoinFromCollectionTest, HdfsRestartWhileUpdatingTest, TestPointFields, 
TestConfigSets, TestDynamicLoading, DisMaxRequestHandlerTest, 
TestRawTransformer, TestShardHandlerFactory, RulesTest, UUIDFieldTest, 
TestLegacyFieldReuse, RequestLoggingTest, TestDownShardTolerantSearch, 
TestBlendedInfixSuggestions, ShardSplitTest, ActionThrottleTest, HLLUtilTest, 
CdcrWithNodesRestartsTest, DistributedMLTComponentTest, RankQueryTest, 
TestRTGBase, ClusterStateTest, DirectSolrConnectionTest, 
DistributedTermsComponentTest, RecoveryAfterSoftCommitTest, 
TestCursorMarkWithoutUniqueKey, PolyFieldTest, TestReplicaProperties, 
PluginInfoTest, TestManagedSynonymGraphFilterFactory, TestChildDocTransformer, 
TestStressCloudBlindAtomicUpdates, TestComponentsName, TestSQLHandler, 
SchemaVersionSpecificBehaviorTest, DocValuesNotIndexedTest, HdfsNNFailoverTest, 
SaslZkACLProviderTest, PeerSyncWithLeaderAndIndexFingerprintCachingTest, 
V2StandaloneTest, PeerSyncTest, TestSolrQueryParser, TestFieldSortValues, 
TestIndexingPerformance, TestPHPSerializedResponseWriter, SuggesterWFSTTest, 
TestStressLucene, FastVectorHighlighterTest, 
PhrasesIdentificationComponentTest, TestDFISimilarityFactory, 
PrimitiveFieldTypeTest, CoreMergeIndexesAdminHandlerTest, 
TestMaxTokenLenTokenizer, TestLockTree, VersionInfoTest, 
TestAuthorizationFramework, AssignBackwardCompatibilityTest, 
BasicDistributedZkTest, BasicZkTest, ChaosMonkeyNothingIsSafeTest, 
ChaosMonkeySafeLeaderTest, ConnectionManagerTest, DeleteInactiveReplicaTest, 
DistributedVersionInfoTest, LeaderElectionTest, 
LeaderFailoverAfterPartitionTest, LeaderFailureAfterFreshStartTest, 
OverseerModifyCollectionTest, OverseerTaskQueueTest, PeerSyncReplicationTest, 
TestCloudConsistency, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, 
TestCryptoKeys, TestDistributedMap, TestHashPartitioner, 
TestLeaderElectionZkExpiry]
   [junit4] Completed [513/838 (1!)] on J1 in 71.05s, 1 test, 1 failure <<< 
FAILURES!

[...truncated 43636 lines...]
[repro] Jenkins log URL: 
https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3509/consoleText

[repro] Revision: 2835e086537ee708dc6037e19b33c5aa3c287a15

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC"
[repro] Repro line:  ant test  -Dtestcase=TestLeaderElectionZkExpiry 
-Dtests.method=testLeaderElectionWithZkExpiry -Dtests.seed=F49C4FF42EEF100B 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=dsb 
-Dtests.timezone=America/Asuncion -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       TestLeaderElectionZkExpiry
[repro] ant compile-test

[...truncated 3588 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestLeaderElectionZkExpiry" -Dtests.showOutput=onerror 
"-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Dtests.seed=F49C4FF42EEF100B 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=dsb 
-Dtests.timezone=America/Asuncion -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[...truncated 75 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.TestLeaderElectionZkExpiry
[repro] Exiting with code 0

[...truncated 43 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to