Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.0/14/

1 tests failed.
FAILED:  
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior

Error Message:
Illegal state, was: down expected:active clusterState:live 
nodes:[]collections:{c1=DocCollection(c1)={   "shards":{"shard1":{       
"parent":null,       "range":null,       "state":"active",       
"replicas":{"core_node1":{           "base_url":"http://127.0.0.1/solr";,        
   "node_name":"node1",           "core":"core1",           "roles":"",         
  "state":"down"}}}},   "router":{"name":"implicit"}}, 
test=LazyCollectionRef(test)}

Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active 
clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
  "shards":{"shard1":{
      "parent":null,
      "range":null,
      "state":"active",
      "replicas":{"core_node1":{
          "base_url":"http://127.0.0.1/solr";,
          "node_name":"node1",
          "core":"core1",
          "roles":"",
          "state":"down"}}}},
  "router":{"name":"implicit"}}, test=LazyCollectionRef(test)}
        at 
__randomizedtesting.SeedInfo.seed([DA0B06A3B6B8D972:B215054F5428833C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.verifyReplicaStatus(AbstractDistribZkTestBase.java:236)
        at 
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1273)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10723 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_DA0B06A3B6B8D972-001/init-core-data-001
   [junit4]   2> 165234 INFO  
(SUITE-OverseerTest-seed#[DA0B06A3B6B8D972]-worker) [    ] o.a.s.SolrTestCaseJ4 
Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 165235 INFO  
(SUITE-OverseerTest-seed#[DA0B06A3B6B8D972]-worker) [    ] o.a.s.SolrTestCaseJ4 
####initCore
   [junit4]   2> 165235 INFO  
(SUITE-OverseerTest-seed#[DA0B06A3B6B8D972]-worker) [    ] o.a.s.SolrTestCaseJ4 
####initCore end
   [junit4]   2> 165236 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testOverseerStatsReset
   [junit4]   2> 165237 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 165241 INFO  (Thread-535) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 165241 INFO  (Thread-535) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 165341 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:60292
   [junit4]   2> 165341 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 165342 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 165357 INFO  (zkCallback-195-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@8e15a45 name:ZooKeeperConnection 
Watcher:127.0.0.1:60292 got event WatchedEvent state:SyncConnected type:None 
path:null path:null type:None
   [junit4]   2> 165357 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 165357 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 165359 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 165359 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 165376 INFO  (zkCallback-196-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6602ac67 
name:ZooKeeperConnection Watcher:127.0.0.1:60292 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 165376 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 165376 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 165377 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 165379 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 165379 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 165386 INFO  (zkCallback-197-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@1b9b7e1b 
name:ZooKeeperConnection Watcher:127.0.0.1:60292/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 165386 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 165387 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 165387 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 165388 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 165388 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 165388 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 165389 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 165389 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 165390 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 165390 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 165391 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 165394 INFO  (zkCallback-198-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@65085a1a 
name:ZooKeeperConnection Watcher:127.0.0.1:60292/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 165394 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 165394 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 165395 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 165396 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 165396 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
   [junit4]   2> 165397 INFO  (zkCallback-197-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 165397 INFO  (zkCallback-197-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 165398 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 165400 INFO  (zkCallback-198-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 165401 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 165401 INFO  (zkCallback-198-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 165402 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 165402 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 165403 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/overseer_elect/election/95907559917092866-127.0.0.1:60292_solr-n_0000000000
   [junit4]   2> 165403 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60292_solr
   [junit4]   2> 165403 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 165404 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907559917092866-127.0.0.1:60292_solr-n_0000000000) starting
   [junit4]   2> 165404 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 165405 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 165406 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 165407 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 165408 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 165412 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 165418 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 165420 INFO  
(OverseerCollectionConfigSetProcessor-95907559917092866-127.0.0.1:60292_solr-n_0000000000)
 [    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer 
operations
   [junit4]   2> 165420 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000000) [    
] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 165425 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000000) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 165425 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"} current state version: 0
   [junit4]   2> 165426 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"}
   [junit4]   2> 165426 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000000) [    
] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 165426 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 165427 INFO  (zkCallback-198-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 165427 INFO  (zkCallback-197-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 165924 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 165925 INFO  (zkCallback-198-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 165925 INFO  (zkCallback-197-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 165927 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection1/leader_elect/shard1/election/95907559917092867-node1_core1-n_0000000000
   [junit4]   2> 165927 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 165928 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard1/leader after winning as 
/collections/collection1/leader_elect/shard1/election/95907559917092867-node1_core1-n_0000000000
   [junit4]   2> 165929 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907559917092866-127.0.0.1:60292_solr-n_0000000000) closing
   [junit4]   2> 165930 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/overseer_elect/election/95907559917092866-127.0.0.1:60292_solr-n_0000000000
   [junit4]   2> 165930 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000000) [    
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60292_solr
   [junit4]   2> 165931 INFO  (OverseerExitThread) [    ] o.a.s.c.Overseer I'm 
exiting , but I'm still the leader
   [junit4]   2> 165931 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/overseer_elect/election/95907559917092866-127.0.0.1:60292_solr-n_0000000001
   [junit4]   2> 165932 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60292_solr
   [junit4]   2> 165932 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 165932 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907559917092866-127.0.0.1:60292_solr-n_0000000001) starting
   [junit4]   2> 165935 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 165935 INFO  
(OverseerCollectionConfigSetProcessor-95907559917092866-127.0.0.1:60292_solr-n_0000000001)
 [    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer 
operations
   [junit4]   2> 165935 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/collections/collection1/leader_elect/shard1/election/95907559917092867-node1_core1-n_0000000000
   [junit4]   2> 165935 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000001) [    
] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 165936 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on 
cancel: /collections/collection1/leaders/shard1/leader 1
   [junit4]   2> 165936 INFO  (zkCallback-198-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 165936 INFO  (zkCallback-197-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 165937 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000001) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 165938 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000001) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://node1/solr/";,
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 165939 INFO  (zkCallback-197-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [0])
   [junit4]   2> 167436 WARN  (zkCallback-198-thread-1) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 167438 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:60292 60292
   [junit4]   2> 167768 INFO  (Thread-535) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:60292 60292
   [junit4]   2> 167769 WARN  (Thread-535) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        2       /solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        2       /solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        2       /solr/overseer/queue
   [junit4]   2>        2       /solr/live_nodes
   [junit4]   2>        2       /solr/collections
   [junit4]   2>        2       /solr/overseer/queue-work
   [junit4]   2>        2       /solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 167770 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.SolrTestCaseJ4 ###Ending testOverseerStatsReset
   [junit4]   2> 167770 INFO  
(TEST-OverseerTest.testOverseerStatsReset-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907559917092866-127.0.0.1:60292_solr-n_0000000001) closing
   [junit4]   2> 167770 INFO  
(OverseerStateUpdate-95907559917092866-127.0.0.1:60292_solr-n_0000000001) [    
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60292_solr
   [junit4]   2> 167771 ERROR (OverseerExitThread) [    ] o.a.s.c.Overseer 
could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:308)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:267)
   [junit4]   2> 167773 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testOverseerFailure
   [junit4]   2> 167773 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 167774 INFO  (Thread-536) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 167774 INFO  (Thread-536) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 167874 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:54832
   [junit4]   2> 167874 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 167880 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 167894 INFO  (zkCallback-201-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4974dfe3 
name:ZooKeeperConnection Watcher:127.0.0.1:54832 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 167894 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 167894 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 167895 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 167895 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 167908 INFO  (zkCallback-202-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@2791d1b9 
name:ZooKeeperConnection Watcher:127.0.0.1:54832 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 167908 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 167908 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 167908 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 167910 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 167911 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 167919 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6a4803f7 
name:ZooKeeperConnection Watcher:127.0.0.1:54832/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 167919 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 167920 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 167920 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 167921 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 167922 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 167922 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 167922 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 167923 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 167923 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 167923 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 167932 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 167942 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3462c846 
name:ZooKeeperConnection Watcher:127.0.0.1:54832/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 167942 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 167943 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 167943 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 167944 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 167944 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /live_nodes/node1
   [junit4]   2> 167945 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 167945 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 167945 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 167945 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 167952 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 167952 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 167954 INFO  (zkCallback-205-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@437487af 
name:ZooKeeperConnection Watcher:127.0.0.1:54832/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 167954 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 167954 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 167955 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 167957 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 167957 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 167958 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 167958 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/overseer_elect/election/95907560082964484-127.0.0.1:54832_solr-n_0000000000
   [junit4]   2> 167958 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54832_solr
   [junit4]   2> 167958 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 167959 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907560082964484-127.0.0.1:54832_solr-n_0000000000) starting
   [junit4]   2> 167959 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 167960 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 167961 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 167961 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 167962 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 167963 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 167964 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 167964 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 167965 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 167965 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 167971 INFO  
(OverseerCollectionConfigSetProcessor-95907560082964484-127.0.0.1:54832_solr-n_0000000000)
 [    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer 
operations
   [junit4]   2> 168974 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"recovering",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"} current state version: 0
   [junit4]   2> 168974 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"recovering",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"}
   [junit4]   2> 168974 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 168974 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 168974 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 168975 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 168975 INFO  (zkCallback-205-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 169473 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 169474 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 169474 INFO  (zkCallback-205-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 169475 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 169479 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000000
   [junit4]   2> 169479 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 169484 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard1/leader after winning as 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000000
   [junit4]   2> 169486 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://node1/solr/";,
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 169494 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 169496 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000000
   [junit4]   2> 169496 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on 
cancel: /collections/collection1/leaders/shard1/leader 1
   [junit4]   2> 169496 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"} current state version: 1
   [junit4]   2> 169496 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 169496 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"}
   [junit4]   2> 169496 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
   [junit4]   2> 169502 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000001
   [junit4]   2> 169503 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard1/leader after winning as 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000001
   [junit4]   2> 169505 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://node1/solr/";,
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 169606 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 169607 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 169607 INFO  (zkCallback-205-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 169972 INFO  
(OverseerCollectionConfigSetProcessor-95907560082964484-127.0.0.1:54832_solr-n_0000000000)
 [    ] o.a.s.c.OverseerTaskProcessor 
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:358)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:175)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 169972 INFO  
(OverseerCollectionConfigSetProcessor-95907560082964484-127.0.0.1:54832_solr-n_0000000000)
 [    ] o.a.s.c.OverseerTaskProcessor According to ZK I 
(id=95907560082964484-127.0.0.1:54832_solr-n_0000000000) am no longer a leader.
   [junit4]   2> 170660 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000001
   [junit4]   2> 170660 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on 
cancel: /collections/collection1/leaders/shard1/leader 2
   [junit4]   2> 170660 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 170662 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000002
   [junit4]   2> 170662 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard1/leader after winning as 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000002
   [junit4]   2> 170664 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 170665 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 170678 INFO  (zkCallback-208-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@8fbdd33 name:ZooKeeperConnection 
Watcher:127.0.0.1:54832/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 170679 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 170680 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 170680 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907560082964484-127.0.0.1:54832_solr-n_0000000000) closing
   [junit4]   2> 170680 INFO  
(OverseerStateUpdate-95907560082964484-127.0.0.1:54832_solr-n_0000000000) [    
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:54832_solr
   [junit4]   2> 170680 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 170683 ERROR (OverseerExitThread) [    ] o.a.s.c.Overseer 
could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:308)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:267)
   [junit4]   2> 170684 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 170684 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/overseer_elect/election/95907560082964485-127.0.0.1:54832_solr-n_0000000001
   [junit4]   2> 170684 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54832_solr
   [junit4]   2> 170684 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 170685 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907560082964485-127.0.0.1:54832_solr-n_0000000001) starting
   [junit4]   2> 170688 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 170689 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 170691 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 170691 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 170693 INFO  
(OverseerCollectionConfigSetProcessor-95907560082964485-127.0.0.1:54832_solr-n_0000000001)
 [    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer 
operations
   [junit4]   2> 170694 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"recovering",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"} current state version: 2
   [junit4]   2> 170694 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"recovering",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://node1/solr/"}
   [junit4]   2> 170694 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.o.ReplicaMutator shard=shard1 is already registered
   [junit4]   2> 170695 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 170695 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 170696 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.Overseer processMessage: queueSize: 0, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://node1/solr/";,
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 170705 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/collections/collection1/leader_elect/shard1/election/95907560082964483-node1_core1-n_0000000002
   [junit4]   2> 170705 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on 
cancel: /collections/collection1/leaders/shard1/leader 3
   [junit4]   2> 170710 INFO  (zkCallback-208-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 170710 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"deletecore",
   [junit4]   2>   "node_name":"node1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "collection":"collection1"} current state version: 3
   [junit4]   2> 170814 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 170815 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 170815 INFO  (zkCallback-208-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 171316 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 171316 INFO  (zkCallback-208-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 171316 INFO  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 171316 INFO  (zkCallback-203-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 171317 INFO  (zkCallback-208-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 172816 WARN  (zkCallback-204-thread-1) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 172819 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:54832 54832
   [junit4]   2> 172980 INFO  (Thread-536) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:54832 54832
   [junit4]   2> 172984 WARN  (Thread-536) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        4       /solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        4       /solr/clusterstate.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        4       /solr/live_nodes
   [junit4]   2>        4       /solr/collections
   [junit4]   2>        2       /solr/overseer/queue
   [junit4]   2>        2       /solr/overseer/queue-work
   [junit4]   2>        2       /solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 172984 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.SolrTestCaseJ4 ###Ending testOverseerFailure
   [junit4]   2> 172984 INFO  
(TEST-OverseerTest.testOverseerFailure-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907560082964485-127.0.0.1:54832_solr-n_0000000001) closing
   [junit4]   2> 172984 INFO  
(OverseerStateUpdate-95907560082964485-127.0.0.1:54832_solr-n_0000000001) [    
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:54832_solr
   [junit4]   2> 172995 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testBadQueueItem
   [junit4]   2> 172996 ERROR (OverseerExitThread) [    ] o.a.s.c.Overseer 
could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:308)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:267)
   [junit4]   2> 172996 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 172996 INFO  (Thread-537) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 172996 INFO  (Thread-537) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 173096 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:37181
   [junit4]   2> 173096 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 173097 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 173099 INFO  (zkCallback-211-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@270c2bb9 
name:ZooKeeperConnection Watcher:127.0.0.1:37181 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 173100 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 173100 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 173104 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 173107 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 173112 INFO  (zkCallback-212-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@32ce7816 
name:ZooKeeperConnection Watcher:127.0.0.1:37181 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 173112 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 173113 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 173113 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 173123 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 173125 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 173128 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@2dd03e5f 
name:ZooKeeperConnection Watcher:127.0.0.1:37181/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 173128 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 173128 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 173129 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 173132 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 173132 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 173133 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 173134 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 173134 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 173134 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 173136 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6edfd8ee 
name:ZooKeeperConnection Watcher:127.0.0.1:37181/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 173136 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 173136 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 173136 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=600000&connTimeout=60000&retry=true
   [junit4]   2> 173138 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 173138 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 173139 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 173139 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/overseer_elect/election/95907560425259011-127.0.0.1:37181_solr-n_0000000000
   [junit4]   2> 173139 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37181_solr
   [junit4]   2> 173139 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 173139 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907560425259011-127.0.0.1:37181_solr-n_0000000000) starting
   [junit4]   2> 173140 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 173141 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 173141 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 173142 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 173143 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 173144 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 173145 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 173147 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 173147 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 173147 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 173148 INFO  
(OverseerCollectionConfigSetProcessor-95907560425259011-127.0.0.1:37181_solr-n_0000000000)
 [    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer 
operations
   [junit4]   2> 173152 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 173153 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 173153 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 173157 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 173164 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5f76443b 
name:ZooKeeperConnection Watcher:127.0.0.1:37181/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 173164 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 173165 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 173165 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 173166 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 173166 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1
   [junit4]   2> 173167 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 173168 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 173168 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 173168 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 173168 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 173174 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 173174 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"} current state version: 0
   [junit4]   2> 173174 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"node1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 173175 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 173175 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 173175 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 173177 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 173178 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 173668 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection1/leader_elect/shard2/election
   [junit4]   2> 173669 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 173669 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 173669 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 173671 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection1/leader_elect/shard2/election/95907560425259012-127.0.0.1_core1-n_0000000000
   [junit4]   2> 173672 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 173673 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard2/leader after winning as 
/collections/collection1/leader_elect/shard2/election/95907560425259012-127.0.0.1_core1-n_0000000000
   [junit4]   2> 173679 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/";,
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 173680 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core2",
   [junit4]   2>   "core_node_name":"node2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"} current state version: 1
   [junit4]   2> 173680 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core2",
   [junit4]   2>   "core_node_name":"node2",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 173680 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
   [junit4]   2> 173681 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
   [junit4]   2> 173782 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 173783 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 173783 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174180 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection1/leader_elect/shard3/election
   [junit4]   2> 174183 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection1/leader_elect/shard3/election/95907560425259012-127.0.0.1_core2-n_0000000000
   [junit4]   2> 174183 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard3
   [junit4]   2> 174184 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard3/leader after winning as 
/collections/collection1/leader_elect/shard3/election/95907560425259012-127.0.0.1_core2-n_0000000000
   [junit4]   2> 174185 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/";,
   [junit4]   2>   "core":"core2",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 174186 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core3",
   [junit4]   2>   "core_node_name":"node3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"} current state version: 2
   [junit4]   2> 174187 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core3",
   [junit4]   2>   "core_node_name":"node3",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 174187 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection already exists with numShards=3
   [junit4]   2> 174187 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 174288 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174289 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174290 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174686 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 174689 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection1/leader_elect/shard1/election/95907560425259012-127.0.0.1_core3-n_0000000000
   [junit4]   2> 174689 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 174690 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard1/leader after winning as 
/collections/collection1/leader_elect/shard1/election/95907560425259012-127.0.0.1_core3-n_0000000000
   [junit4]   2> 174692 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/";,
   [junit4]   2>   "core":"core3",
   [junit4]   2>   "state":"active"} current state version: 3
   [junit4]   2> 174794 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174794 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174794 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174844 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core0",
   [junit4]   2>   "core_node_name":"node0",
   [junit4]   2>   "collection":"",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"} current state version: 4
   [junit4]   2> 174844 ERROR 
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has 
no collection specified: {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core0",
   [junit4]   2>   "core_node_name":"node0",
   [junit4]   2>   "collection":"",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 174845 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"deletecore",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core0",
   [junit4]   2>   "core_node_name":"node0",
   [junit4]   2>   "collection":""} current state version: 4
   [junit4]   2> 174845 ERROR 
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.CollectionMutator Skipping invalid Overseer message because it has 
no collection specified: {
   [junit4]   2>   "operation":"deletecore",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core0",
   [junit4]   2>   "core_node_name":"node0",
   [junit4]   2>   "collection":""}
   [junit4]   2> 174845 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"node1",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"} current state version: 4
   [junit4]   2> 174846 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=3 message={
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "node_name":"127.0.0.1",
   [junit4]   2>   "core":"core1",
   [junit4]   2>   "core_node_name":"node1",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 174846 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ClusterStateMutator building a new cName: collection2
   [junit4]   2> 174846 INFO  
(OverseerStateUpdate-95907560425259011-127.0.0.1:37181_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 174947 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174948 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 174948 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 175345 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/collections/collection1/leader_elect/shard2/election/95907560425259012-127.0.0.1_core1-n_0000000000
   [junit4]   2> 175345 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on 
cancel: /collections/collection1/leaders/shard2/leader 1
   [junit4]   2> 175346 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/collection2/leader_elect/shard2/election
   [junit4]   2> 175346 INFO  (zkCallback-214-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 175346 INFO  (zkCallback-217-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 175347 INFO  (zkCallback-213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 175348 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/collection2/leader_elect/shard2/election/95907560425259012-127.0.0.1_core1-n_0000000000
   [junit4]   2> 175348 INFO  
(TEST-OverseerTest.testBadQueueItem-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection2/leaders/shard2
   [junit4]   2> 175349 INFO 

[...truncated too long message...]

2>      at 
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:730)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.doWork(OverseerAutoReplicaFailoverThread.java:152)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:131)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /clusterprops.json
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:314)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:311)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:311)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:721)
   [junit4]   2>        ... 3 more
   [junit4]   2> 
   [junit4]   2> 227739 WARN  (zkCallback-330-thread-1) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 227741 INFO  
(TEST-OverseerTest.testShardAssignmentBigger-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/collections/collection1/leader_elect/shard1/election/95907563335974926-node10_core10-n_0000000010
   [junit4]   2> 227741 INFO  
(TEST-OverseerTest.testShardAssignmentBigger-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on 
cancel: /collections/collection1/leaders/shard1/leader 7
   [junit4]   2> 227743 INFO  (zkCallback-331-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [2])
   [junit4]   2> 227743 INFO  (zkCallback-332-thread-1) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/collection1/leaders/shard1/leader after winning as 
/collections/collection1/leader_elect/shard1/election/95907563335974927-node11_core11-n_0000000011
   [junit4]   2> 227743 INFO  (zkCallback-332-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [2])
   [junit4]   2> 227744 INFO  (zkCallback-332-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 229243 WARN  (zkCallback-331-thread-1) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 229244 INFO  
(TEST-OverseerTest.testShardAssignmentBigger-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ElectionContext Canceling election 
/collections/collection1/leader_elect/shard1/election/95907563335974927-node11_core11-n_0000000011
   [junit4]   2> 229244 INFO  
(TEST-OverseerTest.testShardAssignmentBigger-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ShardLeaderElectionContextBase Removing leader registration node on 
cancel: /collections/collection1/leaders/shard1/leader 8
   [junit4]   2> 229245 INFO  (zkCallback-332-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 229245 WARN  (zkCallback-332-thread-1) [    ] 
o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: 
[KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 229246 INFO  
(TEST-OverseerTest.testShardAssignmentBigger-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:52898 52898
   [junit4]   2> 229413 INFO  (Thread-547) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:52898 52898
   [junit4]   2> 229414 WARN  (Thread-547) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        14      /solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        14      /solr/clusterstate.json
   [junit4]   2>        5       
/solr/collections/collection1/leader_elect/shard1/election/95907563335974927-node11_core11-n_0000000011
   [junit4]   2>        2       
/solr/collections/collection1/leader_elect/shard1/election/95907563335974923-node7_core7-n_0000000007
   [junit4]   2>        2       
/solr/collections/collection1/leader_elect/shard1/election/95907563335974922-node6_core6-n_0000000004
   [junit4]   2>        2       
/solr/collections/collection1/leader_elect/shard1/election/95907563335974924-node8_core8-n_0000000006
   [junit4]   2>        2       
/solr/collections/collection1/leader_elect/shard1/election/95907563335974922-node6_core18-n_0000000015
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        14      /solr/live_nodes
   [junit4]   2>        14      /solr/collections
   [junit4]   2> 
   [junit4]   2> 229415 INFO  
(TEST-OverseerTest.testShardAssignmentBigger-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.SolrTestCaseJ4 ###Ending testShardAssignmentBigger
   [junit4]   2> 229415 INFO  
(TEST-OverseerTest.testShardAssignmentBigger-seed#[DA0B06A3B6B8D972]) [    ] 
o.a.s.c.Overseer Overseer 
(id=95907563335974915-127.0.0.1:52898_solr-n_0000000000) closing
   [junit4]   2> 229416 INFO  
(OverseerStateUpdate-95907563335974915-127.0.0.1:52898_solr-n_0000000000) [    
] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52898_solr
   [junit4]   2> 229416 ERROR (OverseerExitThread) [    ] o.a.s.c.Overseer 
could not read the data
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: 
KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:348)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:308)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater$2.run(Overseer.java:267)
   [junit4]   2> 232416 INFO  
(SUITE-OverseerTest-seed#[DA0B06A3B6B8D972]-worker) [    ] o.a.s.SolrTestCaseJ4 
###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/solr/build/solr-core/test/J2/temp/solr.cloud.OverseerTest_DA0B06A3B6B8D972-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): {}, 
docValues:{}, maxPointsInLeafNode=1313, maxMBSortInHeap=6.203943683240624, 
sim=ClassicSimilarity, locale=de-CH, timezone=Etc/UTC
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_74 (64-bit)/cpus=4,threads=1,free=152759536,total=386924544
   [junit4]   2> NOTE: All tests run in this JVM: [SampleTest, 
TestStressReorder, TestSchemaVersionResource, TestSweetSpotSimilarityFactory, 
InfoHandlerTest, TestJettySolrRunner, TestXIncludeConfig, 
TestPostingsSolrHighlighter, ShowFileRequestHandlerTest, 
CoreAdminCreateDiscoverTest, DistributedFacetPivotLongTailTest, 
HdfsCollectionsAPIDistributedZkTest, BaseCdcrDistributedZkTest, 
TestSolrConfigHandler, HdfsWriteToMultipleCollectionsTest, QueryParsingTest, 
TestPerFieldSimilarity, TestRTimerTree, TestDynamicFieldCollectionResource, 
RecoveryZkTest, DirectUpdateHandlerOptimizeTest, TestRebalanceLeaders, 
TestSort, TestSolrJ, SpatialRPTFieldTypeTest, TestCoreDiscovery, 
SuggesterWFSTTest, DeleteLastCustomShardedReplicaTest, OverseerTest]
   [junit4] Completed [72/580 (1!)] on J2 in 67.21s, 13 tests, 1 failure, 1 
skipped <<< FAILURES!

[...truncated 1666 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/build.xml:740: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/build.xml:684: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/build.xml:59: The 
following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/solr/build.xml:233: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/solr/common-build.xml:527:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/lucene/common-build.xml:1457:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.0/lucene/common-build.xml:1014:
 There were test failures: 580 suites (10 ignored), 2318 tests, 1 failure, 85 
ignored (70 assumptions) [seed: DA0B06A3B6B8D972]

Total time: 54 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to