Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/4325/
Java: 64bit/jdk1.7.0_72 -XX:-UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest:   
  1) Thread[id=842, name=zkCallback-87-thread-1, state=TIMED_WAITING, 
group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)   
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) 
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.LeaderElectionTest: 
   1) Thread[id=842, name=zkCallback-87-thread-1, state=TIMED_WAITING, 
group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([BF59EA128B49EA81]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=842, name=zkCallback-87-thread-1, state=TIMED_WAITING, 
group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)   
      at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) 
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=842, name=zkCallback-87-thread-1, state=TIMED_WAITING, 
group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([BF59EA128B49EA81]:0)




Build Log:
[...truncated 8916 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderElectionTest
   [junit4]   2> Creating dataDir: 
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderElectionTest
 BF59EA128B49EA81-001\init-core-data-001
   [junit4]   2> 223356 T655 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl 
(false) and clientAuth (false)
   [junit4]   2> 223361 T655 oas.SolrTestCaseJ4.setUp ###Starting 
testParallelElection
   [junit4]   2> 223363 T655 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 223364 T656 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 223478 T655 oasc.ZkTestServer.run start zk server on port:50164
   [junit4]   2> 223497 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 223553 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 223608 T663 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@5bd190a8 
name:ZooKeeperConnection Watcher:127.0.0.1:50164 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 223609 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 223609 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 223613 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 223616 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 223618 T666 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@6e925f2 name:ZooKeeperConnection 
Watcher:127.0.0.1:50164 got event WatchedEvent state:SyncConnected type:None 
path:null path:null type:None
   [junit4]   2> 223619 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 223619 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 223619 T655 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 223635 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 223638 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 223640 T669 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@2639598 name:ZooKeeperConnection 
Watcher:127.0.0.1:50164/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 223640 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 223640 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 223641 T655 oasc.LeaderElectionTest.testParallelElection 
Testing parallel election across 19 shards
   [junit4]   2> 223641 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 223642 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 223645 T672 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@192eb3b5 
name:ZooKeeperConnection Watcher:127.0.0.1:50164/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 223646 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 223646 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 223661 T674 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223662 T678 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223663 T675 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223663 T676 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223665 T673 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223667 T677 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223684 T679 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223684 T681 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223684 T683 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223685 T685 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223685 T687 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223685 T689 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223686 T691 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 223717 T676 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard4/election
   [junit4]   2> 223717 T678 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard6/election
   [junit4]   2> 223718 T674 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard2/election
   [junit4]   2> 223729 T673 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard1/election
   [junit4]   2> 223730 T677 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard5/election
   [junit4]   2> 223731 T679 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard7/election
   [junit4]   2> 223731 T681 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard9/election
   [junit4]   2> 223731 T683 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard11/election
   [junit4]   2> 223731 T685 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard13/election
   [junit4]   2> 223732 T675 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard3/election
   [junit4]   2> 223732 T687 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard15/election
   [junit4]   2> 223732 T689 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard17/election
   [junit4]   2> 223733 T691 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard19/election
   [junit4]   2> 223743 T690 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard18/election
   [junit4]   2> 223744 T688 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard16/election
   [junit4]   2> 223745 T686 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard14/election
   [junit4]   2> 223746 T684 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard12/election
   [junit4]   2> 223747 T682 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard10/election
   [junit4]   2> 223747 T680 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/parshard8/election
   [junit4]   2> 223846 T676 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard4
   [junit4]   2> 223847 T678 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard6
   [junit4]   2> 223847 T674 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard2
   [junit4]   2> 223848 T673 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard1
   [junit4]   2> 223850 T677 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard5
   [junit4]   2> 223850 T679 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard7
   [junit4]   2> 223851 T681 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard9
   [junit4]   2> 223851 T683 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard11
   [junit4]   2> 223852 T685 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard13
   [junit4]   2> 223852 T675 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard3
   [junit4]   2> 223853 T687 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard15
   [junit4]   2> 223853 T689 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard17
   [junit4]   2> 223854 T691 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard19
   [junit4]   2> 223858 T690 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard18
   [junit4]   2> 223859 T688 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard16
   [junit4]   2> 223859 T686 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard14
   [junit4]   2> 223860 T684 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard12
   [junit4]   2> 223860 T682 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard10
   [junit4]   2> 223861 T680 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard8
   [junit4]   2> 223939 T676 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223940 T678 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223940 T674 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223941 T673 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223941 T677 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223941 T679 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223941 T681 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223942 T683 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223942 T685 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223944 T675 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223944 T687 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223945 T689 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223945 T691 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223949 T690 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223950 T688 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223950 T686 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223951 T684 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223951 T682 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 223952 T680 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 224044 T655 oasc.LeaderElectionTest.testParallelElection First 
replicas brought up and registered
   [junit4]   2> 224044 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 224048 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 224059 T694 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@3bb1877c 
name:ZooKeeperConnection Watcher:127.0.0.1:50164/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 224059 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 224059 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 224178 T655 oasc.LeaderElectionTest.testParallelElection 
Second replicas brought up and registered
   [junit4]   2> 224235 T694 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard2
   [junit4]   2> 224237 T715 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard6
   [junit4]   2> 224238 T717 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard3
   [junit4]   2> 224239 T719 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard18
   [junit4]   2> 224239 T721 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard13
   [junit4]   2> 224240 T723 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard9
   [junit4]   2> 224240 T725 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard11
   [junit4]   2> 224241 T727 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard10
   [junit4]   2> 224245 T730 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard16
   [junit4]   2> 224246 T728 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard1
   [junit4]   2> 224246 T726 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard17
   [junit4]   2> 224247 T724 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard12
   [junit4]   2> 224247 T722 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard15
   [junit4]   2> 224248 T720 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard8
   [junit4]   2> 224248 T716 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard4
   [junit4]   2> 224249 T714 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard14
   [junit4]   2> 224249 T718 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard19
   [junit4]   2> 224250 T729 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard7
   [junit4]   2> 224251 T731 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/parshard5
   [junit4]   2> 224359 T694 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224363 T715 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224363 T717 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224364 T719 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224364 T721 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224364 T723 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224365 T725 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224365 T727 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224368 T730 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224369 T728 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224369 T726 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224369 T724 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224370 T722 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224371 T720 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224371 T716 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224371 T714 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224372 T718 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224372 T729 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 224372 T731 
oasc.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess Sleeping for 
2222ms to simulate leadership takeover delay
   [junit4]   2> 226577 T655 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50164 50164
   [junit4]   2> 228800 T656 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50164 50164
   [junit4]   2> 228802 T655 oas.SolrTestCaseJ4.tearDown ###Ending 
testParallelElection
   [junit4]   2> 228811 T655 oas.SolrTestCaseJ4.setUp ###Starting testElection
   [junit4]   2> 228814 T655 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 228815 T732 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 228914 T655 oasc.ZkTestServer.run start zk server on port:50182
   [junit4]   2> 228914 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 228916 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 228926 T739 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@41be96aa 
name:ZooKeeperConnection Watcher:127.0.0.1:50182 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228927 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 228927 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 228945 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 228948 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 228951 T742 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@5551eb4e 
name:ZooKeeperConnection Watcher:127.0.0.1:50182 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228953 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 228953 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 228953 T655 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 228962 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 228965 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 228969 T745 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@189f92d4 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228970 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 228970 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 228973 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 228975 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 228979 T749 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@1c4e8af name:ZooKeeperConnection 
Watcher:127.0.0.1:50182/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 228980 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 228981 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 228981 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 228984 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 228987 T753 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@4f16e7b7 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228988 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 228988 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 228988 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 228992 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 228996 T757 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@12051012 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 228997 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 228998 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 228998 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229001 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229005 T761 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@4d6f119e 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229005 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229005 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229005 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229009 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229011 T765 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@3ab906ea 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229012 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229012 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229012 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229016 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229019 T769 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@377a28a3 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229021 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229021 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229021 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229024 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229027 T773 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@5cfa2f96 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229028 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229028 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229028 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229033 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229036 T777 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@22eb5cb0 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229036 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229036 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229036 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229039 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229044 T781 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@78770d8 name:ZooKeeperConnection 
Watcher:127.0.0.1:50182/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 229044 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229044 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229044 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229047 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229051 T785 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@18ba8912 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229052 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229052 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229052 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229056 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229059 T789 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@77bcae77 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229061 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229061 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229061 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229065 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229068 T793 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@4a2e17ff 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229069 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229069 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229069 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229073 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229085 T797 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@5a2c2990 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229086 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229086 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229086 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229089 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229093 T801 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@6f50fe52 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229094 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229094 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229094 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 229098 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 229102 T805 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@4676dc6e 
name:ZooKeeperConnection Watcher:127.0.0.1:50182/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 229103 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 229103 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 229117 T750 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 229120 T802 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 229124 T794 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 229129 T786 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 229136 T778 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 229151 T770 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229154 T794 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229155 T802 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229158 T750 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229165 T762 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229167 T786 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229175 T754 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229178 T778 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229184 T758 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229184 T798 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229185 T782 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229185 T774 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229186 T766 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229186 T790 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229187 T746 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 229285 T778 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard1
   [junit4]   2> 229306 T778 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 229420 T789 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard1
   [junit4]   2> 229928 T797 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard1
   [junit4]   2> 230435 T793 oasc.LeaderElector.checkIfIamLeader WARN Failed 
setting watch org.apache.zookeeper.KeeperException$NoNodeException: 
KeeperErrorCode = NoNode for 
/collections/collection1/leader_elect/shard1/election/93173505333788687-shard112-n_0000000002
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:344)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:341)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:341)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:183)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:390)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:263)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 230440 T777 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard1
   [junit4]   2> 231916 T793 oasc.LeaderElector.checkIfIamLeader ERROR leader 
elect delete error 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /collections/collection1/leaders/shard1
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:243)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:240)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:240)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:145)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:190)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:390)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:263)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 231917 T793 oasc.LeaderElector$ElectionWatcher.process WARN  
java.lang.UnsupportedOperationException: copy
   [junit4]   2>        at 
org.apache.solr.cloud.ElectionContext.copy(ElectionContext.java:94)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:425)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:153)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:190)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:390)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:263)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 231932 T761 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard1
   [junit4]   2> 233426 T801 oasc.LeaderElector$ElectionWatcher.process WARN  
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:327)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:324)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:324)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:95)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:390)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:263)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 233445 T785 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard1
   [junit4]   2> 235017 T785 oasc.LeaderElector$ElectionWatcher.process WARN  
org.apache.solr.common.SolrException: Could not register as the leader because 
creating the ephemeral registration node in ZooKeeper failed
   [junit4]   2>        at 
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:147)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElectionTest$TestLeaderElectionContext.runLeaderProcess(LeaderElectionTest.java:95)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:198)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:159)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:390)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:263)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /collections/collection1/leaders
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:293)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:290)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:290)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:485)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:454)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:441)
   [junit4]   2>        at 
org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:139)
   [junit4]   2>        at 
org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
   [junit4]   2>        at 
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:135)
   [junit4]   2>        ... 11 more
   [junit4]   2> 
   [junit4]   2> 235025 T655 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50182 50182
   [junit4]   2> 237279 T732 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50182 50182
   [junit4]   2> 237283 T732 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN 
Watch limit violations: 
   [junit4]   2>        Maximum concurrent data watches above limit:
   [junit4]   2>        
   [junit4]   2>                4       
/solr/collections/collection1/leader_elect/shard1/election/93173505333788684-shard19-n_0000000010
   [junit4]   2>                2       
/solr/collections/collection1/leader_elect/shard1/election/93173505333788689-shard114-n_0000000003
   [junit4]   2>                2       
/solr/collections/collection1/leader_elect/shard1/election/93173505333788687-shard112-n_0000000002
   [junit4]   2>        
   [junit4]   2> 237286 T655 oas.SolrTestCaseJ4.tearDown ###Ending testElection
   [junit4]   2> 237295 T655 oas.SolrTestCaseJ4.setUp ###Starting 
testStressElection
   [junit4]   2> 237297 T655 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 237299 T807 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 237399 T655 oasc.ZkTestServer.run start zk server on port:50239
   [junit4]   2> 237399 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 237401 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 237414 T814 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@bbc0765 name:ZooKeeperConnection 
Watcher:127.0.0.1:50239 got event WatchedEvent state:SyncConnected type:None 
path:null path:null type:None
   [junit4]   2> 237414 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 237414 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 237424 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 237426 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 237432 T817 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@6955c933 
name:ZooKeeperConnection Watcher:127.0.0.1:50239 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 237432 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 237432 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 237432 T655 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 237442 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 237445 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 237452 T820 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@55cbe5d2 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 237452 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 237452 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 237454 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 237457 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 237461 T824 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@7a3fafbd 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 237461 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 237461 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 237465 T825 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 237471 T825 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard1/election
   [junit4]   2> 237482 T825 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard1
   [junit4]   2> 237490 T825 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 239469 T826 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 239474 T826 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 241043 T808 oazs.NIOServerCnxn.doIO WARN caught end of stream 
exception EndOfStreamException: Unable to read additional data from client 
sessionid 0x14b04cf59070003, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 241781 T834 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@5c9f4b8f 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 242309 T820 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@55cbe5d2 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 242309 T820 oascc.ConnectionManager.process zkClient has 
disconnected
   [junit4]   2> 242922 T826 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 242922 T826 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 242923 T826 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 242926 T826 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 244278 T842 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@5ce3e704 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 244282 T808 oazs.NIOServerCnxn.doIO WARN caught end of stream 
exception EndOfStreamException: Unable to read additional data from client 
sessionid 0x14b04cf59070004, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 244283 T830 oaz.ClientCnxn$SendThread.run WARN Session 
0x14b04cf59070004 for server 127.0.0.1/127.0.0.1:50239, unexpected error, 
closing socket connection and attempting reconnect 
java.nio.channels.ClosedChannelException
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:252)
   [junit4]   2>        at 
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:295)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 
   [junit4]   2> 244285 T822 oaz.ClientCnxn$SendThread.run WARN Session 
0x14b04cf59070003 for server null, unexpected error, closing socket connection 
and attempting reconnect java.nio.channels.CancelledKeyException
   [junit4]   2>        at 
sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
   [junit4]   2>        at 
sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.enableWrite(ClientCnxnSocketNIO.java:389)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:373)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 
   [junit4]   2> 244287 T810 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected 
Exception:  java.nio.channels.CancelledKeyException
   [junit4]   2>        at 
sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
   [junit4]   2>        at 
sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
   [junit4]   2>        at 
org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
   [junit4]   2>        at 
org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
   [junit4]   2>        at 
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
   [junit4]   2> 
   [junit4]   2> 244291 T826 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 244291 T808 oazs.NIOServerCnxn.doIO WARN caught end of stream 
exception EndOfStreamException: Unable to read additional data from client 
sessionid 0x14b04cf59070005, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 244295 T820 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@55cbe5d2 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:Expired type:None path:null path:null type:None
   [junit4]   2> 244295 T820 oascc.ConnectionManager.process Our previous 
ZooKeeper session was expired. Attempting to reconnect to recover relationship 
with ZooKeeper...
   [junit4]   2> 244298 T820 oascc.DefaultConnectionStrategy.reconnect 
Connection expired - starting a new one...
   [junit4]   2> 244299 T826 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 244300 T820 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 244307 T848 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@55cbe5d2 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 244313 T849 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@e718f00 name:ZooKeeperConnection 
Watcher:127.0.0.1:50239/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 244314 T820 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 244314 T820 oascc.ConnectionManager$1.update Connection with 
ZooKeeper reestablished.
   [junit4]   2> 244314 T820 oascc.DefaultConnectionStrategy.reconnect 
Reconnected to ZooKeeper
   [junit4]   2> 244314 T820 oascc.ConnectionManager.process Connected:true
   [junit4]   2> 244315 T826 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 244315 T826 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 244316 T826 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 244319 T826 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 244322 T854 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@3d98aa72 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 244322 T826 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 244322 T826 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 244327 T837 oasc.LeaderElectionTest$ClientThread.run ERROR 
setup failed java.lang.InterruptedException
   [junit4]   2>        at java.lang.Object.wait(Native Method)
   [junit4]   2>        at java.lang.Object.wait(Object.java:503)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:781)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:385)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:300)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:256)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:164)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:172)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 244393 T831 oaz.ClientCnxn$EventThread.processEvent ERROR 
Error while calling watcher  java.util.concurrent.RejectedExecutionException: 
Task java.util.concurrent.FutureTask@2492eec9 rejected from 
java.util.concurrent.ThreadPoolExecutor@2a7ae0e1[Terminated, pool size = 0, 
active threads = 0, queued tasks = 0, completed tasks = 1]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
   [junit4]   2>        at 
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:260)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 244396 T824 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@7a3fafbd 
name:ZooKeeperConnection Watcher:127.0.0.1:50239/solr got event WatchedEvent 
state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 244396 T824 oascc.ConnectionManager.process Client->ZooKeeper 
status change trigger but we are already closed
   [junit4]   2> 244412 T655 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50239 50239
   [junit4]   2> 246893 T807 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50239 50239
   [junit4]   2> 246897 T655 oas.SolrTestCaseJ4.tearDown ###Ending 
testStressElection
   [junit4]   2> 246906 T655 oas.SolrTestCaseJ4.setUp ###Starting testBasic
   [junit4]   2> 246907 T655 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 246910 T856 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 247020 T655 oasc.ZkTestServer.run start zk server on port:50270
   [junit4]   2> 247020 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 247023 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 247031 T863 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@6b6251c1 
name:ZooKeeperConnection Watcher:127.0.0.1:50270 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 247033 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 247034 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 247041 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 247042 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 247050 T866 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@6ddead5c 
name:ZooKeeperConnection Watcher:127.0.0.1:50270 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 247052 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 247052 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 247052 T655 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 247062 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 247065 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 247067 T869 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@70df8d56 
name:ZooKeeperConnection Watcher:127.0.0.1:50270/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 247068 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 247068 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 247070 T655 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 247074 T655 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard2/election
   [junit4]   2> 247084 T655 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard2
   [junit4]   2> 247091 T655 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 247100 T655 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50270 50270
   [junit4]   2> 249582 T856 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50270 50270
   [junit4]   2> 249587 T655 oas.SolrTestCaseJ4.tearDown ###Ending testBasic
   [junit4]   2> 249596 T655 oas.SolrTestCaseJ4.setUp ###Starting 
testCancelElection
   [junit4]   2> 249599 T655 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 249601 T870 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 249711 T655 oasc.ZkTestServer.run start zk server on port:50282
   [junit4]   2> 249711 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 249714 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 249719 T877 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@2f8d2173 
name:ZooKeeperConnection Watcher:127.0.0.1:50282 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 249720 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 249720 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 249724 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 249727 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 249730 T880 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@59d2d405 
name:ZooKeeperConnection Watcher:127.0.0.1:50282 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 249730 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 249730 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 249730 T655 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 249736 T655 
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default 
ZkCredentialsProvider
   [junit4]   2> 249740 T655 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 249743 T883 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@7b9b3dcf 
name:ZooKeeperConnection Watcher:127.0.0.1:50282/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 249743 T655 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 249744 T655 oascc.SolrZkClient.createZkACLProvider Using 
default ZkACLProvider
   [junit4]   2> 249746 T655 oascc.SolrZkClient.makePath makePath: 
/collections/collection2
   [junit4]   2> 249752 T655 oascc.SolrZkClient.makePath makePath: 
/collections/collection2/leader_elect/slice1/election
   [junit4]   2> 249766 T655 oascc.SolrZkClient.makePath makePath: 
/collections/collection2/leaders/slice1
   [junit4]   2> 249775 T655 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 251989 T655 oasc.ElectionContext.cancelElection canceling 
election 
/collections/collection2/leader_elect/slice1/election/93173506695757826-dummynode1-n_0000000000
   [junit4]   2> 251992 T883 oascc.SolrZkClient.makePath makePath: 
/collections/collection2/leaders/slice1
   [junit4]   2> 253094 T655 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50282 50282
   [junit4]   2> 255345 T870 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:50282 50282
   [junit4]   2> 255349 T655 oas.SolrTestCaseJ4.tearDown ###Ending 
testCancelElection
   [junit4]   2> 255351 T655 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 32009 T654 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will 
linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 52015 T654 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 
thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest: 
   [junit4]   2>           1) Thread[id=842, name=zkCallback-87-thread-1, 
state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 52017 T654 ccr.ThreadLeakControl.tryToInterruptAll Starting to 
interrupt leaked threads:
   [junit4]   2>           1) Thread[id=842, name=zkCallback-87-thread-1, 
state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 55019 T654 ccr.ThreadLeakControl.tryToInterruptAll SEVERE 
There are still zombie threads that couldn't be terminated:
   [junit4]   2>           1) Thread[id=842, name=zkCallback-87-thread-1, 
state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]   2>                at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=CheapBastard, 
sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=zh, 
timezone=ART
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_72 
(64-bit)/cpus=2,threads=2,free=173402888,total=282066944
   [junit4]   2> NOTE: All tests run in this JVM: [TestSchemaResource, 
TestConfigSets, TestDistributedMissingSort, XmlUpdateRequestHandlerTest, 
TestWordDelimiterFilterFactory, UUIDUpdateProcessorFallbackTest, 
TestSortingResponseWriter, TestManagedSchemaFieldResource, FacetPivotSmallTest, 
SuggesterFSTTest, MultiThreadedOCPTest, LeaderElectionTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionTest 
-Dtests.seed=BF59EA128B49EA81 -Dtests.slow=true -Dtests.locale=zh 
-Dtests.timezone=ART -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s | LeaderElectionTest (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.LeaderElectionTest: 
   [junit4]    >    1) Thread[id=842, name=zkCallback-87-thread-1, 
state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([BF59EA128B49EA81]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=842, name=zkCallback-87-thread-1, 
state=TIMED_WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([BF59EA128B49EA81]:0)
   [junit4] Completed in 55.11s, 5 tests, 2 errors <<< FAILURES!

[...truncated 1362 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\build.xml:529: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\build.xml:477: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\build.xml:61: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\extra-targets.xml:39: 
The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\build.xml:190: The 
following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\solr\common-build.xml:508:
 The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\lucene\common-build.xml:1363:
 The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-5.x-Windows\lucene\common-build.xml:970:
 There were test failures: 463 suites, 1873 tests, 2 suite-level errors, 1736 
ignored (12 assumptions)

Total time: 44 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0_72 
-XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to