Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23313/
Java: 64bit/jdk-12-ea+12 -XX:+UseCompressedOops -XX:+UseSerialGC

15 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest

Error Message:
44 threads leaked from SUITE scope at 
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:     1) 
Thread[id=685, name=qtp1309702070-685, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    2) 
Thread[id=740, name=Scheduler-124753301, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    3) 
Thread[id=636, name=qtp1179819950-636, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    4) 
Thread[id=653, 
name=qtp567423036-653-acceptor-0@790c5eb0-ServerConnector@3059f8ce{SSL,[ssl, 
http/1.1]}{127.0.0.1:41425}, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)       
  at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:525)
         at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:277)
         at 
app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:385)  
       at 
app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:648)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    5) 
Thread[id=650, name=qtp567423036-650, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    6) 
Thread[id=683, name=qtp1309702070-683, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    7) 
Thread[id=680, name=qtp1309702070-680, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    8) 
Thread[id=684, name=qtp1309702070-684, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)    9) 
Thread[id=638, name=qtp1179819950-638, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   10) 
Thread[id=651, name=qtp567423036-651, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   11) 
Thread[id=676, name=MetricsHistoryHandler-204-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   12) 
Thread[id=678, name=qtp1309702070-678, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   13) 
Thread[id=655, name=qtp567423036-655, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   14) 
Thread[id=682, 
name=qtp1309702070-682-acceptor-0@1ef8f7b5-ServerConnector@44afc45e{SSL,[ssl, 
http/1.1]}{127.0.0.1:42365}, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)       
  at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:525)
         at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:277)
         at 
app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:385)  
       at 
app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:648)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   15) 
Thread[id=712, name=coreContainerWorkExecutor-198-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]      
   at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   16) 
Thread[id=644, name=Scheduler-1641907467, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   17) 
Thread[id=752, name=updateExecutor-261-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   18) 
Thread[id=738, name=Scheduler-1626290695, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   19) 
Thread[id=711, name=Thread-112, state=WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/java.lang.Object.wait(Native Method)         at 
java.base@12-ea/java.lang.Object.wait(Object.java:328)         at 
app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1899)   20) 
Thread[id=733, name=coreContainerWorkExecutor-207-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]      
   at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   21) 
Thread[id=690, name=Thread-111, state=WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/java.lang.Object.wait(Native Method)         at 
java.base@12-ea/java.lang.Object.wait(Object.java:328)         at 
app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1899)   22) 
Thread[id=679, name=qtp1309702070-679, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   23) 
Thread[id=652, name=qtp567423036-652, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   24) 
Thread[id=637, name=qtp1179819950-637, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   25) 
Thread[id=640, 
name=qtp1179819950-640-acceptor-0@69c626f5-ServerConnector@7c161655{SSL,[ssl, 
http/1.1]}{127.0.0.1:44953}, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)       
  at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:525)
         at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:277)
         at 
app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:385)  
       at 
app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:648)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   26) 
Thread[id=656, name=qtp567423036-656, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   27) 
Thread[id=730, name=SolrRrdBackendFactory-219-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   28) 
Thread[id=686, name=Scheduler-1373336164, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   29) 
Thread[id=649, name=qtp567423036-649, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   30) 
Thread[id=709, name=SolrRrdBackendFactory-212-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   31) 
Thread[id=643, name=qtp1179819950-643, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   32) 
Thread[id=731, name=MetricsHistoryHandler-220-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   33) 
Thread[id=641, name=qtp1179819950-641, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   34) 
Thread[id=753, name=recoveryExecutor-262-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   35) 
Thread[id=654, name=qtp567423036-654, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   36) 
Thread[id=691, name=coreContainerWorkExecutor-196-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]      
   at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
         at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   37) 
Thread[id=639, name=qtp1179819950-639, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   38) 
Thread[id=681, name=qtp1309702070-681, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)         at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
         at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)  
       at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) 
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
         at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
         at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
         at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   39) 
Thread[id=710, name=MetricsHistoryHandler-213-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   40) 
Thread[id=732, name=Thread-113, state=WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/java.lang.Object.wait(Native Method)         at 
java.base@12-ea/java.lang.Object.wait(Object.java:328)         at 
app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1899)   41) 
Thread[id=675, name=SolrRrdBackendFactory-203-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   42) 
Thread[id=642, name=qtp1179819950-642, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
         at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   43) 
Thread[id=657, name=Scheduler-1010320365, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)   44) 
Thread[id=739, name=Scheduler-1047015637, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)         at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
         at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
         at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
         at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
         at java.base@12-ea/java.lang.Thread.run(Thread.java:835)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 44 threads leaked from 
SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 
   1) Thread[id=685, name=qtp1309702070-685, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   2) Thread[id=740, name=Scheduler-124753301, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   3) Thread[id=636, name=qtp1179819950-636, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   4) Thread[id=653, 
name=qtp567423036-653-acceptor-0@790c5eb0-ServerConnector@3059f8ce{SSL,[ssl, 
http/1.1]}{127.0.0.1:41425}, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept0(Native 
Method)
        at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:525)
        at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:277)
        at 
app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:385)
        at 
app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:648)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   5) Thread[id=650, name=qtp567423036-650, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   6) Thread[id=683, name=qtp1309702070-683, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   7) Thread[id=680, name=qtp1309702070-680, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   8) Thread[id=684, name=qtp1309702070-684, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
   9) Thread[id=638, name=qtp1179819950-638, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  10) Thread[id=651, name=qtp567423036-651, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  11) Thread[id=676, name=MetricsHistoryHandler-204-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  12) Thread[id=678, name=qtp1309702070-678, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  13) Thread[id=655, name=qtp567423036-655, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  14) Thread[id=682, 
name=qtp1309702070-682-acceptor-0@1ef8f7b5-ServerConnector@44afc45e{SSL,[ssl, 
http/1.1]}{127.0.0.1:42365}, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept0(Native 
Method)
        at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:525)
        at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:277)
        at 
app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:385)
        at 
app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:648)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  15) Thread[id=712, name=coreContainerWorkExecutor-198-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  16) Thread[id=644, name=Scheduler-1641907467, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  17) Thread[id=752, name=updateExecutor-261-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  18) Thread[id=738, name=Scheduler-1626290695, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  19) Thread[id=711, name=Thread-112, state=WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/java.lang.Object.wait(Native Method)
        at java.base@12-ea/java.lang.Object.wait(Object.java:328)
        at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1899)
  20) Thread[id=733, name=coreContainerWorkExecutor-207-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  21) Thread[id=690, name=Thread-111, state=WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/java.lang.Object.wait(Native Method)
        at java.base@12-ea/java.lang.Object.wait(Object.java:328)
        at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1899)
  22) Thread[id=679, name=qtp1309702070-679, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  23) Thread[id=652, name=qtp567423036-652, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  24) Thread[id=637, name=qtp1179819950-637, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  25) Thread[id=640, 
name=qtp1179819950-640-acceptor-0@69c626f5-ServerConnector@7c161655{SSL,[ssl, 
http/1.1]}{127.0.0.1:44953}, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept0(Native 
Method)
        at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:525)
        at 
java.base@12-ea/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:277)
        at 
app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:385)
        at 
app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:648)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  26) Thread[id=656, name=qtp567423036-656, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  27) Thread[id=730, name=SolrRrdBackendFactory-219-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  28) Thread[id=686, name=Scheduler-1373336164, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  29) Thread[id=649, name=qtp567423036-649, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  30) Thread[id=709, name=SolrRrdBackendFactory-212-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  31) Thread[id=643, name=qtp1179819950-643, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  32) Thread[id=731, name=MetricsHistoryHandler-220-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at 
java.base@12-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  33) Thread[id=641, name=qtp1179819950-641, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  34) Thread[id=753, name=recoveryExecutor-262-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  35) Thread[id=654, name=qtp567423036-654, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:656)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:46)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:720)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  36) Thread[id=691, name=coreContainerWorkExecutor-196-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at 
java.base@12-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
java.base@12-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  37) Thread[id=639, name=qtp1179819950-639, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  38) Thread[id=681, name=qtp1309702070-681, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/sun.nio.ch.EPoll.wait(Native Method)
        at 
java.base@12-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at 
java.base@12-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@12-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:423)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$166/0x0000000100387440.run(Unknown
 Source)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at 
app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base@12-ea/java.lang.Thread.run(Thread.java:835)
  39) Thread[id=710, name=MetricsHistoryHandler-213-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.base@12-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at 
java.base@12-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
java.base@12-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123

[...truncated too long message...]

   ] o.a.s.c.OverseerElectionContext I am going to be the leader 
127.0.0.1:46015_solr
   [junit4]   2> 104724 INFO  (zkCallback-1077-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 104724 INFO  (jetty-closer-1042-thread-1) [    ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, 
tag=null
   [junit4]   2> 104724 INFO  (jetty-closer-1042-thread-1) [    ] 
o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@5e4a0931: rootName = 
solr_46015, domain = solr.jetty, service url = null, agent id = null] for 
registry solr.jetty / com.codahale.metrics.MetricRegistry@7301ea96
   [junit4]   2> 104724 INFO  (jetty-closer-1042-thread-1) [    ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, 
tag=null
   [junit4]   2> 104725 INFO  (jetty-closer-1042-thread-2) [    ] 
o.a.s.c.Overseer Overseer 
(id=72161558582788102-127.0.0.1:34949_solr-n_0000000000) closing
   [junit4]   2> 104725 INFO  (closeThreadPool-1090-thread-1) [    ] 
o.a.s.c.Overseer Overseer 
(id=72161558582788102-127.0.0.1:34949_solr-n_0000000000) closing
   [junit4]   2> 104727 INFO  (jetty-closer-1042-thread-1) [    ] 
o.e.j.s.AbstractConnector Stopped ServerConnector@500aad1b{SSL,[ssl, 
http/1.1]}{127.0.0.1:0}
   [junit4]   2> 104728 INFO  (jetty-closer-1042-thread-1) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@7814b2fd{/solr,null,UNAVAILABLE}
   [junit4]   2> 104728 INFO  (jetty-closer-1042-thread-1) [    ] 
o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 104729 INFO  (jetty-closer-1042-thread-2) [    ] 
o.e.j.s.AbstractConnector Stopped ServerConnector@23c6b1dc{SSL,[ssl, 
http/1.1]}{127.0.0.1:0}
   [junit4]   2> 104729 INFO  (jetty-closer-1042-thread-2) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@139c3d25{/solr,null,UNAVAILABLE}
   [junit4]   2> 104729 INFO  (jetty-closer-1042-thread-2) [    ] 
o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 104730 INFO  
(TEST-CloudMLTQParserTest.testMLTQParser-seed#[A95595E15BB29190]) [    ] 
o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 104730 ERROR 
(TEST-CloudMLTQParserTest.testMLTQParser-seed#[A95595E15BB29190]) [    ] 
o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper 
server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 104730 WARN  (ZkTestServer Run Thread) [    ] 
o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/aliases.json
   [junit4]   2>        3       /solr/collections/mlt-collection/terms/shard2
   [junit4]   2>        3       /solr/collections/mlt-collection/terms/shard1
   [junit4]   2>        2       /solr/security.json
   [junit4]   2>        2       /solr/configs/conf
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        13      /solr/collections/mlt-collection/state.json
   [junit4]   2>        5       /solr/clusterstate.json
   [junit4]   2>        5       /solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        5       /solr/live_nodes
   [junit4]   2>        5       /solr/collections
   [junit4]   2> 
   [junit4]   2> 104731 INFO  
(TEST-CloudMLTQParserTest.testMLTQParser-seed#[A95595E15BB29190]) [    ] 
o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43165
   [junit4]   2> 104731 INFO  
(TEST-CloudMLTQParserTest.testMLTQParser-seed#[A95595E15BB29190]) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1 43165
   [junit4]   2> 104731 INFO  
(TEST-CloudMLTQParserTest.testMLTQParser-seed#[A95595E15BB29190]) [    ] 
o.a.s.SolrTestCaseJ4 ###Ending testMLTQParser
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CloudMLTQParserTest 
-Dtests.method=testMLTQParser -Dtests.seed=A95595E15BB29190 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=teo-UG -Dtests.timezone=Asia/Damascus -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] ERROR   3.39s J2 | CloudMLTQParserTest.testMLTQParser <<<
   [junit4]    > Throwable #1: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:34949/solr/mlt-collection: Error from server 
at https://127.0.0.1:34949/solr/mlt-collection_shard1_replica_n1: Error 
completing MLT request. Could not fetch document with id [17]
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([A95595E15BB29190:9AE5DC1A29318F2A]:0)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1110)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
   [junit4]    >        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]    >        at 
org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:983)
   [junit4]    >        at 
org.apache.solr.search.mlt.CloudMLTQParserTest.testMLTQParser(CloudMLTQParserTest.java:102)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    >        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    >        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    >        at 
java.base/java.lang.reflect.Method.invoke(Method.java:566)
   [junit4]    >        at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   2> 104732 INFO  
(SUITE-CloudMLTQParserTest-seed#[A95595E15BB29190]-worker) [    ] 
o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 104732 INFO  
(SUITE-CloudMLTQParserTest-seed#[A95595E15BB29190]-worker) [    ] 
o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43165
   [junit4]   2> 104732 INFO  
(SUITE-CloudMLTQParserTest-seed#[A95595E15BB29190]-worker) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1 43165
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.search.mlt.CloudMLTQParserTest_A95595E15BB29190-002
   [junit4]   2> Dec 09, 2018 2:20:13 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 8 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): 
{_version_=PostingsFormat(name=Direct), _root_=PostingsFormat(name=Direct), 
lowerfilt_u=PostingsFormat(name=Direct), id=Lucene50(blocksize=128), 
lowerfilt1_u=Lucene50(blocksize=128)}, 
docValues:{_version_=DocValuesFormat(name=Asserting)}, 
maxPointsInLeafNode=1441, maxMBSortInHeap=7.584417788666809, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@72307a97),
 locale=teo-UG, timezone=Asia/Damascus
   [junit4]   2> NOTE: Linux 4.15.0-39-generic amd64/Oracle Corporation 12-ea 
(64-bit)/cpus=8,threads=1,free=117217992,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: 
[ChaosMonkeySafeLeaderWithPullReplicasTest, CloudMLTQParserTest, 
CloudMLTQParserTest]
   [junit4] Completed [10/10 (4!)] on J2 in 31.80s, 9 tests, 4 errors <<< 
FAILURES!

[...truncated 19 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1572: 
The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1099: 
There were test failures: 10 suites, 50 tests, 4 suite-level errors, 7 errors, 
27 ignored [seed: A95595E15BB29190]

Total time: 1 minute 47 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   2/5 failed: 
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
[repro]   2/5 failed: org.apache.solr.search.mlt.CloudMLTQParserTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all 
warnings.
<Git Blamer> GIT_COMMIT=df84a3c9815a85aa140e5013b44488f45de9a203, 
workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #23312
Recording test results
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to