[JENKINS] Lucene-Solr-cloud2refimpl-Windows (64bit/jdk-11.0.6) - Build # 49 - Still Failing!

Policeman Jenkins Server Tue, 20 Oct 2020 16:07:31 -0700

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-cloud2refimpl-Windows/49/
Java: 64bit/jdk-11.0.6 -XX:-UseCompressedOops -XX:+UseParallelGC


58 tests failed.
FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: 
Timeout waiting for live nodes, currently they are: [127.0.0.1:49631_solr, 
127.0.0.1:49632_solr]

Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: 
Timeout waiting for live nodes, currently they are: [127.0.0.1:49631_solr, 
127.0.0.1:49632_solr]
        at __randomizedtesting.SeedInfo.seed([9D7B9A480C41C393]:0)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:340)
        at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.build(SolrCloudTestCase.java:251)
        at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:241)
        at 
org.apache.solr.cloud.AddReplicaTest.setupCluster(AddReplicaTest.java:60)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:882)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.util.concurrent.TimeoutException: Timeout waiting for live 
nodes, currently they are: [127.0.0.1:49631_solr, 127.0.0.1:49632_solr]
        at 
org.apache.solr.common.cloud.ZkStateReader.waitForLiveNodes(ZkStateReader.java:1825)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.waitForNode(MiniSolrCloudCluster.java:390)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.waitForAllNodes(MiniSolrCloudCluster.java:364)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:335)
        ... 26 more


FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
java.lang.RuntimeException: Timeout waiting for executor to shutdown

Stack Trace:
java.lang.RuntimeException: Timeout waiting for executor to shutdown
        at __randomizedtesting.SeedInfo.seed([9D7B9A480C41C393]:0)
        at 
org.apache.solr.common.util.ExecutorUtil.awaitTermination(ExecutorUtil.java:89)
        at 
org.apache.solr.common.util.ExecutorUtil.shutdownAndAwaitTermination(ExecutorUtil.java:78)
        at 
org.apache.solr.common.ParWork.shutdownRootSharedExec(ParWork.java:104)
        at 
org.apache.solr.common.ParWork.shutdownRootSharedExec(ParWork.java:93)
        at org.apache.solr.SolrTestCase.afterSolrTestCase(SolrTestCase.java:464)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:905)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 The process cannot access the file because it is being used by another process.

   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001


Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 The process cannot access the file because it is being used by another process.

   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data\version-2
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1\data
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper\server1
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001\zookeeper
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001\tempDir-001
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_9D7B9A480C41C393-001

        at org.apache.lucene.util.IOUtils.rm(IOUtils.java:319)
        at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
        at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 98 threads leaked from 
SUITE scope at org.apache.solr.cloud.AddReplicaTest: 
   1) Thread[id=600, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker541, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   2) Thread[id=427, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker374, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   3) Thread[id=396, name=ProcessThread(sid:0 cport:49605):, state=WAITING, 
group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:134)
   4) Thread[id=385, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker356, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   5) Thread[id=414, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker361-EventThread,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
   6) Thread[id=441, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker388, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   7) Thread[id=491, 
name=zkCallbackExecutor-RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker361434,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   8) Thread[id=391, name=NIOServerCxnFactory.SelectorThread-0, state=RUNNABLE, 
group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391)
   9) Thread[id=444, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker391, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  10) Thread[id=401, 
name=zkConnManagerCallbackExecutor-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker359,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  11) Thread[id=464, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker411, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  12) Thread[id=440, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker387, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  13) Thread[id=397, 
name=SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker-SendThread(127.0.0.1:49605),
 state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at 
app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345)
        at 
app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)
  14) Thread[id=467, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker414, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  15) Thread[id=398, 
name=SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker-EventThread, 
state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
  16) Thread[id=412, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker361-SendThread(127.0.0.1:49605),
 state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at 
app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345)
        at 
app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)
  17) Thread[id=474, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker421, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  18) Thread[id=433, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker380, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  19) Thread[id=415, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker362-SendThread(127.0.0.1:49605),
 state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at 
app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345)
        at 
app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)
  20) Thread[id=597, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker538, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  21) Thread[id=443, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker390, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  22) Thread[id=462, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker409, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  23) Thread[id=388, name=Timer-0, state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/java.lang.Object.wait(Native Method)
        at [email protected]/java.lang.Object.wait(Object.java:328)
        at [email protected]/java.util.TimerThread.mainLoop(Timer.java:527)
        at [email protected]/java.util.TimerThread.run(Timer.java:506)
  24) Thread[id=416, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker362-EventThread,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
  25) Thread[id=450, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker397, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  26) Thread[id=473, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker420, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  27) Thread[id=393, name=NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:0, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.select(NIOServerCnxnFactory.java:229)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.run(NIOServerCnxnFactory.java:205)
  28) Thread[id=426, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker373, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  29) Thread[id=452, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker399, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  30) Thread[id=490, 
name=zkCallbackExecutor-RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker360433,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  31) Thread[id=451, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker398, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  32) Thread[id=456, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker403, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  33) Thread[id=446, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker393, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  34) Thread[id=434, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker381, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  35) Thread[id=599, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker540, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  36) Thread[id=593, 
name=SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker-EventThread, 
state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
  37) Thread[id=435, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker382, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  38) Thread[id=392, name=NIOServerCxnFactory.SelectorThread-1, state=RUNNABLE, 
group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391)
  39) Thread[id=439, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker386, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  40) Thread[id=436, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker383, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  41) Thread[id=463, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker410, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  42) Thread[id=471, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker418, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  43) Thread[id=469, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker416, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  44) Thread[id=448, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker395, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  45) Thread[id=595, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker536, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000100543440.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$224/0x00000001004cb040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  46) Thread[id=430, 
name=RootExec-SUITE-AddReplicaTest-seed#[9D7B9A480C41C393]-worker377, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at app//org.apache.solr.common.util.ExecutorUtil$MDCAwar

[...truncated too long message...]

ith --info or --debug option to get more log output. Run with --scan to get 
full insights.
==============================================================================

47: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:analytics:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

48: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task ':solr:contrib:analytics:validateSourcePatterns'.
> Found 11 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

49: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task ':solr:contrib:clustering:validateSourcePatterns'.
> Found 3 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

50: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:dataimporthandler:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

51: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:dataimporthandler:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

52: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task 
':solr:contrib:dataimporthandler:validateSourcePatterns'.
> Found 1 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

53: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:dataimporthandler-extras:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

54: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:extraction:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

55: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:extraction:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

56: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task 
':solr:contrib:jaegertracer-configurator:validateSourcePatterns'.
> Found 1 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

57: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:ltr:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

58: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:ltr:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

59: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task ':solr:contrib:ltr:validateSourcePatterns'.
> Found 4 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

60: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:prometheus-exporter:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

61: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:prometheus-exporter:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-11.0.6\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

62: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task 
':solr:contrib:prometheus-exporter:validateSourcePatterns'.
> Found 2 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

63: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task ':solr:contrib:velocity:validateSourcePatterns'.
> Found 1 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with 
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See 
https://docs.gradle.org/6.4.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 2h 33m 59s
792 actionable tasks: 792 executed
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Archiving artifacts
[Java] Skipping execution of recorder since overall result is 'FAILURE'
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

[JENKINS] Lucene-Solr-cloud2refimpl-Windows (64bit/jdk-11.0.6) - Build # 49 - Still Failing!

Reply via email to