[JENKINS] Lucene-Solr-cloud2refimpl-Windows (64bit/jdk-12.0.2) - Build # 50 - Still Failing!

Policeman Jenkins Server Wed, 21 Oct 2020 04:39:31 -0700

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-cloud2refimpl-Windows/50/
Java: 64bit/jdk-12.0.2 -XX:-UseCompressedOops -XX:+UseG1GC


58 tests failed.
FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: 
Timeout waiting for live nodes, currently they are: [127.0.0.1:62964_solr, 
127.0.0.1:62965_solr]

Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: 
Timeout waiting for live nodes, currently they are: [127.0.0.1:62964_solr, 
127.0.0.1:62965_solr]
        at __randomizedtesting.SeedInfo.seed([DA24AF4C200B94E8]:0)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:340)
        at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.build(SolrCloudTestCase.java:251)
        at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:241)
        at 
org.apache.solr.cloud.AddReplicaTest.setupCluster(AddReplicaTest.java:60)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:882)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
        at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.util.concurrent.TimeoutException: Timeout waiting for live 
nodes, currently they are: [127.0.0.1:62964_solr, 127.0.0.1:62965_solr]
        at 
org.apache.solr.common.cloud.ZkStateReader.waitForLiveNodes(ZkStateReader.java:1825)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.waitForNode(MiniSolrCloudCluster.java:390)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.waitForAllNodes(MiniSolrCloudCluster.java:364)
        at 
org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:335)
        ... 26 more


FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
java.lang.RuntimeException: Timeout waiting for executor to shutdown

Stack Trace:
java.lang.RuntimeException: Timeout waiting for executor to shutdown
        at __randomizedtesting.SeedInfo.seed([DA24AF4C200B94E8]:0)
        at 
org.apache.solr.common.util.ExecutorUtil.awaitTermination(ExecutorUtil.java:89)
        at 
org.apache.solr.common.util.ExecutorUtil.shutdownAndAwaitTermination(ExecutorUtil.java:78)
        at 
org.apache.solr.common.ParWork.shutdownRootSharedExec(ParWork.java:104)
        at 
org.apache.solr.common.ParWork.shutdownRootSharedExec(ParWork.java:93)
        at org.apache.solr.SolrTestCase.afterSolrTestCase(SolrTestCase.java:464)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:567)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:905)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
        at java.base/java.lang.Thread.run(Thread.java:835)


FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 The process cannot access the file because it is being used by another process.

   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001


Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 java.nio.file.FileSystemException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2\log.1:
 The process cannot access the file because it is being used by another process.

   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data\version-2
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1\data
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper\server1
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001\zookeeper
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001\tempDir-001
   
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\solr\core\build\tmp\tests-tmp\solr.cloud.AddReplicaTest_DA24AF4C200B94E8-001

        at org.apache.lucene.util.IOUtils.rm(IOUtils.java:319)
        at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
        at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
        at java.base/java.lang.Thread.run(Thread.java:835)


FAILED:  org.apache.solr.cloud.AddReplicaTest.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 98 threads leaked from 
SUITE scope at org.apache.solr.cloud.AddReplicaTest: 
   1) Thread[id=451, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker398, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   2) Thread[id=474, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker421, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   3) Thread[id=469, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker416, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   4) Thread[id=446, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker393, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   5) Thread[id=416, 
name=zkConnManagerCallbackExecutor-RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker360363,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   6) Thread[id=382, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker353-acceptor-0@3b3c6364-ServerConnector@458253ba{HTTP/1.1,
 (http/1.1, h2c)}{127.0.0.1:62964}, state=RUNNABLE, group=TGRP-AddReplicaTest]
        at [email protected]/sun.nio.ch.ServerSocketChannelImpl.accept0(Native 
Method)
        at 
[email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:525)
        at 
[email protected]/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:277)
        at 
app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388)
        at 
app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   7) Thread[id=475, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker422, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   8) Thread[id=430, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker377, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
   9) Thread[id=384, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker355, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  10) Thread[id=464, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker411, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  11) Thread[id=477, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker424, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  12) Thread[id=394, name=ProcessThread(sid:0 cport:62937):, state=WAITING, 
group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:134)
  13) Thread[id=432, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker379, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  14) Thread[id=476, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker423, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  15) Thread[id=387, name=ZkTestServer Run Thread, state=WAITING, 
group=TGRP-AddReplicaTest]
        at [email protected]/java.lang.Object.wait(Native Method)
        at [email protected]/java.lang.Thread.join(Thread.java:1308)
        at [email protected]/java.lang.Thread.join(Thread.java:1375)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:950)
        at 
app//org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:361)
        at app//org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:574)
  16) Thread[id=486, 
name=zkCallbackExecutor-RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker360431,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  17) Thread[id=381, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker352, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  18) Thread[id=472, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker419, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  19) Thread[id=385, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker356, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  20) Thread[id=552, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker495, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  21) Thread[id=546, 
name=SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker-SendThread(127.0.0.1:62937),
 state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at 
app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345)
        at 
app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)
  22) Thread[id=393, name=SyncThread:0, state=WAITING, 
group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:109)
  23) Thread[id=447, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker394, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  24) Thread[id=409, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker358-SendThread(127.0.0.1:62937),
 state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at 
app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345)
        at 
app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223)
  25) Thread[id=392, name=SessionTracker, state=TIMED_WAITING, 
group=TGRP-AddReplicaTest]
        at [email protected]/java.lang.Thread.sleep(Native Method)
        at 
app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151)
  26) Thread[id=436, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker383, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  27) Thread[id=434, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker381, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  28) Thread[id=554, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker497, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  29) Thread[id=463, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker410, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  30) Thread[id=435, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker382, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  31) Thread[id=454, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker401, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  32) Thread[id=466, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker413, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  33) Thread[id=390, name=NIOServerCxnFactory.SelectorThread-1, state=RUNNABLE, 
group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426)
        at 
app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391)
  34) Thread[id=417, 
name=zkConnManagerCallbackExecutor-RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker359364,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  35) Thread[id=438, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker385, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  36) Thread[id=468, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker415, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  37) Thread[id=487, 
name=zkCallbackExecutor-RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker358432,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  38) Thread[id=414, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker359-EventThread,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
[email protected]/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at 
app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
  39) Thread[id=461, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker408, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  40) Thread[id=553, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker496, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  41) Thread[id=378, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker349, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  42) Thread[id=452, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker399, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  43) Thread[id=551, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker494, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  44) Thread[id=425, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker372, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:339)
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:167)
        at 
[email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at 
[email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at 
app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
        at 
app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
        at 
app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at 
app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at 
app//org.eclipse.jetty.io.ManagedSelector$$Lambda$263/0x0000000801624840.run(Unknown
 Source)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool.runJob(SolrQueuedThreadPool.java:677)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:846)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  45) Thread[id=399, 
name=zkConnManagerCallbackExecutor-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker357,
 state=WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.take(BlockingArrayQueue.java:342)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  46) Thread[id=408, name=Thread-9, state=TIMED_WAITING, 
group=TGRP-AddReplicaTest]
        at [email protected]/java.lang.Thread.sleep(Native Method)
        at 
app//org.apache.solr.servlet.SolrDispatchFilter$LiveThread.run(SolrDispatchFilter.java:118)
  47) Thread[id=450, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker397, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  48) Thread[id=439, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker386, 
state=TIMED_WAITING, group=TGRP-AddReplicaTest]
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at 
[email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:235)
        at 
[email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at 
app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.idleJobPoll(SolrQueuedThreadPool.java:766)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$Runner.run(SolrQueuedThreadPool.java:827)
        at 
app//org.apache.solr.common.util.SolrQueuedThreadPool$MyRunnable.run(SolrQueuedThreadPool.java:720)
        at 
[email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
[email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:230)
        at 
app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$235/0x0000000801594040.run(Unknown
 Source)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
[email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at 
app//org.apache.solr.common.ParWorkExecutor$ParWorkThreadFactory$1.run(ParWorkExecutor.java:98)
  49) Thread[id=448, 
name=RootExec-SUITE-AddReplicaTest-seed#[DA24AF4C200B94E8]-worker395, 
state=RUNNABLE, group=TGRP-AddReplicaTest]
        at 
[email protected]/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at [email protected]/sun.nio.ch.WindowsSelectorImpl$SubSel

[...truncated too long message...]

ce-patterns.gradle' line: 324

* What went wrong:
Execution failed for task ':solr:contrib:analytics:validateSourcePatterns'.
> Found 11 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

48: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task ':solr:contrib:clustering:validateSourcePatterns'.
> Found 3 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

49: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:dataimporthandler:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

50: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:dataimporthandler:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

51: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task 
':solr:contrib:dataimporthandler:validateSourcePatterns'.
> Found 1 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

52: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:dataimporthandler-extras:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

53: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:extraction:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

54: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:extraction:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

55: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task 
':solr:contrib:jaegertracer-configurator:validateSourcePatterns'.
> Found 1 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

56: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:ltr:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

57: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:ltr:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

58: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\documentation\render-javadoc.gradle'
 line: 576

* What went wrong:
Execution failed for task ':lucene:core:renderJavadoc'.
> Javadoc generation failed for :lucene:core,
    Options file at: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\lucene\core\build\tmp\renderJavadoc\javadoc-options.txt
    Command output at: 
C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\lucene\core\build\tmp\renderJavadoc\javadoc-output.txt

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

59: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task ':solr:contrib:ltr:validateSourcePatterns'.
> Found 4 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

60: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:prometheus-exporter:ecjLintMain'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

61: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:contrib:prometheus-exporter:ecjLintTest'.
> Process 'command 'C:\Users\jenkins\tools\java\64bit\jdk-12.0.2\bin\java.exe'' 
> finished with non-zero exit value -1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

62: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task 
':solr:contrib:prometheus-exporter:validateSourcePatterns'.
> Found 2 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

63: Task failed with an exception.
-----------
* Where:
Script 
'C:\Users\jenkins\workspace\Lucene-Solr-cloud2refimpl-Windows\gradle\validation\validate-source-patterns.gradle'
 line: 324

* What went wrong:
Execution failed for task ':solr:contrib:velocity:validateSourcePatterns'.
> Found 1 violations in source files (nocommit).

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with 
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See 
https://docs.gradle.org/6.4.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 2h 32m 36s
792 actionable tasks: 792 executed
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Archiving artifacts
[Java] Skipping execution of recorder since overall result is 'FAILURE'
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

[JENKINS] Lucene-Solr-cloud2refimpl-Windows (64bit/jdk-12.0.2) - Build # 50 - Still Failing!

Reply via email to