Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/950/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

5 tests failed.
FAILED:  org.apache.solr.cloud.MultiThreadedOCPTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=33904, 
name=OverseerThreadFactory-10809-thread-5, state=RUNNABLE, group=Overseer 
collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=33904, 
name=OverseerThreadFactory-10809-thread-5, state=RUNNABLE, group=Overseer 
collection creation process.]
        at 
__randomizedtesting.SeedInfo.seed([3C5AC9B5D62D852E:B40EF66F78D1E8D6]:0)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
        at __randomizedtesting.SeedInfo.seed([3C5AC9B5D62D852E]:0)
        at java.util.Collections$UnmodifiableMap.entrySet(Collections.java:1481)
        at java.util.HashMap.putMapEntries(HashMap.java:512)
        at java.util.HashMap.<init>(HashMap.java:490)
        at 
org.apache.logging.log4j.spi.DefaultThreadContextMap.getCopy(DefaultThreadContextMap.java:186)
        at 
org.apache.logging.log4j.ThreadContext.getContext(ThreadContext.java:343)
        at 
org.apache.logging.slf4j.Log4jMDCAdapter.getCopyOfContextMap(Log4jMDCAdapter.java:51)
        at org.slf4j.MDC.getCopyOfContextMap(MDC.java:248)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:169)
        at 
com.codahale.metrics.InstrumentedExecutorService.execute(InstrumentedExecutorService.java:61)
        at 
java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)
        at 
org.apache.solr.handler.component.HttpShardHandler.submit(HttpShardHandler.java:203)
        at 
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.waitForCoreAdminAsyncCallToComplete(OverseerCollectionMessageHandler.java:926)
        at 
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.waitForAsyncCallsToComplete(OverseerCollectionMessageHandler.java:906)
        at 
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:776)
        at 
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:752)
        at 
org.apache.solr.cloud.api.collections.AddReplicaCmd.lambda$addReplica$1(AddReplicaCmd.java:168)
        at 
org.apache.solr.cloud.api.collections.AddReplicaCmd$$Lambda$666/1199632155.run(Unknown
 Source)
        at 
org.apache.solr.cloud.api.collections.AddReplicaCmd.addReplica(AddReplicaCmd.java:190)
        at 
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.addReplica(OverseerCollectionMessageHandler.java:747)
        at 
org.apache.solr.cloud.api.collections.SplitShardCmd.split(SplitShardCmd.java:280)
        at 
org.apache.solr.cloud.api.collections.SplitShardCmd.call(SplitShardCmd.java:94)
        at 
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:292)
        at 
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:496)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$9/557611722.run(Unknown
 Source)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
junit.framework.TestSuite.org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest

Error Message:
Clean up static fields (in @AfterClass?), your test seems to hang on to 
approximately 10,753,088 bytes (threshold is 10,485,760). Field reference sizes 
(counted individually):   - 10,752,104 bytes, private static 
javax.management.MBeanServer 
org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer   - 296 
bytes, public static org.junit.rules.TestRule 
org.apache.solr.SolrTestCaseJ4.solrClassRules   - 152 bytes, private static 
java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels   - 144 bytes, 
private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp   - 
128 bytes, private static java.lang.String 
org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION   - 120 
bytes, protected static volatile java.util.concurrent.ExecutorService 
org.apache.solr.SolrTestCaseJ4.testExecutor   - 80 bytes, protected static 
java.lang.String org.apache.solr.SolrTestCaseJ4.coreName   - 64 bytes, private 
static java.lang.String org.apache.solr.SolrTestCaseJ4.initialRootLogLevel

Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?), 
your test seems to hang on to approximately 10,753,088 bytes (threshold is 
10,485,760). Field reference sizes (counted individually):
  - 10,752,104 bytes, private static javax.management.MBeanServer 
org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer
  - 296 bytes, public static org.junit.rules.TestRule 
org.apache.solr.SolrTestCaseJ4.solrClassRules
  - 152 bytes, private static java.util.Map 
org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
  - 144 bytes, private static java.lang.String 
org.apache.solr.SolrTestCaseJ4.factoryProp
  - 128 bytes, private static java.lang.String 
org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION
  - 120 bytes, protected static volatile java.util.concurrent.ExecutorService 
org.apache.solr.SolrTestCaseJ4.testExecutor
  - 80 bytes, protected static java.lang.String 
org.apache.solr.SolrTestCaseJ4.coreName
  - 64 bytes, private static java.lang.String 
org.apache.solr.SolrTestCaseJ4.initialRootLogLevel
        at __randomizedtesting.SeedInfo.seed([3C5AC9B5D62D852E]:0)
        at 
com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:170)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.common.cloud.SolrZkClientTest.testSimpleUpdateACLs

Error Message:
java.lang.InterruptedException: sleep interrupted

Stack Trace:
java.lang.RuntimeException: java.lang.InterruptedException: sleep interrupted
        at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:600)
        at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:524)
        at 
org.apache.solr.common.cloud.SolrZkClientTest.setUp(SolrZkClientTest.java:61)
        at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:969)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException: sleep interrupted
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:582)
        ... 39 more


FAILED:  junit.framework.TestSuite.org.apache.solr.common.cloud.SolrZkClientTest

Error Message:
5 threads leaked from SUITE scope at 
org.apache.solr.common.cloud.SolrZkClientTest:     1) Thread[id=6626, 
name=SyncThread:0, state=WAITING, group=TGRP-SolrZkClientTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
    2) Thread[id=6627, name=ProcessThread(sid:0 cport:49726):, state=WAITING, 
group=TGRP-SolrZkClientTest]         at sun.misc.Unsafe.park(Native Method)     
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)        
 at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:123)
    3) Thread[id=6625, name=SessionTracker, state=TIMED_WAITING, 
group=TGRP-SolrZkClientTest]         at java.lang.Object.wait(Native Method)    
     at 
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147) 
   4) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, 
state=RUNNABLE, group=TGRP-SolrZkClientTest]         at 
sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)         at 
sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)         at 
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=6623, 
name=ZkTestServer Run Thread, state=WAITING, group=TGRP-SolrZkClientTest]       
  at java.lang.Object.wait(Native Method)         at 
java.lang.Thread.join(Thread.java:1252)         at 
java.lang.Thread.join(Thread.java:1326)         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313)
         at 
org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:343)
         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:564)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE 
scope at org.apache.solr.common.cloud.SolrZkClientTest: 
   1) Thread[id=6626, name=SyncThread:0, state=WAITING, 
group=TGRP-SolrZkClientTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
   2) Thread[id=6627, name=ProcessThread(sid:0 cport:49726):, state=WAITING, 
group=TGRP-SolrZkClientTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:123)
   3) Thread[id=6625, name=SessionTracker, state=TIMED_WAITING, 
group=TGRP-SolrZkClientTest]
        at java.lang.Object.wait(Native Method)
        at 
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147)
   4) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, 
state=RUNNABLE, group=TGRP-SolrZkClientTest]
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=6623, name=ZkTestServer Run Thread, state=WAITING, 
group=TGRP-SolrZkClientTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Thread.join(Thread.java:1252)
        at java.lang.Thread.join(Thread.java:1326)
        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313)
        at 
org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:343)
        at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:564)
        at __randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.common.cloud.SolrZkClientTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, 
group=TGRP-SolrZkClientTest]         at 
sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)         at 
sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)         
at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)         
at 
java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)   
      at 
java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)         
at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)         
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, 
state=RUNNABLE, group=TGRP-SolrZkClientTest]
        at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)
        at 
sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
        at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
        at 
java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
        at 
java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)




Build Log:
[...truncated 14160 lines...]
   [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/init-core-data-001
   [junit4]   2> 1891595 WARN  
(SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
   [junit4]   2> 1891596 INFO  
(SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1891597 INFO  
(SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1891598 INFO  
(SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1891598 INFO  
(SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1891600 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1891601 INFO  (ZkTestServer Run Thread) [    ] 
o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1891601 INFO  (ZkTestServer Run Thread) [    ] 
o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1891703 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:48217
   [junit4]   2> 1891703 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer parse host and port list: localhost:48217
   [junit4]   2> 1891703 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer connecting to localhost 48217
   [junit4]   2> 1891710 INFO  (zkConnectionManagerCallback-11200-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1891713 INFO  (zkConnectionManagerCallback-11202-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1891718 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1891720 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1891721 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1891723 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1891724 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1891725 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1891726 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1891727 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1891728 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1891729 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1891730 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkTestServer put 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1891733 INFO  (zkConnectionManagerCallback-11206-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1891735 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly 
asked otherwise
   [junit4]   2> 1891864 WARN  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1891864 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1891864 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1891864 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] o.e.j.s.Server 
jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: 
c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
   [junit4]   2> 1891865 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] o.e.j.s.session 
DefaultSessionIdManager workerName=node0
   [junit4]   2> 1891865 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] o.e.j.s.session 
No SessionScavenger set, using defaults
   [junit4]   2> 1891865 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] o.e.j.s.session 
node0 Scavenging every 600000ms
   [junit4]   2> 1891865 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@47e5d8e2{/,null,AVAILABLE}
   [junit4]   2> 1891865 WARN  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for 
SslContextFactory@1c00cf0a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@4c00cf{SSL,[ssl, 
http/1.1]}{127.0.0.1:34629}
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] o.e.j.s.Server 
Started @1893639ms
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/control/data,
 hostContext=/, hostPort=34629, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores}
   [junit4]   2> 1891867 ERROR 
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.7.0
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1891867 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-11T10:29:00.085Z
   [junit4]   2> 1891868 INFO  (zkConnectionManagerCallback-11208-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1891870 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1891870 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/solr.xml
   [junit4]   2> 1891885 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1891885 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1891886 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1892108 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
   [junit4]   2> 1892109 INFO  (zkConnectionManagerCallback-11212-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1892112 INFO  (zkConnectionManagerCallback-11214-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1892192 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34629_
   [junit4]   2> 1892193 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.c.Overseer Overseer 
(id=72210581052588037-127.0.0.1:34629_-n_0000000000) starting
   [junit4]   2> 1892199 INFO  (zkConnectionManagerCallback-11221-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1892202 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
   [junit4]   2> 1892203 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:34629_
   [junit4]   2> 1892207 INFO  
(OverseerStateUpdate-72210581052588037-127.0.0.1:34629_-n_0000000000) 
[n:127.0.0.1:34629_    ] o.a.s.c.Overseer Starting to work on the main queue : 
127.0.0.1:34629_
   [junit4]   2> 1892208 INFO  
(OverseerStateUpdate-72210581052588037-127.0.0.1:34629_-n_0000000000) 
[n:127.0.0.1:34629_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 1892211 INFO  
(OverseerAutoScalingTriggerThread-72210581052588037-127.0.0.1:34629_-n_0000000000)
 [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: []
   [junit4]   2> 1892214 INFO  (ScheduledTrigger-10807-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 0 and last live nodes: 0
   [junit4]   2> 1892216 INFO  (zkCallback-11220-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1892223 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics 
history in memory.
   [junit4]   2> 1892236 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 
'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1892241 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1892242 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1892243 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_    
] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores
   [junit4]   2> 1892257 INFO  (zkConnectionManagerCallback-11227-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1892258 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1892260 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
   [junit4]   2> 1892274 INFO  (qtp1461441321-33724) [n:127.0.0.1:34629_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34629_&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1892278 INFO  
(OverseerThreadFactory-10809-thread-1-processing-n:127.0.0.1:34629_) 
[n:127.0.0.1:34629_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
control_collection
   [junit4]   2> 1892388 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1892391 INFO  (qtp1461441321-33725) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=2
   [junit4]   2> 1892392 INFO  (qtp1461441321-33726) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1892393 INFO  (qtp1461441321-33722) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1892394 INFO  (qtp1461441321-33721) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1892397 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    
x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core 
create command 
qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1892397 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    
x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault 
Allocating transient cache for 4 transient cores
   [junit4]   2> 1893224 INFO  (ScheduledTrigger-10807-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 0
   [junit4]   2> 1893424 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.7.0
   [junit4]   2> 1893434 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1893896 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1893919 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' 
using configuration from collection control_collection, trusted=true
   [junit4]   2> 1893920 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.core.control_collection.shard1.replica_n1' (registry 
'solr.core.control_collection.shard1.replica_n1') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1893920 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore 
at 
[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores/control_collection_shard1_replica_n1],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 1893921 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@65de66b5
   [junit4]   2> 1893923 WARN  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1893969 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1893969 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1893971 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1893971 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1893972 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=29, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=74.759765625, 
floorSegmentMB=1.9189453125, forceMergeDeletesPctAllowed=24.5937728965898, 
segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.12210359616822314, deletesPctAllowed=23.034847242467393
   [junit4]   2> 1893972 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@5d84d8e8[control_collection_shard1_replica_n1] main]
   [junit4]   2> 1893974 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 1893975 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1893975 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1893975 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1619550946924167168
   [junit4]   2> 1893979 INFO  
(searcherExecutor-10814-thread-1-processing-n:127.0.0.1:34629_ 
x:control_collection_shard1_replica_n1 c:control_collection s:shard1) 
[n:127.0.0.1:34629_ c:control_collection s:shard1  
x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore 
[control_collection_shard1_replica_n1] Registered new searcher 
Searcher@5d84d8e8[control_collection_shard1_replica_n1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1893984 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ZkShardTerms Successful update of terms at 
/collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, 
version=0}
   [junit4]   2> 1893984 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContextBase make sure parent is created 
/collections/control_collection/leaders/shard1
   [junit4]   2> 1893989 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1893989 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1893989 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SyncStrategy Sync replicas to 
https://127.0.0.1:34629/control_collection_shard1_replica_n1/
   [junit4]   2> 1893989 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1893990 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.SyncStrategy 
https://127.0.0.1:34629/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1893990 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 1893990 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/control_collection/leaders/shard1/leader after winning as 
/collections/control_collection/leader_elect/shard1/election/72210581052588037-core_node2-n_0000000000
   [junit4]   2> 1893993 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:34629/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1894095 INFO  (zkCallback-11213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged 
path:/collections/control_collection/state.json] for collection 
[control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1894096 INFO  (zkCallback-11213-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged 
path:/collections/control_collection/state.json] for collection 
[control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1894097 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1894100 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_ 
c:control_collection s:shard1  x:control_collection_shard1_replica_n1] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT}
 status=0 QTime=1704
   [junit4]   2> 1894103 INFO  (qtp1461441321-33724) [n:127.0.0.1:34629_    ] 
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
45 seconds. Check all shard replicas
   [junit4]   2> 1894201 INFO  (zkCallback-11213-thread-2) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged 
path:/collections/control_collection/state.json] for collection 
[control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1894201 INFO  (zkCallback-11213-thread-3) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged 
path:/collections/control_collection/state.json] for collection 
[control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1894201 INFO  (zkCallback-11213-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged 
path:/collections/control_collection/state.json] for collection 
[control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1894202 INFO  (qtp1461441321-33724) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34629_&wt=javabin&version=2}
 status=0 QTime=1928
   [junit4]   2> 1894208 INFO  (zkConnectionManagerCallback-11232-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1894209 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1894211 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
   [junit4]   2> 1894211 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 1894214 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1894218 INFO  
(OverseerThreadFactory-10809-thread-2-processing-n:127.0.0.1:34629_) 
[n:127.0.0.1:34629_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
collection1
   [junit4]   2> 1894218 INFO  
(OverseerCollectionConfigSetProcessor-72210581052588037-127.0.0.1:34629_-n_0000000000)
 [n:127.0.0.1:34629_    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1894227 INFO  (ScheduledTrigger-10807-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 1894424 WARN  
(OverseerThreadFactory-10809-thread-2-processing-n:127.0.0.1:34629_) 
[n:127.0.0.1:34629_    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to 
create a collection (collection1) without cores.
   [junit4]   2> 1894427 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    ] 
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
45 seconds. Check all shard replicas
   [junit4]   2> 1894427 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2}
 status=0 QTime=213
   [junit4]   2> 1894428 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances 
pullReplicaCount=0 numOtherReplicas=3
   [junit4]   2> 1894563 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001
 of type NRT
   [junit4]   2> 1894565 WARN  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1894565 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1894565 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1894565 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: 
c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
   [junit4]   2> 1894567 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1894567 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1894567 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1894567 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@7fcf9b3{/,null,AVAILABLE}
   [junit4]   2> 1894567 WARN  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for 
SslContextFactory@6a4828b3[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@2e572a0f{SSL,[ssl, 
http/1.1]}{127.0.0.1:56235}
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.e.j.s.Server Started @1896343ms
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/jetty1,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=56235, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001/cores}
   [junit4]   2> 1894569 ERROR (closeThreadPool-11233-thread-1) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.7.0
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1894569 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-11T10:29:02.787Z
   [junit4]   2> 1894576 INFO  (zkConnectionManagerCallback-11235-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1894577 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1894577 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001/solr.xml
   [junit4]   2> 1894580 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1894580 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1894581 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1894735 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001
 of type NRT
   [junit4]   2> 1894737 WARN  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1894737 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1894737 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1894737 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: 
c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
   [junit4]   2> 1894738 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1894738 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1894738 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1894738 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@14b9900a{/,null,AVAILABLE}
   [junit4]   2> 1894739 WARN  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for 
SslContextFactory@47bbb4bc[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@6356379d{SSL,[ssl, 
http/1.1]}{127.0.0.1:43090}
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.e.j.s.Server Started @1896515ms
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/jetty2,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=43090, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001/cores}
   [junit4]   2> 1894741 ERROR (closeThreadPool-11233-thread-2) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.7.0
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1894741 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-11T10:29:02.959Z
   [junit4]   2> 1894743 INFO  (zkConnectionManagerCallback-11237-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1894744 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1894744 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001/solr.xml
   [junit4]   2> 1894758 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1894758 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1894759 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1894887 INFO  
(TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001
 of type NRT
   [junit4]   2> 1894888 WARN  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1894889 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1894889 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1894889 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: 
c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
   [junit4]   2> 1894889 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1894889 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1894889 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1894890 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@487abad0{/,null,AVAILABLE}
   [junit4]   2> 1894890 WARN  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for 
SslContextFactory@49eb70fa[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1894902 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@4783da7{SSL,[ssl, 
http/1.1]}{127.0.0.1:36150}
   [junit4]   2> 1894902 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.e.j.s.Server Started @1896676ms
   [junit4]   2> 1894902 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/jetty3,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=36150, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001/cores}
   [junit4]   2> 1894902 ERROR (closeThreadPool-11233-thread-3) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1894902 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1894902 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.7.0
   [junit4]   2> 1894903 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1894907 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1894907 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-12-11T10:29:03.125Z
   [junit4]   2> 1894909 INFO  (zkConnectionManagerCallback-11239-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1894910 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1894910 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001/solr.xml
   [junit4]   2> 1894913 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay 
is ignored
   [junit4]   2> 1894913 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.SolrXmlConfig Configuration parameter 
autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1894914 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 1895227 INFO  (ScheduledTrigger-10807-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1 and last live nodes: 1
   [junit4]   2> 1895243 INFO  (closeThreadPool-11233-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
   [junit4]   2> 1895245 INFO  (zkConnectionManagerCallback-11243-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895247 INFO  (zkConnectionManagerCallback-11245-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895251 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 1895253 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.c.ZkController Publish node=127.0.0.1:56235_ as 
DOWN
   [junit4]   2> 1895254 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating 
transient cache for 4 transient cores
   [junit4]   2> 1895254 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:56235_
   [junit4]   2> 1895255 INFO  (zkCallback-11213-thread-3) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1895256 INFO  (zkCallback-11220-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1895256 INFO  (zkCallback-11231-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1895256 INFO  (zkCallback-11244-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1895272 INFO  (zkConnectionManagerCallback-11252-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895273 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 1895274 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 
localhost:48217/solr ready
   [junit4]   2> 1895274 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, 
keeping metrics history in memory.
   [junit4]   2> 1895287 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895292 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895292 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895293 INFO  (closeThreadPool-11233-thread-1) 
[n:127.0.0.1:56235_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions 
underneath 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001/cores
   [junit4]   2> 1895524 INFO  (closeThreadPool-11233-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
   [junit4]   2> 1895526 INFO  (zkConnectionManagerCallback-11257-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895528 INFO  (zkConnectionManagerCallback-11259-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895532 INFO  (closeThreadPool-11233-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
   [junit4]   2> 1895536 INFO  (zkConnectionManagerCallback-11265-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895538 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 1895539 INFO  (zkConnectionManagerCallback-11267-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895542 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.c.ZkController Publish node=127.0.0.1:36150_ as 
DOWN
   [junit4]   2> 1895544 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating 
transient cache for 4 transient cores
   [junit4]   2> 1895544 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36150_
   [junit4]   2> 1895545 INFO  (zkCallback-11213-thread-3) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1895545 INFO  (zkCallback-11231-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1895546 INFO  (zkCallback-11220-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1895546 INFO  (zkCallback-11244-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1895546 INFO  (zkCallback-11251-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1895546 INFO  (zkCallback-11258-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1895547 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 1895551 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.c.ZkController Publish node=127.0.0.1:43090_ as 
DOWN
   [junit4]   2> 1895552 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating 
transient cache for 4 transient cores
   [junit4]   2> 1895553 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:43090_
   [junit4]   2> 1895554 INFO  (zkCallback-11213-thread-3) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1895554 INFO  (zkCallback-11244-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1895554 INFO  (zkCallback-11231-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1895554 INFO  (zkCallback-11220-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1895554 INFO  (zkCallback-11251-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1895555 INFO  (zkCallback-11258-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1895556 INFO  (zkCallback-11266-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1895568 INFO  (zkConnectionManagerCallback-11274-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895569 INFO  (zkConnectionManagerCallback-11279-thread-1) [   
 ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1895570 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (4)
   [junit4]   2> 1895570 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (4)
   [junit4]   2> 1895572 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 
localhost:48217/solr ready
   [junit4]   2> 1895572 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 
localhost:48217/solr ready
   [junit4]   2> 1895572 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, 
keeping metrics history in memory.
   [junit4]   2> 1895572 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, 
keeping metrics history in memory.
   [junit4]   2> 1895585 WARN  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.m.r.j.JmxMetricsReporter Unable to register gauge
   [junit4]   2> javax.management.InstanceNotFoundException: 
solr:dom1=node,category=QUERY,scope=httpShardHandler,name=leasedConnections
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
 ~[?:1.8.0_191]
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)
 ~[?:1.8.0_191]
   [junit4]   2>        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
 ~[?:1.8.0_191]
   [junit4]   2>        at 
com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) 
~[?:1.8.0_191]
   [junit4]   2>        at 
org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:531)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onGaugeAdded(JmxMetricsReporter.java:569)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.lambda$start$0(JmxMetricsReporter.java:742)
 ~[java/:?]
   [junit4]   2>        at java.util.HashMap.forEach(HashMap.java:1289) 
~[?:1.8.0_191]
   [junit4]   2>        at 
org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.start(JmxMetricsReporter.java:732)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.metrics.reporters.SolrJmxReporter.doInit(SolrJmxReporter.java:110)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.metrics.SolrMetricReporter.init(SolrMetricReporter.java:70) 
~[java/:?]
   [junit4]   2>        at 
org.apache.solr.metrics.SolrMetricManager.loadReporter(SolrMetricManager.java:941)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.metrics.SolrMetricManager.loadReporters(SolrMetricManager.java:868)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.load(CoreContainer.java:594) ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) 
~[java/:?]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:136) 
~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
 ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) 
~[?:1.8.0_191]
   [junit4]   2>        at 
java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) 
~[?:1.8.0_191]
   [junit4]   2>        at 
java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) 
~[?:1.8.0_191]
   [junit4]   2>        at 
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) 
~[?:1.8.0_191]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) 
~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1449)
 ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1513)
 ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1158)
 ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:995)
 ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:467)
 ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:347)
 ~[java/:?]
   [junit4]   2>        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
 ~[jetty-util-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
 ~[jetty-util-9.4.14.v20181114.jar:9.4.14.v20181114]
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:510)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:449)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:417)
 ~[java/:?]
   [junit4]   2>        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.lambda$createJettys$2(AbstractFullDistribZkTestBase.java:464)
 ~[java/:?]
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[?:1.8.0_191]
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_191]
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
 ~[java/:?]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
[?:1.8.0_191]
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
[?:1.8.0_191]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
   [junit4]   2> 1895588 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895589 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895594 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895594 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895595 INFO  (closeThreadPool-11233-thread-3) 
[n:127.0.0.1:36150_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions 
underneath 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001/cores
   [junit4]   2> 1895595 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895595 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
   [junit4]   2> 1895596 INFO  (closeThreadPool-11233-thread-2) 
[n:127.0.0.1:43090_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions 
underneath 
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001/cores
   [junit4]   2> 1895652 INFO  (qtp1461441321-33724) [n:127.0.0.1:34629_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params 
node=127.0.0.1:56235_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1895657 INFO  (qtp1461441321-33726) [n:127.0.0.1:34629_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params 
node=127.0.0.1:43090_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1895657 INFO  (qtp1461441321-33725) [n:127.0.0.1:34629_    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params 
node=127.0.0.1:36150_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1895657 INFO  
(OverseerCollectionConfigSetProcessor-72210581052588037-127.0.0.1:34629_-n_0000000000)
 [n:127.0.0.1:34629_    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1895661 INFO  (qtp1461441321-33722) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes}
 status=0 QTime=1
   [junit4]   2> 1895661 INFO  (qtp1461441321-33721) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes}
 status=0 QTime=0
   [junit4]   2> 1895662 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes}
 status=0 QTime=0
   [junit4]   2> 1895663 INFO  (qtp1461441321-33722) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895663 INFO  (qtp1461441321-33721) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895664 INFO  (qtp1461441321-33723) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895665 INFO  (qtp1461441321-33722) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895666 INFO  (qtp1461441321-33721) [n:127.0.0.1:34629_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895682 INFO  (qtp1322148053-33808) [n:127.0.0.1:36150_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895683 INFO  (qtp1322148053-33816) [n:127.0.0.1:36150_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895684 INFO  (qtp1322148053-33813) [n:127.0.0.1:36150_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895685 INFO  (qtp1322148053-33811) [n:127.0.0.1:36150_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895694 INFO  (qtp1322148053-33809) [n:127.0.0.1:36150_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895712 INFO  (qtp2109900352-33796) [n:127.0.0.1:43090_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895713 INFO  (qtp2109900352-33797) [n:127.0.0.1:43090_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895714 INFO  (qtp2109900352-33799) [n:127.0.0.1:43090_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895714 INFO  (qtp2109900352-33798) [n:127.0.0.1:43090_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895715 INFO  (qtp2109900352-33800) [n:127.0.0.1:43090_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895732 INFO  (qtp709852480-33786) [n:127.0.0.1:56235_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895733 INFO  (qtp709852480-33787) [n:127.0.0.1:56235_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895734 INFO  (qtp709852480-33788) [n:127.0.0.1:56235_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895735 INFO  (qtp709852480-33783) [n:127.0.0.1:56235_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895735 INFO  (qtp709852480-33784) [n:127.0.0.1:56235_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 1895737 INFO  
(OverseerThreadFactory-10809-thread-3-processing-n:127.0.0.1:34629_) 
[n:127.0.0.1:34629_ c:collection1 s:shard2  ] o.a.s.c.a.c.AddReplicaCmd Node 
Identified 127.0.0.1:56235_ for creating new replica of shard shard2 for 
collection collection1
   [junit4]   2> 1895738 INFO  
(OverseerThreadFactory-10809-thread-4-processing-n:127.0.0.1:34629_) 
[n:127.0.0.1:34629_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node 
Identified 127.0.0.1:43090_ for creating new replica of shard shard1 for 
collection collection1
   [junit4]   2> 1895740 INFO  
(OverseerThreadFactory-10809-thread-3-processing-n:127.0.0.1:34629_) 
[n:127.0.0.1:34629_ c:collection1 s:shard2  ] o.a.s.c.a.c.AddReplicaCmd 
Returning CreateReplica command.
   [junit4]   2> 1895741 INFO  
(OverseerThreadFactory-10809-thread-4-processing-n:127.0.0.1:34629_) 
[n:127.0.0.1:34629_ c:collection1 s:shard1  ] o.a.s.c.a.c.AddReplicaCmd 
Returning CreateReplica command.
   [junit4]   2> 1895746 INFO  (qtp2109900352-33796) [n:127.0.0.1:43090_    
x:collection1_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create 
command 
qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n2&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1895746 INFO  (qtp709852480-33787) [n:127.0.0.1:56235_    
x:collection1_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create 
command 
qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n1&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1896233 INFO  (ScheduledTrigger-10807-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 4 and last live nodes: 1
   [junit4]   2> 1896764 INFO  (qtp2109900352-33796) [n:127.0.0.1:43090_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n2] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.7.0
   [junit4]   2> 1896765 INFO  (qtp709852480-33787) [n:127.0.0.1:56235_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.7.0
   [junit4]   2> 1896782 INFO  (qtp2109900352-33796) [n:127.0.0.1:43090_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema 
[collection1_shard1_replica_n2] Schema name=test
   [junit4]   2> 1896782 INFO  (qtp709852480-33787) [n:127.0.0.1:56235_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema 
[collection1_shard2_replica_n1] Schema name=test
   [junit4]   2> 1896907 INFO  (qtp2109900352-33796) [n:127.0.0.1:43090_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema 
Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1896908 INFO  (qtp709852480-33787) [n:127.0.0.1:56235_ 
c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema 
Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1896930 INFO  (qtp2109900352-33796) [n:127.0.0.1:43090_ 
c:collection1 s:shard1  x:collection1_shard1_replica_n2] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1_shard1_replica_n2' using configuration from 
collection collection1, trusted=tru

[...truncated too long message...]

nit4]   1> java.lang.InterruptedException: null
   [junit4]   1>        at java.lang.Object.wait(Native Method) ~[?:1.8.0_191]
   [junit4]   1>        at 
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147) 
[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   2> Dec 11, 2018 10:57:05 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be 
terminated:
   [junit4]   2>    1) Thread[id=6624, 
name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, 
group=TGRP-SolrZkClientTest]
   [junit4]   2>         at sun.nio.ch.DevPollArrayWrapper.interrupt(Native 
Method)
   [junit4]   2>         at 
sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
   [junit4]   2>         at 
sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
   [junit4]   2>         at 
java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
   [junit4]   2>         at 
java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
   [junit4]   2>         at 
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
   [junit4]   2>         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2>         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, 
docValues:{}, maxPointsInLeafNode=1186, maxMBSortInHeap=5.053221858295194, 
sim=RandomSimilarity(queryNorm=false): {}, locale=en-MT, timezone=Europe/Paris
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_191 
(64-bit)/cpus=3,threads=2,free=291442256,total=503316480
   [junit4]   2> NOTE: All tests run in this JVM: [SolrQueryTest, 
AnlysisResponseBaseTest, ExclusiveOrEvaluatorTest, TestValidatingJsonMap, 
MultiplyEvaluatorTest, StreamingTest, ModifiableSolrParamsTest, 
CloudSolrClientCacheTest, TestSolrJErrorHandling, TestCollectionStateWatchers, 
TestSuggesterResponse, SolrParamTest, RangeFacetMapTest, GraphTest, 
SubtractEvaluatorTest, TestCollectionAdminRequest, HttpClientUtilTest, 
StreamExpressionToExpessionTest, TestDelegationTokenResponse, 
SolrExampleJettyTest, JsonQueryRequestUnitTest, TestPathTrie, 
NormalDistributionEvaluatorTest, LessThanEvaluatorTest, 
CloudSolrClientRetryTest, TestLang, TestSolrJsonWriter, ReverseEvaluatorTest, 
TestV1toV2ApiMapper, CloudSolrClientBadInputTest, 
DirectJsonQueryRequestFacetingIntegrationTest, ArrayEvaluatorTest, 
CumulativeProbabilityEvaluatorTest, UuidEvaluatorTest, SchemaTest, 
SolrExampleStreamingTest, ConcurrentUpdateSolrClientBadInputTest, 
CosineEvaluatorTest, NoOpResponseParserTest, TestJsonRecordReader, 
HttpSolrClientSSLAuthConPoolTest, SolrExceptionTest, TestBatchUpdate, 
TestLBHttpSolrClient, ConcurrentUpdateSolrClientMultiCollectionTest, 
HttpSolrClientBuilderTest, HttpSolrClientConPoolTest, JDBCStreamTest, 
MathExpressionTest, StreamExpressionTest, AddEvaluatorTest, AndEvaluatorTest, 
AppendEvaluatorTest, ArcCosineEvaluatorTest, ArcSineEvaluatorTest, 
ArcTangentEvaluatorTest, CeilingEvaluatorTest, CoalesceEvaluatorTest, 
EmpiricalDistributionEvaluatorTest, EqualToEvaluatorTest, FloorEvaluatorTest, 
GreaterThanEqualToEvaluatorTest, GreaterThanEvaluatorTest, 
HyperbolicCosineEvaluatorTest, HyperbolicSineEvaluatorTest, 
HyperbolicTangentEvaluatorTest, LengthEvaluatorTest, 
LessThanEqualToEvaluatorTest, NotEvaluatorTest, OrEvaluatorTest, 
PowerEvaluatorTest, RawValueEvaluatorTest, RecursiveEvaluatorTest, 
RegressionEvaluatorTest, SineEvaluatorTest, SquareRootEvaluatorTest, 
UniformDistributionEvaluatorTest, ConcatOperationTest, OperationsTest, 
TestCoreAdmin, FacetFieldTest, FieldAnalysisResponseTest, TermsResponseTest, 
TestClusteringResponse, SolrDocumentTest, TestToleratedUpdateError, 
SolrZkClientTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrZkClientTest 
-Dtests.seed=C23229FC9129A0C6 -Dtests.slow=true -Dtests.locale=en-MT 
-Dtests.timezone=Europe/Paris -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | SolrZkClientTest (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE 
scope at org.apache.solr.common.cloud.SolrZkClientTest: 
   [junit4]    >    1) Thread[id=6626, name=SyncThread:0, state=WAITING, 
group=TGRP-SolrZkClientTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
   [junit4]    >    2) Thread[id=6627, name=ProcessThread(sid:0 cport:49726):, 
state=WAITING, group=TGRP-SolrZkClientTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:123)
   [junit4]    >    3) Thread[id=6625, name=SessionTracker, 
state=TIMED_WAITING, group=TGRP-SolrZkClientTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147)
   [junit4]    >    4) Thread[id=6624, 
name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, 
group=TGRP-SolrZkClientTest]
   [junit4]    >         at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
   [junit4]    >         at 
sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
   [junit4]    >         at 
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
   [junit4]    >         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    5) Thread[id=6623, name=ZkTestServer Run Thread, 
state=WAITING, group=TGRP-SolrZkClientTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Thread.join(Thread.java:1252)
   [junit4]    >         at java.lang.Thread.join(Thread.java:1326)
   [junit4]    >         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313)
   [junit4]    >         at 
org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:343)
   [junit4]    >         at 
org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:564)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=6624, 
name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, 
group=TGRP-SolrZkClientTest]
   [junit4]    >         at sun.nio.ch.DevPollArrayWrapper.interrupt(Native 
Method)
   [junit4]    >         at 
sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
   [junit4]    >         at 
sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
   [junit4]    >         at 
java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
   [junit4]    >         at 
java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
   [junit4]    >         at 
sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
   [junit4]    >         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)
   [junit4] Completed [174/174 (1!)] on J0 in 6.36s, 2 tests, 3 errors <<< 
FAILURES!

[...truncated 7874 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/build.xml:633: The 
following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/build.xml:585: Some of 
the tests produced a heap dump, but did not fail. Maybe a suppressed 
OutOfMemoryError? Dumps created:
* java_pid24294.hprof

Total time: 86 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to