Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.0/43/

6 tests failed.
FAILED:  org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([36541951E3BFD641]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([36541951E3BFD641]:0)


FAILED:  org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test

Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth 
investigating!

Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties 
were stopped - this is worth investigating!
        at 
__randomizedtesting.SeedInfo.seed([45527DBA3A5217C7:CD06426094AE7A3F]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
        at 
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest

Error Message:
8 threads leaked from SUITE scope at 
org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:     1) 
Thread[id=54112, name=StoppableIndexingThread-EventThread, state=WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)    
2) Thread[id=54056, name=StoppableIndexingThread, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
    3) Thread[id=54218, name=zkCallback-3929-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    4) Thread[id=54057, 
name=StoppableIndexingThread, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
    5) Thread[id=54219, name=zkCallback-3929-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    6) Thread[id=54111, 
name=StoppableIndexingThread-SendThread(127.0.0.1:44864), state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)    7) 
Thread[id=54058, name=StoppableCommitThread, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)  
  8) Thread[id=54113, name=zkCallback-3929-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE 
scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 
   1) Thread[id=54112, name=StoppableIndexingThread-EventThread, state=WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
   2) Thread[id=54056, name=StoppableIndexingThread, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
   3) Thread[id=54218, name=zkCallback-3929-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=54057, name=StoppableIndexingThread, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
   5) Thread[id=54219, name=zkCallback-3929-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=54111, 
name=StoppableIndexingThread-SendThread(127.0.0.1:44864), state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
   7) Thread[id=54058, name=StoppableCommitThread, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)
   8) Thread[id=54113, name=zkCallback-3929-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([45527DBA3A5217C7]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=54056, name=StoppableIndexingThread, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at 
org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)   
      at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at 
org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)   
      at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at 
org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)     
    at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
         at 
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
         at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)         
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)       
  at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
         at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
    2) Thread[id=54218, name=zkCallback-3929-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    3) Thread[id=54057, 
name=StoppableIndexingThread, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at 
org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at 
org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)   
      at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at 
org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)   
      at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at 
org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)  
       at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)        
 at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)     
    at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
         at 
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
         at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)         
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)       
  at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
         at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
    4) Thread[id=54219, name=zkCallback-3929-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)    5) Thread[id=54111, 
name=StoppableIndexingThread-SendThread(127.0.0.1:44864), state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)    6) 
Thread[id=54113, name=zkCallback-3929-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=54056, name=StoppableIndexingThread, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
        at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
        at 
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
        at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
        at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
        at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
   2) Thread[id=54218, name=zkCallback-3929-thread-2, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=54057, name=StoppableIndexingThread, state=RUNNABLE, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
        at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
        at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
        at 
org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
        at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
        at 
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
        at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
        at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
        at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
        at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
   4) Thread[id=54219, name=zkCallback-3929-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=54111, 
name=StoppableIndexingThread-SendThread(127.0.0.1:44864), state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
   6) Thread[id=54113, name=zkCallback-3929-thread-1, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([45527DBA3A5217C7]:0)


FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1505272155843,generation=2,filelist=[_5.cfe, _5.cfs, 
_5.si, _6.cfe, _6.cfs, _6.si, _7.cfe, _7.cfs, _7.si, segments_2]}]> but 
was:<[{indexVersion=1505272155843,generation=2,filelist=[_5.cfe, _5.cfs, _5.si, 
_6.cfe, _6.cfs, _6.si, _7.cfe, _7.cfs, _7.si, segments_2]}, 
{indexVersion=1505272155843,generation=3,filelist=[_8.cfe, _8.cfs, _8.si, 
segments_3]}]>

Stack Trace:
java.lang.AssertionError: 
expected:<[{indexVersion=1505272155843,generation=2,filelist=[_5.cfe, _5.cfs, 
_5.si, _6.cfe, _6.cfs, _6.si, _7.cfe, _7.cfs, _7.si, segments_2]}]> but 
was:<[{indexVersion=1505272155843,generation=2,filelist=[_5.cfe, _5.cfs, _5.si, 
_6.cfe, _6.cfs, _6.si, _7.cfe, _7.cfs, _7.si, segments_2]}, 
{indexVersion=1505272155843,generation=3,filelist=[_8.cfe, _8.cfs, _8.si, 
segments_3]}]>
        at 
__randomizedtesting.SeedInfo.seed([45527DBA3A5217C7:6085668A4A1A19C4]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:147)
        at 
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1277)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 10241 lines...]
   [junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
   [junit4]   2> сеп 13, 2017 3:18:04 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: 
org.apache.lucene.spatial3d.TestGeo3DPoint
   [junit4]   2>    1) Thread[id=9, name=JUnit4-serializer-daemon, 
state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at 
com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
   [junit4]   2>    2) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1252)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1326)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
   [junit4]   2>         at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
   [junit4]   2>         at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
   [junit4]   2>         at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
   [junit4]   2>    3) Thread[id=12, 
name=TEST-TestGeo3DPoint.testRandomBig-seed#[36541951E3BFD641], state=RUNNABLE, 
group=TGRP-TestGeo3DPoint]
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.GeoComplexPolygon.isWithin(GeoComplexPolygon.java:326)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.GeoBaseMembershipShape.isWithin(GeoBaseMembershipShape.java:36)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.GeoBaseShape.getBounds(GeoBaseShape.java:49)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.GeoComplexPolygon.getBounds(GeoComplexPolygon.java:375)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2>         at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>    4) Thread[id=11, 
name=SUITE-TestGeo3DPoint-seed#[36541951E3BFD641], state=RUNNABLE, 
group=TGRP-TestGeo3DPoint]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1559)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native 
Method)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
-Dtests.method=testRandomBig -Dtests.seed=36541951E3BFD641 -Dtests.multiplier=2 
-Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/test-data/enwiki.random.lines.txt
 -Dtests.locale=sr-ME -Dtests.timezone=Africa/Tripoli -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   6783s J2 | TestGeo3DPoint.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because 
suite timeout was reached.
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([36541951E3BFD641]:0)
   [junit4]   2> сеп 13, 2017 3:18:05 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> сеп 13, 2017 3:18:25 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at 
org.apache.lucene.spatial3d.TestGeo3DPoint: 
   [junit4]   2>    1) Thread[id=12, 
name=TEST-TestGeo3DPoint.testRandomBig-seed#[36541951E3BFD641], state=RUNNABLE, 
group=TGRP-TestGeo3DPoint]
   [junit4]   2>         at 
org.apache.lucene.store.MockIndexInputWrapper.readShort(MockIndexInputWrapper.java:159)
   [junit4]   2>         at 
org.apache.lucene.store.IndexInput$1.readShort(IndexInput.java:138)
   [junit4]   2>         at 
org.apache.lucene.util.packed.DirectReader$DirectPackedReader16.get(DirectReader.java:185)
   [junit4]   2>         at 
org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$2.longValue(Lucene70DocValuesProducer.java:471)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:840)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2>         at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> сеп 13, 2017 3:18:25 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=12, 
name=TEST-TestGeo3DPoint.testRandomBig-seed#[36541951E3BFD641], state=RUNNABLE, 
group=TGRP-TestGeo3DPoint]
   [junit4]   2> сеп 13, 2017 3:18:28 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be 
terminated:
   [junit4]   2>    1) Thread[id=12, 
name=TEST-TestGeo3DPoint.testRandomBig-seed#[36541951E3BFD641], state=RUNNABLE, 
group=TGRP-TestGeo3DPoint]
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.Plane.findIntersectionBounds(Plane.java:1097)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1214)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.XYZBounds.addIntersection(XYZBounds.java:262)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.GeoConcavePolygon.getBounds(GeoConcavePolygon.java:439)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.geom.GeoCompositeMembershipShape.getBounds(GeoCompositeMembershipShape.java:75)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
   [junit4]   2>         at 
org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2>         at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{id=PostingsFormat(name=Memory)}, docValues:{id=DocValuesFormat(name=Lucene70), 
point=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=271, 
maxMBSortInHeap=5.640720867930634, sim=RandomSimilarity(queryNorm=true): {}, 
locale=sr-ME, timezone=Africa/Tripoli
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 
1.8.0_144 (64-bit)/cpus=4,threads=2,free=242358640,total=475529216
   [junit4]   2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGeo3DPoint 
-Dtests.seed=36541951E3BFD641 -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/test-data/enwiki.random.lines.txt
 -Dtests.locale=sr-ME -Dtests.timezone=Africa/Tripoli -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | TestGeo3DPoint (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 
7200000 msec).
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([36541951E3BFD641]:0)
   [junit4] Completed [11/11 (1!)] on J2 in 7224.55s, 11 tests, 2 errors <<< 
FAILURES!

[...truncated 2806 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/init-core-data-001
   [junit4]   2> 5762760 WARN  
(SUITE-TestReplicationHandler-seed#[45527DBA3A5217C7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=74 numCloses=74
   [junit4]   2> 5762760 INFO  
(SUITE-TestReplicationHandler-seed#[45527DBA3A5217C7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 5762761 INFO  
(SUITE-TestReplicationHandler-seed#[45527DBA3A5217C7]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 5762762 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting doTestStopPoll
   [junit4]   2> 5762762 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/collection1
   [junit4]   2> 5762781 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 5762812 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@63e1d34f{/solr,null,AVAILABLE}
   [junit4]   2> 5762812 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@1e175c07{HTTP/1.1,[http/1.1]}{127.0.0.1:44514}
   [junit4]   2> 5762824 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.Server Started @5778059ms
   [junit4]   2> 5762824 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/collection1/data,
 hostContext=/solr, hostPort=44514}
   [junit4]   2> 5762825 ERROR 
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 5762825 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.0.0
   [junit4]   2> 5762825 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on 
port null
   [junit4]   2> 5762825 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config 
dir: null
   [junit4]   2> 5762825 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-09-13T03:03:55.458Z
   [junit4]   2> 5762825 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/solr.xml
   [junit4]   2> 5762855 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 5763053 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5763095 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5763095 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5763133 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/.
   [junit4]   2> 5763133 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 5763330 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 5763367 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 5763369 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 5763369 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using 
configuration from instancedir 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/./collection1,
 trusted=true
   [junit4]   2> 5763385 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5763386 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 5763386 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/collection1],
 
dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 5763387 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=32, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=43.126953125, 
floorSegmentMB=0.7705078125, forceMergeDeletesPctAllowed=24.60716279491469, 
segmentsPerTier=3.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.28720781238189314
   [junit4]   2> 5763533 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 5763533 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 5763534 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@7ca74773
   [junit4]   2> 5763534 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.s.SolrIndexSearcher Opening 
[Searcher@6d3eb122[collection1] main]
   [junit4]   2> 5763535 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to 
use dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-001/collection1/conf
   [junit4]   2> 5763535 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following 
config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 5763535 INFO  (coreLoadExecutor-7442-thread-1) [    
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 5763549 INFO  
(searcherExecutor-7443-thread-1-processing-x:collection1) [    x:collection1] 
o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@6d3eb122[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 5763794 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/collection1
   [junit4]   2> 5763809 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1fdfbd77{/solr,null,AVAILABLE}
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@12e039cf{HTTP/1.1,[http/1.1]}{127.0.0.1:35161}
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.e.j.s.Server Started @5779044ms
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/collection1/data,
 hostContext=/solr, hostPort=35161}
   [junit4]   2> 5763810 ERROR 
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.0.0
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on 
port null
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config 
dir: null
   [junit4]   2> 5763810 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-09-13T03:03:56.443Z
   [junit4]   2> 5763811 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/solr.xml
   [junit4]   2> 5763936 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.SolrXmlConfig MBean server found: 
com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60, but no JMX reporters were 
configured - adding default JMX reporter.
   [junit4]   2> 5764125 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5764181 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') 
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5764181 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5764195 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/.
   [junit4]   2> 5764195 INFO  
(TEST-TestReplicationHandler.doTestStopPoll-seed#[45527DBA3A5217C7]) [    ] 
o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 5764300 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 5764359 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 5764361 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 5764361 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using 
configuration from instancedir 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/./collection1,
 trusted=true
   [junit4]   2> 5764373 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@68dbcd60
   [junit4]   2> 5764373 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 5764373 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/collection1],
 
dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 5764375 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=32, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=43.126953125, 
floorSegmentMB=0.7705078125, forceMergeDeletesPctAllowed=24.60716279491469, 
segmentsPerTier=3.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.28720781238189314
   [junit4]   2> 5764612 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 5764613 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 5764614 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@239466b1
   [junit4]   2> 5764614 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.s.SolrIndexSearcher Opening 
[Searcher@7f134e15[collection1] main]
   [junit4]   2> 5764614 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to 
use dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_45527DBA3A5217C7-001/solr-instance-002/collection1/conf
   [junit4]   2> 5764629 INFO  (coreLoadExecutor-7452-thread-1) [    
x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 
1000ms
   [junit4]   2> 5764650 INFO  
(searcherExecutor-7453-thread-1-processing-x:collection1) [    x:collection1] 
o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@7f134e15[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 5764703 INFO  (qtp1176835690-165123) [    x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 
QTime=0
   [junit4]   2> 5764703 INFO  (indexFetcher-7457-thread-1) [    x:collection1] 
o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 5764703 INFO  (indexFetcher-7457-thread-1) [    x:collection1] 
o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 5764703 INFO  (indexFetcher-7457-thread-1) [    x:collection1] 
o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 5764703 INFO  (indexFetcher-7457-thread-1) [    x:collection1] 
o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 5764983 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/solr path=/select 
params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 5764985 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[0]} 0 1
   [junit4]   2> 5764985 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 5764985 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 5764986 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 5765002 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[4]} 0 16
   [junit4]   2> 5765003 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 5765003 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 5765003 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 5765003 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 5765004 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 5765004 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 5765004 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 5765004 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 5765005 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 5765005 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 5765005 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 5765005 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 5765006 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 5765006 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 5765006 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 5765006 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 5765020 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 5765035 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 5765036 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 5765036 INFO  (qtp1176835690-165127) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 5765036 INFO  (qtp1176835690-165120) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 5765037 INFO  (qtp1176835690-165124) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 5765051 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 5765052 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 5765052 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 5765052 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 5765052 INFO  (qtp1176835690-165120) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 5765053 INFO  (qtp1176835690-165124) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 5765053 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 5765053 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 5765053 INFO  (qtp1176835690-165127) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 5765054 INFO  (qtp1176835690-165127) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 5765054 INFO  (qtp1176835690-165127) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 5765054 INFO  (qtp1176835690-165127) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 5765054 INFO  (qtp1176835690-165121) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 5765055 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 5765055 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 5765055 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 5765055 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 5765056 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 5765056 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 5765056 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 5765057 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 5765057 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 5765057 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 5765057 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 5765058 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 5765058 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 5765058 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 5765058 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 5765059 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 5765059 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 5765076 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 5765076 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 5765076 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 5765076 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 5765077 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[61]} 0 0
   [junit4]   2> 5765077 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 5765077 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 5765077 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 5765078 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 5765078 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 5765078 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 5765078 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 5765079 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 5765079 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 5765079 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 5765079 INFO  (qtp1176835690-165125) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 5765092 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 5765092 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 5765093 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 5765093 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 5765093 INFO  (qtp1176835690-165126) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  w

[...truncated too long message...]

)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([45527DBA3A5217C7]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=54056, name=StoppableIndexingThread, 
state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
   [junit4]    >         at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
   [junit4]    >         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
   [junit4]    >         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
   [junit4]    >         at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]    >         at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
   [junit4]    >         at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
   [junit4]    >         at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
   [junit4]    >    2) Thread[id=54218, name=zkCallback-3929-thread-2, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    3) Thread[id=54057, name=StoppableIndexingThread, 
state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
   [junit4]    >         at 
org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
   [junit4]    >         at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
   [junit4]    >         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
   [junit4]    >         at 
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
   [junit4]    >         at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
   [junit4]    >         at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]    >         at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
   [junit4]    >         at 
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
   [junit4]    >         at 
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
   [junit4]    >    4) Thread[id=54219, name=zkCallback-3929-thread-3, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    5) Thread[id=54111, 
name=StoppableIndexingThread-SendThread(127.0.0.1:44864), state=TIMED_WAITING, 
group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
   [junit4]    >    6) Thread[id=54113, name=zkCallback-3929-thread-1, 
state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([45527DBA3A5217C7]:0)
   [junit4] Completed [504/729 (2!)] on J2 in 174.73s, 1 test, 1 failure, 2 
errors <<< FAILURES!

[...truncated 43981 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to