Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1645/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCloudDeleteByQuery

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.TestCloudDeleteByQuery:     1) Thread[id=30399, 
name=qtp477159732-30399, state=TIMED_WAITING, 
group=TGRP-TestCloudDeleteByQuery]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.TestCloudDeleteByQuery: 
   1) Thread[id=30399, name=qtp477159732-30399, state=TIMED_WAITING, 
group=TGRP-TestCloudDeleteByQuery]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([5F5FC887741571]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCloudDeleteByQuery

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=30399, name=qtp477159732-30399, state=TIMED_WAITING, 
group=TGRP-TestCloudDeleteByQuery]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=30399, name=qtp477159732-30399, state=TIMED_WAITING, 
group=TGRP-TestCloudDeleteByQuery]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([5F5FC887741571]:0)




Build Log:
[...truncated 13395 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestCloudDeleteByQuery
   [junit4]   2> 2875682 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/init-core-data-001
   [junit4]   2> 2875683 WARN  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
   [junit4]   2> 2875683 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2875684 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2875684 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001
   [junit4]   2> 2875684 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2875685 INFO  (Thread-7070) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2875685 INFO  (Thread-7070) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 2875686 ERROR (Thread-7070) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2875785 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.c.ZkTestServer start zk server on port:46478
   [junit4]   2> 2875789 INFO  (zkConnectionManagerCallback-7687-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875799 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T04:27:37+07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2875799 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T04:27:37+07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2875799 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T04:27:37+07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2875802 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2875803 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2875803 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 2875802 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T04:27:37+07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2875804 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2875804 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2875804 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 2875801 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T04:27:37+07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2875807 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2875807 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2875807 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 2875809 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2875809 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2875809 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 2875812 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2875812 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2875812 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 2875812 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@6b20a6e3{/solr,null,AVAILABLE}
   [junit4]   2> 2875813 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@514a93b9{/solr,null,AVAILABLE}
   [junit4]   2> 2875813 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@64057248{/solr,null,AVAILABLE}
   [junit4]   2> 2875814 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@3608a9cb{/solr,null,AVAILABLE}
   [junit4]   2> 2875814 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@2abe35f7{/solr,null,AVAILABLE}
   [junit4]   2> 2875814 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@219b1760{HTTP/1.1,[http/1.1]}{127.0.0.1:36237}
   [junit4]   2> 2875814 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.e.j.s.Server Started @2881692ms
   [junit4]   2> 2875815 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=36237}
   [junit4]   2> 2875815 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@439b9c2f{HTTP/1.1,[http/1.1]}{127.0.0.1:45353}
   [junit4]   2> 2875815 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.e.j.s.Server Started @2881692ms
   [junit4]   2> 2875815 ERROR (jetty-launcher-7684-thread-5) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2875815 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@61c39d7d{HTTP/1.1,[http/1.1]}{127.0.0.1:56956}
   [junit4]   2> 2875815 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 2875815 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=45353}
   [junit4]   2> 2875817 ERROR (jetty-launcher-7684-thread-3) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2875816 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@5d95d5ee{HTTP/1.1,[http/1.1]}{127.0.0.1:48205}
   [junit4]   2> 2875817 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.e.j.s.Server Started @2881694ms
   [junit4]   2> 2875817 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=48205}
   [junit4]   2> 2875817 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2875818 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2875818 ERROR (jetty-launcher-7684-thread-2) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2875816 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@6bf93c8b{HTTP/1.1,[http/1.1]}{127.0.0.1:36738}
   [junit4]   2> 2875818 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 2875815 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.e.j.s.Server Started @2881692ms
   [junit4]   2> 2875818 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=56956}
   [junit4]   2> 2875818 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2875818 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2875818 ERROR (jetty-launcher-7684-thread-1) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2875818 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.e.j.s.Server Started @2881695ms
   [junit4]   2> 2875819 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 2875819 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2875818 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-25T05:16:34.965Z
   [junit4]   2> 2875817 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 2875819 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2875820 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2875819 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2875820 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-25T05:16:34.967Z
   [junit4]   2> 2875819 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=36738}
   [junit4]   2> 2875819 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-25T05:16:34.965Z
   [junit4]   2> 2875820 ERROR (jetty-launcher-7684-thread-4) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2875821 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
8.0.0
   [junit4]   2> 2875821 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2875821 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2875821 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-25T05:16:34.968Z
   [junit4]   2> 2875820 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-01-25T05:16:34.967Z
   [junit4]   2> 2875825 INFO  (zkConnectionManagerCallback-7693-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875825 INFO  (zkConnectionManagerCallback-7691-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875825 INFO  (zkConnectionManagerCallback-7695-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875826 INFO  (zkConnectionManagerCallback-7689-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875826 INFO  (zkConnectionManagerCallback-7697-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875826 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2875826 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2875827 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2875826 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2875827 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2875850 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x1612bbf06910003, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 2875851 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x1612bbf06910001, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 2875853 INFO  (jetty-launcher-7684-thread-5) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46478/solr
   [junit4]   2> 2875853 INFO  (jetty-launcher-7684-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46478/solr
   [junit4]   2> 2875855 INFO  (zkConnectionManagerCallback-7704-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875855 INFO  (jetty-launcher-7684-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46478/solr
   [junit4]   2> 2875856 INFO  (zkConnectionManagerCallback-7706-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875856 INFO  (jetty-launcher-7684-thread-4) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46478/solr
   [junit4]   2> 2875856 INFO  (jetty-launcher-7684-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46478/solr
   [junit4]   2> 2875857 INFO  (zkConnectionManagerCallback-7710-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875858 INFO  (zkConnectionManagerCallback-7715-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875858 INFO  
(zkConnectionManagerCallback-7719-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875859 INFO  
(zkConnectionManagerCallback-7721-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875860 INFO  
(zkConnectionManagerCallback-7725-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875860 INFO  
(zkConnectionManagerCallback-7723-thread-1-processing-n:127.0.0.1:56956_solr) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875861 INFO  (zkConnectionManagerCallback-7717-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2875869 INFO  
(zkConnectionManagerCallback-7727-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2876072 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2876073 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:36237_solr
   [junit4]   2> 2876075 INFO  (jetty-launcher-7684-thread-1) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2876075 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.Overseer Overseer 
(id=99408766227775499-127.0.0.1:36237_solr-n_0000000000) starting
   [junit4]   2> 2876077 INFO  (jetty-launcher-7684-thread-1) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:56956_solr
   [junit4]   2> 2876080 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 2876080 INFO  
(zkCallback-7722-thread-1-processing-n:127.0.0.1:56956_solr) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 2876080 INFO  (jetty-launcher-7684-thread-3) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 2876081 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 2876083 INFO  (jetty-launcher-7684-thread-2) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2876084 INFO  (jetty-launcher-7684-thread-4) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 2876084 INFO  (jetty-launcher-7684-thread-3) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2876088 INFO  (jetty-launcher-7684-thread-2) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:48205_solr
   [junit4]   2> 2876091 INFO  (jetty-launcher-7684-thread-3) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:45353_solr
   [junit4]   2> 2876092 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 2876092 INFO  
(zkCallback-7722-thread-1-processing-n:127.0.0.1:56956_solr) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 2876093 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 2876104 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 2876106 INFO  
(zkCallback-7722-thread-1-processing-n:127.0.0.1:56956_solr) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 2876109 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (3)
   [junit4]   2> 2876109 INFO  
(OverseerStateUpdate-99408766227775499-127.0.0.1:36237_solr-n_0000000000) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (3)
   [junit4]   2> 2876115 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 2876115 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36237_solr
   [junit4]   2> 2876121 INFO  (jetty-launcher-7684-thread-4) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2876124 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 2876125 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 2876125 INFO  
(zkCallback-7722-thread-1-processing-n:127.0.0.1:56956_solr) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 2876131 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 2876131 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 2876167 INFO  (jetty-launcher-7684-thread-4) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36738_solr
   [junit4]   2> 2876213 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 2876213 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 2876216 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 2876216 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 2876223 INFO  (jetty-launcher-7684-thread-1) 
[n:127.0.0.1:56956_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_56956.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876228 INFO  (jetty-launcher-7684-thread-1) 
[n:127.0.0.1:56956_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_56956.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876229 INFO  (jetty-launcher-7684-thread-1) 
[n:127.0.0.1:56956_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_56956.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876229 INFO  (jetty-launcher-7684-thread-1) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node1/.
   [junit4]   2> 2876233 INFO  
(zkCallback-7722-thread-1-processing-n:127.0.0.1:56956_solr) 
[n:127.0.0.1:56956_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 2876235 INFO  (jetty-launcher-7684-thread-3) 
[n:127.0.0.1:45353_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_45353.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876241 INFO  (jetty-launcher-7684-thread-3) 
[n:127.0.0.1:45353_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_45353.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876241 INFO  (jetty-launcher-7684-thread-3) 
[n:127.0.0.1:45353_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_45353.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876242 INFO  (jetty-launcher-7684-thread-3) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node3/.
   [junit4]   2> 2876266 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36237.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876342 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36237.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876344 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36237.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876344 INFO  (jetty-launcher-7684-thread-2) 
[n:127.0.0.1:48205_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_48205.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876346 INFO  (jetty-launcher-7684-thread-4) 
[n:127.0.0.1:36738_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36738.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876353 INFO  (jetty-launcher-7684-thread-5) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node5/.
   [junit4]   2> 2876386 INFO  (jetty-launcher-7684-thread-2) 
[n:127.0.0.1:48205_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_48205.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876386 INFO  (jetty-launcher-7684-thread-4) 
[n:127.0.0.1:36738_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36738.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876386 INFO  (jetty-launcher-7684-thread-2) 
[n:127.0.0.1:48205_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_48205.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876386 INFO  (jetty-launcher-7684-thread-4) 
[n:127.0.0.1:36738_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36738.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2876387 INFO  (jetty-launcher-7684-thread-4) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node4/.
   [junit4]   2> 2876387 INFO  (jetty-launcher-7684-thread-2) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node2/.
   [junit4]   2> 2876491 INFO  (zkConnectionManagerCallback-7739-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2876494 INFO  (zkConnectionManagerCallback-7743-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2876496 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 2876497 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46478/solr ready
   [junit4]   2> 2876968 INFO  (qtp860975917-30414) [n:127.0.0.1:48205_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=2&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&collection.configName=solrCloudCollectionConfig&name=test_col&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 2876971 INFO  
(OverseerThreadFactory-11073-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
test_col
   [junit4]   2> 2877088 INFO  
(OverseerStateUpdate-99408766227775499-127.0.0.1:36237_solr-n_0000000000) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"test_col",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"test_col_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36738/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2877094 INFO  
(OverseerStateUpdate-99408766227775499-127.0.0.1:36237_solr-n_0000000000) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"test_col",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"test_col_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36237/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2877102 INFO  
(OverseerStateUpdate-99408766227775499-127.0.0.1:36237_solr-n_0000000000) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"test_col",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"test_col_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:45353/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2877109 INFO  
(OverseerStateUpdate-99408766227775499-127.0.0.1:36237_solr-n_0000000000) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"test_col",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"test_col_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:48205/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2877315 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node3&name=test_col_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 2877315 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node7&name=test_col_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 2877316 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr    
] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 
2147483647 transient cores
   [junit4]   2> 2877316 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr    
] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 
2147483647 transient cores
   [junit4]   2> 2877318 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node5&name=test_col_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 2877318 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node8&name=test_col_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 2877318 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr    
] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 
2147483647 transient cores
   [junit4]   2> 2877319 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr    
] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 
2147483647 transient cores
   [junit4]   2> 2877432 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2877432 INFO  
(zkCallback-7720-thread-2-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2877432 INFO  
(zkCallback-7726-thread-2-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2877432 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2877433 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2877433 INFO  
(zkCallback-7718-thread-2-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2877433 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2877433 INFO  
(zkCallback-7724-thread-2-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878373 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2878374 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2878376 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2878383 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2878385 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.s.IndexSchema [test_col_shard1_replica_n2] Schema name=test
   [junit4]   2> 2878395 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.s.IndexSchema [test_col_shard2_replica_n4] Schema name=test
   [junit4]   2> 2878403 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.s.IndexSchema [test_col_shard1_replica_n1] Schema name=test
   [junit4]   2> 2878405 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.s.IndexSchema [test_col_shard2_replica_n6] Schema name=test
   [junit4]   2> 2878462 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2878491 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2878506 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard1_replica_n2' using 
configuration from collection test_col, trusted=true
   [junit4]   2> 2878507 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36237.solr.core.test_col.shard1.replica_n2' (registry 
'solr.core.test_col.shard1.replica_n2') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2878507 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] o.a.s.c.SolrCore 
solr.RecoveryStrategy.Builder
   [junit4]   2> 2878507 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] o.a.s.c.SolrCore 
[[test_col_shard1_replica_n2] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node5/test_col_shard1_replica_n2],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node5/./test_col_shard1_replica_n2/data/]
   [junit4]   2> 2878509 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=10.287109375, 
floorSegmentMB=2.1953125, forceMergeDeletesPctAllowed=20.62421038487977, 
segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.15113424844762557
   [junit4]   2> 2878511 WARN  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2878512 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2878515 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2878533 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard2_replica_n6' using 
configuration from collection test_col, trusted=true
   [junit4]   2> 2878533 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_48205.solr.core.test_col.shard2.replica_n6' (registry 
'solr.core.test_col.shard2.replica_n6') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2878533 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] o.a.s.c.SolrCore 
solr.RecoveryStrategy.Builder
   [junit4]   2> 2878533 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] o.a.s.c.SolrCore 
[[test_col_shard2_replica_n6] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node2/test_col_shard2_replica_n6],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node2/./test_col_shard2_replica_n6/data/]
   [junit4]   2> 2878535 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=10.287109375, 
floorSegmentMB=2.1953125, forceMergeDeletesPctAllowed=20.62421038487977, 
segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.15113424844762557
   [junit4]   2> 2878538 WARN  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2878548 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard1_replica_n1' using 
configuration from collection test_col, trusted=true
   [junit4]   2> 2878548 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36738.solr.core.test_col.shard1.replica_n1' (registry 
'solr.core.test_col.shard1.replica_n1') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2878548 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] o.a.s.c.SolrCore 
solr.RecoveryStrategy.Builder
   [junit4]   2> 2878548 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] o.a.s.c.SolrCore 
[[test_col_shard1_replica_n1] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node4/test_col_shard1_replica_n1],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node4/./test_col_shard1_replica_n1/data/]
   [junit4]   2> 2878550 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=10.287109375, 
floorSegmentMB=2.1953125, forceMergeDeletesPctAllowed=20.62421038487977, 
segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.15113424844762557
   [junit4]   2> 2878552 WARN  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2878579 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2878579 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2878580 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2878580 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2878581 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard2_replica_n4' using 
configuration from collection test_col, trusted=true
   [junit4]   2> 2878582 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_45353.solr.core.test_col.shard2.replica_n4' (registry 
'solr.core.test_col.shard2.replica_n4') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@28f55af0
   [junit4]   2> 2878582 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] o.a.s.c.SolrCore 
solr.RecoveryStrategy.Builder
   [junit4]   2> 2878582 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] o.a.s.c.SolrCore 
[[test_col_shard2_replica_n4] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node3/test_col_shard2_replica_n4],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudDeleteByQuery_5F5FC887741571-001/tempDir-001/node3/./test_col_shard2_replica_n4/data/]
   [junit4]   2> 2878583 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=10.287109375, 
floorSegmentMB=2.1953125, forceMergeDeletesPctAllowed=20.62421038487977, 
segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.15113424844762557
   [junit4]   2> 2878585 WARN  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2878591 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=470650684, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 2878592 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2878592 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2878593 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2878593 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2878595 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=470650684, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 2878595 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@e8f8f08[test_col_shard1_replica_n2] 
main]
   [junit4]   2> 2878595 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@3fa55a63[test_col_shard1_replica_n1] main]
   [junit4]   2> 2878601 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 2878603 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2878603 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2878604 INFO  
(searcherExecutor-11084-thread-1-processing-n:127.0.0.1:36237_solr 
x:test_col_shard1_replica_n2 s:shard1 c:test_col r:core_node5) 
[n:127.0.0.1:36237_solr c:test_col s:shard1 r:core_node5 
x:test_col_shard1_replica_n2] o.a.s.c.SolrCore [test_col_shard1_replica_n2] 
Registered new searcher Searcher@e8f8f08[test_col_shard1_replica_n2] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2878615 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2878616 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2878616 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2878616 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2878618 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1590540262718832640
   [junit4]   2> 2878620 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=470650684, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 2878620 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@2fa4b100[test_col_shard2_replica_n6] main]
   [junit4]   2> 2878620 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 2878621 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2878621 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2878622 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 2878622 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2878623 INFO  
(searcherExecutor-11086-thread-1-processing-n:127.0.0.1:36738_solr 
x:test_col_shard1_replica_n1 s:shard1 c:test_col r:core_node3) 
[n:127.0.0.1:36738_solr c:test_col s:shard1 r:core_node3 
x:test_col_shard1_replica_n1] o.a.s.c.SolrCore [test_col_shard1_replica_n1] 
Registered new searcher Searcher@3fa55a63[test_col_shard1_replica_n1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2878623 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2878623 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1590540262724075520
   [junit4]   2> 2878623 INFO  
(searcherExecutor-11085-thread-1-processing-n:127.0.0.1:48205_solr 
x:test_col_shard2_replica_n6 s:shard2 c:test_col r:core_node8) 
[n:127.0.0.1:48205_solr c:test_col s:shard2 r:core_node8 
x:test_col_shard2_replica_n6] o.a.s.c.SolrCore [test_col_shard2_replica_n6] 
Registered new searcher Searcher@2fa4b100[test_col_shard2_replica_n6] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2878624 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1590540262725124096
   [junit4]   2> 2878630 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2878631 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2878631 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:36738/solr/test_col_shard1_replica_n1/
   [junit4]   2> 2878631 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] o.a.s.u.PeerSync 
PeerSync: core=test_col_shard1_replica_n1 url=http://127.0.0.1:36738/solr START 
replicas=[http://127.0.0.1:36237/solr/test_col_shard1_replica_n2/] nUpdates=100
   [junit4]   2> 2878634 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2878634 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2878635 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2878635 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2878637 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for 
shard shard2: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2878637 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=470650684, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 2878637 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@777ae141[test_col_shard2_replica_n4] main]
   [junit4]   2> 2878638 INFO  (qtp311980014-30427) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.c.S.Request [test_col_shard1_replica_n2]  webapp=/solr path=/get 
params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   2> 2878639 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 2878639 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2878640 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2878640 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1590540262741901312
   [junit4]   2> 2878640 INFO  
(searcherExecutor-11092-thread-1-processing-n:127.0.0.1:45353_solr 
x:test_col_shard2_replica_n4 s:shard2 c:test_col r:core_node7) 
[n:127.0.0.1:45353_solr c:test_col s:shard2 r:core_node7 
x:test_col_shard2_replica_n4] o.a.s.c.SolrCore [test_col_shard2_replica_n4] 
Registered new searcher Searcher@777ae141[test_col_shard2_replica_n4] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2878738 INFO  
(zkCallback-7718-thread-2-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878738 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878738 INFO  
(zkCallback-7726-thread-2-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878738 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878738 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878738 INFO  
(zkCallback-7720-thread-2-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878738 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878738 INFO  
(zkCallback-7724-thread-2-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2878932 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] o.a.s.u.PeerSync 
PeerSync: core=test_col_shard1_replica_n1 url=http://127.0.0.1:36738/solr DONE. 
 We have no versions.  sync failed.
   [junit4]   2> 2878932 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the 
next candidate
   [junit4]   2> 2878932 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we 
can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2878932 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 2878944 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:36738/solr/test_col_shard1_replica_n1/ shard1
   [junit4]   2> 2879048 INFO  
(zkCallback-7720-thread-3-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879048 INFO  
(zkCallback-7726-thread-2-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879049 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879049 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879049 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879049 INFO  
(zkCallback-7724-thread-2-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879049 INFO  
(zkCallback-7718-thread-2-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879049 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879096 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2879101 INFO  (qtp302106822-30432) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node3&name=test_col_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin}
 status=0 QTime=1786
   [junit4]   2> 2879139 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2879139 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2879139 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:48205/solr/test_col_shard2_replica_n6/
   [junit4]   2> 2879139 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] o.a.s.u.PeerSync 
PeerSync: core=test_col_shard2_replica_n6 url=http://127.0.0.1:48205/solr START 
replicas=[http://127.0.0.1:45353/solr/test_col_shard2_replica_n4/] nUpdates=100
   [junit4]   2> 2879142 INFO  (qtp1623261494-30407) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.c.S.Request [test_col_shard2_replica_n4]  webapp=/solr path=/get 
params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   2> 2879203 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879203 INFO  
(zkCallback-7720-thread-3-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879203 INFO  
(zkCallback-7724-thread-2-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879204 INFO  
(zkCallback-7718-thread-2-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879204 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879204 INFO  
(zkCallback-7726-thread-2-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879204 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879204 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879440 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] o.a.s.u.PeerSync 
PeerSync: core=test_col_shard2_replica_n6 url=http://127.0.0.1:48205/solr DONE. 
 We have no versions.  sync failed.
   [junit4]   2> 2879440 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the 
next candidate
   [junit4]   2> 2879440 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we 
can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2879440 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 2879453 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:48205/solr/test_col_shard2_replica_n6/ shard2
   [junit4]   2> 2879557 INFO  
(zkCallback-7726-thread-2-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879557 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879557 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879557 INFO  
(zkCallback-7724-thread-2-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879558 INFO  
(zkCallback-7720-thread-3-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879558 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879558 INFO  
(zkCallback-7718-thread-2-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879559 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879606 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2879610 INFO  (qtp860975917-30418) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node8&name=test_col_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin}
 status=0 QTime=2292
   [junit4]   2> 2879645 INFO  (qtp311980014-30422) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node5&name=test_col_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin}
 status=0 QTime=2328
   [junit4]   2> 2879650 INFO  (qtp1623261494-30406) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&coreNodeName=core_node7&name=test_col_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin}
 status=0 QTime=2335
   [junit4]   2> 2879656 INFO  (qtp860975917-30414) [n:127.0.0.1:48205_solr    
] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 2879752 INFO  
(zkCallback-7726-thread-2-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879752 INFO  
(zkCallback-7726-thread-1-processing-n:127.0.0.1:48205_solr) 
[n:127.0.0.1:48205_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879752 INFO  
(zkCallback-7718-thread-2-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879752 INFO  
(zkCallback-7724-thread-1-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879753 INFO  
(zkCallback-7720-thread-1-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879752 INFO  
(zkCallback-7718-thread-1-processing-n:127.0.0.1:45353_solr) 
[n:127.0.0.1:45353_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879754 INFO  
(zkCallback-7724-thread-2-processing-n:127.0.0.1:36738_solr) 
[n:127.0.0.1:36738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2879754 INFO  
(zkCallback-7720-thread-3-processing-n:127.0.0.1:36237_solr) 
[n:127.0.0.1:36237_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/test_col/state.json] for collection [test_col] has occurred - 
updating... (live nodes size: [5])
   [junit4]   2> 2880658 INFO  (qtp860975917-30414) [n:127.0.0.1:48205_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=2&property.config=solrconfig-tlog.xml&property.schema=schema15.xml&collection.configName=solrCloudCollectionConfig&name=test_col&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2}
 status=0 QTime=3690
   [junit4]   2> 2880658 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: 
test_col failOnTimeout:true timeout (sec):330
   [junit4]   1> -
   [junit4]   1> replica:core_node3 rstate:active live:true
   [junit4]   1> replica:core_node5 rstate:active live:true
   [junit4]   1> replica:core_node7 rstate:active live:true
   [junit4]   1> replica:core_node8 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   2> 2880660 INFO  
(SUITE-TestCloudDeleteByQuery-seed#[5F5FC887741571]-worker) [    ] 
o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: test_col
   [junit4]   2> 2880678 INFO  (qtp311980014-30424) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.p.LogUpdateProcessorFactory [test_col_shard1_replica_n2]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:36738/solr/test_col_shard1_replica_n1/&wt=javabin&version=2}{add=[abc!809411098
 (1590540264874704896)]} 0 1
   [junit4]   2> 2880678 INFO  (qtp302106822-30437) [n:127.0.0.1:36738_solr 
c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] 
o.a.s.u.p.LogUpdateProcessorFactory [test_col_shard1_replica_n1]  webapp=/solr 
path=/update params={wt=javabin&version=2}{add=[abc!809411098 
(1590540264874704896)]} 0 4
   [junit4]   2> 2880683 INFO  (qtp1623261494-30408) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.p.LogUpdateProcessorFactory [test_col_shard2_replica_n4]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:48205/solr/test_col_shard2_replica_n6/&wt=javabin&version=2}{add=[XYZ!1954668424
 (1590540264880996352)]} 0 0
   [junit4]   2> 2880683 INFO  (qtp860975917-30419) [n:127.0.0.1:48205_solr 
c:test_col s:shard2 r:core_node8 x:test_col_shard2_replica_n6] 
o.a.s.u.p.LogUpdateProcessorFactory [test_col_shard2_replica_n6]  webapp=/solr 
path=/update params={wt=javabin&version=2}{add=[XYZ!1954668424 
(1590540264880996352)]} 0 3
   [junit4]   2> 2880686 INFO  (qtp1623261494-30407) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.DirectUpdateHandler2 start 
commit{_version_=1590540264887287808,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2880686 INFO  (qtp1623261494-30407) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@14b37722 
commitCommandVersion:1590540264887287808
   [junit4]   2> 2880688 INFO  (qtp311980014-30424) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.DirectUpdateHandler2 start 
commit{_version_=1590540264889384960,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2880688 INFO  (qtp311980014-30424) [n:127.0.0.1:36237_solr 
c:test_col s:shard1 r:core_node5 x:test_col_shard1_replica_n2] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@3093011e 
commitCommandVersion:1590540264889384960
   [junit4]   2> 2880688 INFO  (qtp1623261494-30407) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node7 x:test_col_shard2_replica_n4] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@6f96857c[test_col_shard2_replica_n4] main]
   [junit4]   2> 2880688 INFO  (qtp1623261494-30407) [n:127.0.0.1:45353_solr 
c:test_col s:shard2 r:core_node

[...truncated too long message...]

t java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=CheapBastard, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@296837e3),
 locale=lt, timezone=Asia/Dacca
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 
(64-bit)/cpus=3,threads=2,free=256719480,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [SpellingQueryConverterTest, 
RAMDirectoryFactoryTest, PathHierarchyTokenizerFactoryTest, 
TestImpersonationWithHadoopAuth, TestSweetSpotSimilarityFactory, 
MetricTriggerTest, CollectionsAPISolrJTest, DistribJoinFromCollectionTest, 
SearchHandlerTest, DeleteInactiveReplicaTest, SolrRequestParserTest, 
PluginInfoTest, CopyFieldTest, BlockJoinFacetRandomTest, TestCloudSchemaless, 
TestCollapseQParserPlugin, TestOrdValues, TestFieldCacheSort, 
CdcrBootstrapTest, ComputePlanActionTest, TestReloadAndDeleteDocs, 
TestDocBasedVersionConstraints, SolrJmxReporterTest, TestSchemaResource, 
BlockCacheTest, TestOmitPositions, TestOnReconnectListenerSupport, 
MultiThreadedOCPTest, TestLazyCores, TestCursorMarkWithoutUniqueKey, 
TestSQLHandler, InfixSuggestersTest, TestCryptoKeys, TestSolrCoreSnapshots, 
LeaderFailoverAfterPartitionTest, QueryElevationComponentTest, 
SpatialFilterTest, DirectSolrConnectionTest, HdfsChaosMonkeySafeLeaderTest, 
PKIAuthenticationIntegrationTest, DistributedQueryComponentCustomSortTest, 
TestJavabinTupleStreamParser, BasicZkTest, TestBlendedInfixSuggestions, 
DistributedFacetSimpleRefinementLongTailTest, BinaryUpdateRequestHandlerTest, 
DocumentBuilderTest, TestGroupingSearch, SearchHandlerTest, HighlighterTest, 
TestMultiValuedNumericRangeQuery, TestScoreJoinQPScore, 
TestRandomCollapseQParserPlugin, TestComplexPhraseQParserPlugin, 
SynonymTokenizerTest, AddReplicaTest, TestCSVResponseWriter, TestHdfsUpdateLog, 
TestFiltering, TestSubQueryTransformerCrossCore, 
OverseerCollectionConfigSetProcessorTest, 
TestDistributedStatsComponentCardinality, StatsReloadRaceTest, TestInitQParser, 
MergeStrategyTest, CollectionsAPIAsyncDistributedZkTest, 
HdfsRestartWhileUpdatingTest, TestAuthenticationFramework, 
TestFastOutputStream, SolrGraphiteReporterTest, BJQParserTest, TestUtilizeNode, 
RestartWhileUpdatingTest, TestSha256AuthenticationProvider, 
TestSimpleQParserPlugin, TestRealTimeGet, TestSuggestSpellingConverter, 
ChangedSchemaMergeTest, SSLMigrationTest, ResponseHeaderTest, 
TestMaxTokenLenTokenizer, LeaderElectionIntegrationTest, 
TriLevelCompositeIdRoutingTest, HighlighterMaxOffsetTest, ForceLeaderTest, 
TestSystemCollAutoCreate, TestUniqueKeyFieldResource, TestCloudPivotFacet, 
HdfsLockFactoryTest, TestIndexSearcher, StressHdfsTest, TestQueryUtils, 
SolrInfoBeanTest, OutputWriterTest, BadCopyFieldTest, ExplicitHLLTest, 
HdfsDirectoryFactoryTest, TestNumericRangeQuery64, LegacyCloudClusterPropTest, 
DistanceUnitsTest, TestCollectionsAPIViaSolrCloudCluster, 
DirectSolrSpellCheckerTest, IndexSchemaRuntimeFieldTest, V2StandaloneTest, 
TestExecutePlanAction, DistributedFacetPivotLongTailTest, 
TestPhraseSuggestions, HttpPartitionTest, HdfsRecoverLeaseTest, 
TestBM25SimilarityFactory, TestReloadDeadlock, SolrCoreTest, 
MetricsHandlerTest, SolrPluginUtilsTest, TestHalfAndHalfDocValues, 
ConnectionManagerTest, TestEmbeddedSolrServerAdminHandler, 
CoreAdminCreateDiscoverTest, WordBreakSolrSpellCheckerTest, RulesTest, 
TestBackupRepositoryFactory, ReplaceNodeTest, TestCharFilters, 
TestZkAclsWithHadoopAuth, SolrCloudReportersTest, TestClusterProperties, 
TestOverriddenPrefixQueryForCustomFieldType, TestReRankQParserPlugin, 
DistributedMLTComponentTest, SparseHLLTest, 
HdfsCollectionsAPIDistributedZkTest, TestSolrConfigHandlerCloud, 
AssignBackwardCompatibilityTest, TestFoldingMultitermQuery, 
HLLSerializationTest, BaseCdcrDistributedZkTest, TestRandomFlRTGCloud, 
TestNestedDocsSort, HdfsAutoAddReplicasIntegrationTest, ZkStateReaderTest, 
LeaderInitiatedRecoveryOnCommitTest, TestJoin, TestZkChroot, 
TestPerFieldSimilarity, DistribCursorPagingTest, TestRemoteStreaming, 
TestMergePolicyConfig, UniqFieldsUpdateProcessorFactoryTest, TestLockTree, 
PeerSyncWithIndexFingerprintCachingTest, TestMultiWordSynonyms, MultiTermTest, 
TestQuerySenderNoQuery, TestGeoJSONResponseWriter, 
TermVectorComponentDistributedTest, TestInitParams, 
TolerantUpdateProcessorTest, DocExpirationUpdateProcessorFactoryTest, 
TestSSLRandomization, SecurityConfHandlerTest, TestFunctionQuery, 
TestSimpleTextCodec, TestNumericTerms64, NumericFieldsTest, MBeansHandlerTest, 
TestPKIAuthenticationPlugin, TestBulkSchemaConcurrent, 
OutOfBoxZkACLAndCredentialsProvidersTest, SOLR749Test, PolyFieldTest, 
CloudExitableDirectoryReaderTest, TestSolr4Spatial2, TestLargeCluster, 
LeaderElectionTest, FastVectorHighlighterTest, DistributedFacetPivotSmallTest, 
TestRTimerTree, TestHighlightDedupGrouping, 
SignatureUpdateProcessorFactoryTest, CdcrVersionReplicationTest, 
HdfsThreadLeakTest, TestRequestStatusCollectionAPI, TestSolrJ, 
BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, 
OpenCloseCoreStressTest, RecoveryZkTest, TestReplicationHandler, 
ClusterStateUpdateTest, TestRandomFaceting, ZkSolrClientTest, ZkCLITest, 
ShardRoutingCustomTest, TestDistributedGrouping, TestFaceting, TestRecovery, 
DistributedSpellCheckComponentTest, TestReload, DistributedTermsComponentTest, 
TestRangeQuery, TestCoreContainer, SimpleFacetsTest, StatsComponentTest, 
SpellCheckComponentTest, SolrCmdDistributorTest, ConvertedLegacyTest, 
BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, 
SoftAutoCommitTest, XmlUpdateRequestHandlerTest, TestValueSourceCache, 
TestIndexingPerformance, MoreLikeThisHandlerTest, 
FieldAnalysisRequestHandlerTest, TestJmxIntegration, ReturnFieldsTest, 
PingRequestHandlerTest, AlternateDirectoryTest, ResponseLogComponentTest, 
TestStressRecovery, TestPostingsSolrHighlighter, TestXIncludeConfig, 
TestDFRSimilarityFactory, TestLMJelinekMercerSimilarityFactory, 
TestIBSimilarityFactory, TimeZoneUtilsTest, ScriptEngineTest, 
URLClassifyProcessorTest, TestSystemIdResolver, DateFieldTest, ZkNodePropsTest, 
FileUtilsTest, CursorPagingTest, SolrTestCaseJ4Test, 
TestSimpleTrackingShardHandler, TestEmbeddedSolrServerSchemaAPI, 
ConnectionReuseTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, 
HealthCheckHandlerTest, LeaderElectionContextKeyTest, 
MoveReplicaHDFSFailoverTest, MoveReplicaTest, OverseerRolesTest, 
OverseerTaskQueueTest, RecoveryAfterSoftCommitTest, 
SharedFSAutoReplicaFailoverTest, SolrCloudExampleTest, TestCloudDeleteByQuery]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestCloudDeleteByQuery -Dtests.seed=5F5FC887741571 -Dtests.slow=true 
-Dtests.locale=lt -Dtests.timezone=Asia/Dacca -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | TestCloudDeleteByQuery (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.TestCloudDeleteByQuery: 
   [junit4]    >    1) Thread[id=30399, name=qtp477159732-30399, 
state=TIMED_WAITING, group=TGRP-TestCloudDeleteByQuery]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([5F5FC887741571]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=30399, name=qtp477159732-30399, 
state=TIMED_WAITING, group=TGRP-TestCloudDeleteByQuery]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at __randomizedtesting.SeedInfo.seed([5F5FC887741571]:0)
   [junit4] Completed [538/768 (1!)] on J0 in 68.74s, 6 tests, 2 errors <<< 
FAILURES!

[...truncated 48836 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to