[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 31 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/31/

No tests ran.

Build Log:
[...truncated 1780 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/build/core/test/temp/junit4-J1-20180404_034607_8616246385250962902160.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] codec: FastDecompressionCompressingStoredFields, pf: FSTOrd50, dvf: 
Memory
   [junit4] <<< JVM J1: EOF ----

[...truncated 10409 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.ComputePlanActionTest
   [junit4]   2> 324117 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ComputePlanActionTest_AE5AB9149E50D967-001/init-core-data-001
   [junit4]   2> 324189 WARN  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 324245 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 324246 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 324247 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ComputePlanActionTest_AE5AB9149E50D967-001/tempDir-001
   [junit4]   2> 324248 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 324248 INFO  (Thread-45) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 324248 INFO  (Thread-45) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 324250 ERROR (Thread-45) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 324348 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.c.ZkTestServer start zk server on port:41904
   [junit4]   2> 324351 INFO  (zkConnectionManagerCallback-85-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 324382 INFO  (jetty-launcher-82-thread-1) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 324396 INFO  (jetty-launcher-82-thread-1) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 324396 INFO  (jetty-launcher-82-thread-1) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 324396 INFO  (jetty-launcher-82-thread-1) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 324396 INFO  (jetty-launcher-82-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@487d739{/solr,null,AVAILABLE}
   [junit4]   2> 324462 INFO  (jetty-launcher-82-thread-1) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@86725e4{SSL,[ssl, 
http/1.1]}{127.0.0.1:33709}
   [junit4]   2> 324462 INFO  (jetty-launcher-82-thread-1) [    ] 
o.e.j.s.Server Started @324615ms
   [junit4]   2> 324462 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=33709}
   [junit4]   2> 324463 ERROR (jetty-launcher-82-thread-1) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 324463 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 324463 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.4.0
   [junit4]   2> 324463 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 324463 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 324463 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-04-04T04:46:56.551Z
   [junit4]   2> 324482 INFO  (zkConnectionManagerCallback-87-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 324482 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 324500 INFO  (jetty-launcher-82-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41904/solr
   [junit4]   2> 324502 INFO  (zkConnectionManagerCallback-91-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 324526 INFO  (zkConnectionManagerCallback-93-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 324840 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 324840 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:33709_solr
   [junit4]   2> 324841 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.c.Overseer Overseer 
(id=72395407035596803-127.0.0.1:33709_solr-n_0000000000) starting
   [junit4]   2> 324874 INFO  (zkConnectionManagerCallback-100-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 324888 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster 
at 127.0.0.1:41904/solr ready
   [junit4]   2> 324889 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 324889 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:33709_solr
   [junit4]   2> 324890 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and 
.scheduled_maintenance triggers
   [junit4]   2> 324902 INFO  (zkCallback-99-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 324903 INFO  
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 324903 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 1
   [junit4]   2> 324904 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, 
lastZnodeVersion -1
   [junit4]   2> 324904 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto 
znodeVersion 1
   [junit4]   2> 324933 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial 
livenodes: [127.0.0.1:33709_solr]
   [junit4]   2> 324933 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 324934 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, 
lastZnodeVersion 1
   [junit4]   2> 324934 DEBUG (ScheduledTrigger-100-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 325815 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33709.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 325838 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33709.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 325838 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33709.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 325839 INFO  (jetty-launcher-82-thread-1) 
[n:127.0.0.1:33709_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ComputePlanActionTest_AE5AB9149E50D967-001/tempDir-001/node1/.
   [junit4]   2> 325999 DEBUG (ScheduledTrigger-100-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 1
   [junit4]   2> 326074 INFO  (zkConnectionManagerCallback-102-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 326088 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x101333d37ba0005, likely client has closed socket
   [junit4]   2> 326110 INFO  (zkConnectionManagerCallback-107-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 326112 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 326141 INFO  
(SUITE-ComputePlanActionTest-seed#[AE5AB9149E50D967]-worker) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41904/solr ready
   [junit4]   2> 326203 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeWithMultipleReplicasLost
   [junit4]   2> 326204 DEBUG (zkCallback-99-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
2
   [junit4]   2> 326204 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto 
znodeVersion 2
   [junit4]   2> 326205 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 326207 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 326207 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 326208 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, 
lastZnodeVersion 2
   [junit4]   2> 326869 INFO  (qtp1534579564-269) [n:127.0.0.1:33709_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=20
   [junit4]   2> 326873 DEBUG (qtp1534579564-265) [n:127.0.0.1:33709_solr    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 326881 INFO  (qtp1534579564-265) [n:127.0.0.1:33709_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling 
params={wt=javabin&version=2} status=0 QTime=203
   [junit4]   2> 326881 DEBUG (zkCallback-99-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 326881 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto 
znodeVersion 3
   [junit4]   2> 326881 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 326881 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 326882 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, 
lastZnodeVersion 3
   [junit4]   2> 326887 INFO  (qtp1534579564-267) [n:127.0.0.1:33709_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=1
   [junit4]   2> 326896 DEBUG (qtp1534579564-268) [n:127.0.0.1:33709_solr    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 326897 INFO  (qtp1534579564-268) [n:127.0.0.1:33709_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling 
params={wt=javabin&version=2} status=0 QTime=14
   [junit4]   2> 326897 DEBUG (zkCallback-99-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
4
   [junit4]   2> 326897 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto 
znodeVersion 4
   [junit4]   2> 326897 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 326898 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 326898 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, 
lastZnodeVersion 4
   [junit4]   2> 326910 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 326926 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 326926 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 326926 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 326927 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@76c3d4f8{/solr,null,AVAILABLE}
   [junit4]   2> 326954 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.AbstractConnector Started ServerConnector@36c0996a{SSL,[ssl, 
http/1.1]}{127.0.0.1:40404}
   [junit4]   2> 326954 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.Server Started @327107ms
   [junit4]   2> 326954 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=40404}
   [junit4]   2> 326955 ERROR 
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging 
may be missing or incomplete.
   [junit4]   2> 326955 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 326955 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ 
version 7.4.0
   [junit4]   2> 326955 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on 
port null
   [junit4]   2> 326955 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 326955 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-04-04T04:46:59.043Z
   [junit4]   2> 326965 INFO  (zkConnectionManagerCallback-111-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 326966 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 327023 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41904/solr
   [junit4]   2> 327026 INFO  (zkConnectionManagerCallback-115-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 327062 INFO  (zkConnectionManagerCallback-117-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 327092 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 327094 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 327109 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating 
transient cache for 2147483647 transient cores
   [junit4]   2> 327109 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:40404_solr
   [junit4]   2> 327110 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40404_solr"} current state version: 0
   [junit4]   2> 327110 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:40404_solr
   [junit4]   2> 327110 INFO  (zkCallback-99-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 327111 INFO  (zkCallback-92-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 327111 INFO  (zkCallback-106-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 327144 INFO  (zkCallback-116-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 327180 INFO  (zkCallback-106-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 327479 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_40404.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 327499 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_40404.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 327499 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_40404.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 327500 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ComputePlanActionTest_AE5AB9149E50D967-001/tempDir-001/node2/.
   [junit4]   2> 327547 INFO  (zkConnectionManagerCallback-124-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 327549 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 327550 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:40404_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster 
at 127.0.0.1:41904/solr ready
   [junit4]   2> 327649 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 327653 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 327653 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 327653 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 327653 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1093d0ff{/solr,null,AVAILABLE}
   [junit4]   2> 327654 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.AbstractConnector Started ServerConnector@435d561d{SSL,[ssl, 
http/1.1]}{127.0.0.1:45170}
   [junit4]   2> 327654 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.Server Started @327807ms
   [junit4]   2> 327654 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=45170}
   [junit4]   2> 327654 ERROR 
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging 
may be missing or incomplete.
   [junit4]   2> 327654 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 327654 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ 
version 7.4.0
   [junit4]   2> 327654 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on 
port null
   [junit4]   2> 327654 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 327655 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-04-04T04:46:59.743Z
   [junit4]   2> 327711 INFO  (zkConnectionManagerCallback-126-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 327713 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 327745 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41904/solr
   [junit4]   2> 327775 INFO  (zkConnectionManagerCallback-130-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 327807 INFO  (zkConnectionManagerCallback-132-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 327835 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 327837 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 327848 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating 
transient cache for 2147483647 transient cores
   [junit4]   2> 327848 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:45170_solr
   [junit4]   2> 327850 INFO  (zkCallback-106-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 327850 INFO  (zkCallback-116-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 327850 INFO  (zkCallback-106-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 327858 INFO  (zkCallback-99-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 327908 INFO  (zkCallback-131-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 327908 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45170_solr"} current state version: 0
   [junit4]   2> 327908 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:45170_solr
   [junit4]   2> 327908 INFO  (zkCallback-92-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 327908 INFO  (zkCallback-123-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 328225 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_45170.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 328270 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_45170.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 328270 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_45170.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 328272 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ComputePlanActionTest_AE5AB9149E50D967-001/tempDir-001/node3/.
   [junit4]   2> 328285 INFO  (zkConnectionManagerCallback-139-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 328319 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 328320 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:45170_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster 
at 127.0.0.1:41904/solr ready
   [junit4]   2> 328426 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 328436 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 328436 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 328436 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 328436 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1af64112{/solr,null,AVAILABLE}
   [junit4]   2> 328437 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.AbstractConnector Started ServerConnector@659981e3{SSL,[ssl, 
http/1.1]}{127.0.0.1:39340}
   [junit4]   2> 328437 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.e.j.s.Server Started @328590ms
   [junit4]   2> 328437 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=39340}
   [junit4]   2> 328437 ERROR 
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging 
may be missing or incomplete.
   [junit4]   2> 328437 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter Using logger factory 
org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 328437 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ 
version 7.4.0
   [junit4]   2> 328437 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on 
port null
   [junit4]   2> 328437 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 328438 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-04-04T04:47:00.526Z
   [junit4]   2> 328541 INFO  (zkConnectionManagerCallback-141-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 328557 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 328583 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41904/solr
   [junit4]   2> 328643 INFO  (zkConnectionManagerCallback-145-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 328667 INFO  (zkConnectionManagerCallback-147-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 328684 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 328687 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 328708 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating 
transient cache for 2147483647 transient cores
   [junit4]   2> 328708 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:39340_solr
   [junit4]   2> 328709 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:39340_solr"} current state version: 0
   [junit4]   2> 328709 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:39340_solr
   [junit4]   2> 328723 INFO  (zkCallback-106-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328725 INFO  (zkCallback-99-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328725 INFO  (zkCallback-123-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328726 INFO  (zkCallback-92-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328772 INFO  (zkCallback-146-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328772 INFO  (zkCallback-116-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328772 INFO  (zkCallback-131-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328772 INFO  (zkCallback-138-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 328773 INFO  (zkCallback-106-thread-2) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 329336 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39340.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 329360 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39340.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 329433 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39340.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@78c79271
   [junit4]   2> 329435 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ComputePlanActionTest_AE5AB9149E50D967-001/tempDir-001/node4/.
   [junit4]   2> 329500 INFO  (zkConnectionManagerCallback-154-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 329501 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (4)
   [junit4]   2> 329502 INFO  
(TEST-ComputePlanActionTest.testNodeWithMultipleReplicasLost-seed#[AE5AB9149E50D967])
 [n:127.0.0.1:39340_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster 
at 127.0.0.1:41904/solr ready
   [junit4]   2> 329708 INFO  (zkConnectionManagerCallback-156-thread-1) [    ] 
o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 329902 INFO  (qtp1534579564-269) [n:127.0.0.1:33709_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 330099 INFO  (qtp1082391194-368) [n:127.0.0.1:39340_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=1
   [junit4]   2> 330330 INFO  (qtp1813385384-311) [n:127.0.0.1:40404_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 330461 INFO  (qtp524182178-337) [n:127.0.0.1:45170_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=16
   [junit4]   2> 330462 DEBUG (qtp1534579564-265) [n:127.0.0.1:33709_solr    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 330463 INFO  (qtp1534579564-265) [n:127.0.0.1:33709_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling 
params={wt=javabin&version=2} status=0 QTime=656
   [junit4]   2> 330477 DEBUG (zkCallback-99-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
5
   [junit4]   2> 330510 INFO  (qtp1534579564-268) [n:127.0.0.1:33709_solr    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=3&collection.configName=conf&name=testNodeWithMultipleReplicasLost&nrtReplicas=3&action=CREATE&numShards=2&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 330510 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto 
znodeVersion 5
   [junit4]   2> 330554 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial 
livenodes: [127.0.0.1:45170_solr, 127.0.0.1:33709_solr, 127.0.0.1:39340_solr, 
127.0.0.1:40404_solr]
   [junit4]   2> 330555 INFO  (OverseerThreadFactory-102-thread-1) [    ] 
o.a.s.c.a.c.CreateCollectionCmd Create collection 
testNodeWithMultipleReplicasLost
   [junit4]   2> 330555 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 330556 DEBUG 
(OverseerAutoScalingTriggerThread-72395407035596803-127.0.0.1:33709_solr-n_0000000000)
 [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, 
lastZnodeVersion 5
   [junit4]   2> 330556 DEBUG (ScheduledTrigger-100-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with 
currently live nodes: 4
   [junit4]   2> 330601 INFO  (qtp1534579564-267) [n:127.0.0.1:33709_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 330707 INFO  (qtp1082391194-366) [n:127.0.0.1:39340_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 330759 INFO  (qtp1813385384-308) [n:127.0.0.1:40404_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=0
   [junit4]   2> 330875 INFO  (qtp524182178-339) [n:127.0.0.1:45170_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics 
params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
 status=0 QTime=36
   [junit4]   2> 330972 INFO  (OverseerThreadFactory-102-thread-1) [    ] 
o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 5154623997 
sessionWrapper.createTime 5154623901771479, this.sessionWrapper.createTime 
5154623901771479 
   [junit4]   2> 331024 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "collection.configName":"conf",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 331024 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ClusterStateMutator building a new cName: 
testNodeWithMultipleReplicasLost
   [junit4]   2> 331025 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ZkStateWriter going to create_collection 
/collections/testNodeWithMultipleReplicasLost/state.json
   [junit4]   2> 331129 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 331129 INFO  
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 331130 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-ffffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 331130 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-ffffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331131 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 331131 INFO  
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 331132 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-ffffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331133 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-ffffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331134 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 331134 INFO  
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 331152 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-ffffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331152 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-ffffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node7":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>       "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331154 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 331154 INFO  
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 331155 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator Old Slice: shard2:{
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 331155 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator New Slice: shard2:{
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node8":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>       "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331158 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 5, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 331158 INFO  
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 331175 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator Old Slice: shard2:{
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node8":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>       "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331175 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator New Slice: shard2:{
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node8":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>       "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node11":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>       "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331177 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 331177 INFO  
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 331178 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator Old Slice: shard2:{
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node8":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>       "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node11":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>       "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331179 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.SliceMutator New Slice: shard2:{
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node8":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>       "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node11":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>       "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node12":{
   [junit4]   2>       
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>       "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 331279 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ZkStateWriter going to update_collection 
/collections/testNodeWithMultipleReplicasLost/state.json version: 0
   [junit4]   2> 331464 INFO  (qtp1534579564-265) [n:127.0.0.1:33709_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=testNodeWithMultipleReplicasLost_shard2_replica_n6&action=CREATE&numShards=2&collection=testNodeWithMultipleReplicasLost&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 331464 INFO  (qtp1082391194-370) [n:127.0.0.1:39340_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf&newCollection=true&name=testNodeWithMultipleReplicasLost_shard2_replica_n9&action=CREATE&numShards=2&collection=testNodeWithMultipleReplicasLost&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 331465 INFO  (qtp1534579564-265) [n:127.0.0.1:33709_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 331466 INFO  (qtp1813385384-310) [n:127.0.0.1:40404_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&newCollection=true&name=testNodeWithMultipleReplicasLost_shard2_replica_n10&action=CREATE&numShards=2&collection=testNodeWithMultipleReplicasLost&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 331466 INFO  (qtp1082391194-368) [n:127.0.0.1:39340_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=testNodeWithMultipleReplicasLost_shard1_replica_n4&action=CREATE&numShards=2&collection=testNodeWithMultipleReplicasLost&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 331489 INFO  (qtp1813385384-312) [n:127.0.0.1:40404_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=testNodeWithMultipleReplicasLost_shard1_replica_n2&action=CREATE&numShards=2&collection=testNodeWithMultipleReplicasLost&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 331489 INFO  (qtp524182178-340) [n:127.0.0.1:45170_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testNodeWithMultipleReplicasLost_shard1_replica_n1&action=CREATE&numShards=2&collection=testNodeWithMultipleReplicasLost&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 331514 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>   "core_node_name":"core_node11",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 331514 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>   "core_node_name":"core_node11",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 331514 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Will update state for replica: 
core_node11:{"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9","base_url":"https://127.0.0.1:39340/solr","node_name":"127.0.0.1:39340_solr","state":"down","type":"NRT"}
   [junit4]   2> 331514 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(testNodeWithMultipleReplicasLost//collections/testNodeWithMultipleReplicasLost/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard1":{
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node7":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}},
   [junit4]   2>     "shard2":{
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node8":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node11":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node12":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 331514 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node7",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 331514 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node7",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 331514 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Will update state for replica: 
core_node7:{"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4","base_url":"https://127.0.0.1:39340/solr","node_name":"127.0.0.1:39340_solr","state":"down","type":"NRT"}
   [junit4]   2> 331515 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(testNodeWithMultipleReplicasLost//collections/testNodeWithMultipleReplicasLost/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard1":{
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node7":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}},
   [junit4]   2>     "shard2":{
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node8":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node11":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node12":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 331536 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>   "core_node_name":"core_node8",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:33709_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 331536 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>   "core_node_name":"core_node8",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:33709_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 331536 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Will update state for replica: 
core_node8:{"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6","base_url":"https://127.0.0.1:33709/solr","node_name":"127.0.0.1:33709_solr","state":"down","type":"NRT"}
   [junit4]   2> 331549 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(testNodeWithMultipleReplicasLost//collections/testNodeWithMultipleReplicasLost/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard1":{
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node7":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}},
   [junit4]   2>     "shard2":{
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node8":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:33709_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node11":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node12":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 331549 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:45170_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 331549 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:45170_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 331549 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Will update state for replica: 
core_node3:{"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1","base_url":"https://127.0.0.1:45170/solr","node_name":"127.0.0.1:45170_solr","state":"down","type":"NRT"}
   [junit4]   2> 331549 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(testNodeWithMultipleReplicasLost//collections/testNodeWithMultipleReplicasLost/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard1":{
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:45170_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node7":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}},
   [junit4]   2>     "shard2":{
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node8":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:33709_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node11":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node12":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:40404_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:40404_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Will update state for replica: 
core_node5:{"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2","base_url":"https://127.0.0.1:40404/solr","node_name":"127.0.0.1:40404_solr","state":"down","type":"NRT"}
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(testNodeWithMultipleReplicasLost//collections/testNodeWithMultipleReplicasLost/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard1":{
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:45170_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:40404_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node7":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}},
   [junit4]   2>     "shard2":{
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node8":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:33709_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node11":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node12":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>   "core_node_name":"core_node12",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:40404_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"testNodeWithMultipleReplicasLost_shard2_replica_n10",
   [junit4]   2>   "core_node_name":"core_node12",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>   "node_name":"127.0.0.1:40404_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testNodeWithMultipleReplicasLost",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Will update state for replica: 
core_node12:{"core":"testNodeWithMultipleReplicasLost_shard2_replica_n10","base_url":"https://127.0.0.1:40404/solr","node_name":"127.0.0.1:40404_solr","state":"down","type":"NRT"}
   [junit4]   2> 331551 DEBUG 
(OverseerStateUpdate-72395407035596803-127.0.0.1:33709_solr-n_0000000000) [    
] o.a.s.c.o.ReplicaMutator Collection is now: 
DocCollection(testNodeWithMultipleReplicasLost//collections/testNodeWithMultipleReplicasLost/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard1":{
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45170/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:45170_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:40404/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:40404_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node7":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}},
   [junit4]   2>     "shard2":{
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node8":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "base_url":"https://127.0.0.1:33709/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:33709_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node11":{
   [junit4]   2>           
"core":"testNodeWithMultipleReplicasLost_shard2_replica_n9",
   [junit4]   2>           "base_url":"https://127.0.0.1:39340/solr";,
   [junit4]   2>           "node_name":"127.0.0.1:39340_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node12":{
   [junit4]   2>  

[...truncated too long message...]

.0.0.1:34064/_/w
   [junit4]   2> 3698458 WARN  
(TEST-HttpPartitionTest.test-seed#[AE5AB9149E50D967]) [    ] 
o.a.s.c.SocketProxy Closing 6 connections to: http://127.0.0.1:44132/_/w, 
target: http://127.0.0.1:46758/_/w
   [junit4]   2> 3698458 WARN  
(TEST-HttpPartitionTest.test-seed#[AE5AB9149E50D967]) [    ] 
o.a.s.c.SocketProxy Closing 12 connections to: http://127.0.0.1:36174/_/w, 
target: http://127.0.0.1:41526/_/w
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HttpPartitionTest 
-Dtests.method=test -Dtests.seed=AE5AB9149E50D967 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=hr 
-Dtests.timezone=Asia/Kuwait -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   67.8s J0 | HttpPartitionTest.test <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: Could not 
load collection from ZK: collMinRf_1x3
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([AE5AB9149E50D967:260E86CE30ACB49F]:0)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1250)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:679)
   [junit4]    >        at 
org.apache.solr.common.cloud.ClusterState$CollectionRef.get(ClusterState.java:386)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1208)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
   [junit4]    >        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
   [junit4]    >        at 
org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:694)
   [junit4]    >        at 
org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:681)
   [junit4]    >        at 
org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:277)
   [junit4]    >        at 
org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:138)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: 
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /collections/collMinRf_1x3/state.json
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]    >        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1262)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1248)
   [junit4]    >        ... 50 more
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_AE5AB9149E50D967-001
   [junit4]   2> Apr 04, 2018 5:43:10 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene70, 
sim=RandomSimilarity(queryNorm=false): {}, locale=hr, timezone=Asia/Kuwait
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 
1.8.0_152 (64-bit)/cpus=4,threads=1,free=272122824,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [V2StandaloneTest, 
TestHalfAndHalfDocValues, NodeMarkersRegistrationTest, TestDocumentBuilder, 
IndexBasedSpellCheckerTest, TestAuthenticationFramework, 
ConfigureRecoveryStrategyTest, TestSolrQueryParser, 
ChaosMonkeyNothingIsSafeWithPullReplicasTest, ReplicationFactorTest, 
BlobRepositoryCloudTest, TestLRUStatsCache, CoreAdminOperationTest, 
TestSolr4Spatial2, TestSerializedLuceneMatchVersion, 
TestUnifiedSolrHighlighter, TestApiFramework, SSLMigrationTest, 
UUIDUpdateProcessorFallbackTest, TestSubQueryTransformer, TestBulkSchemaAPI, 
ImplicitSnitchTest, TestDistribIDF, TestSolrCloudSnapshots, 
TestNodeAddedTrigger, TestHdfsUpdateLog, TestConfig, TestPseudoReturnFields, 
PreAnalyzedUpdateProcessorTest, TestImpersonationWithHadoopAuth, 
ZkNodePropsTest, OverseerModifyCollectionTest, TestSearcherReuse, 
TestRemoteStreaming, HdfsDirectoryTest, DistributedSuggestComponentTest, 
TestFileDictionaryLookup, TestHashPartitioner, TestClassicSimilarityFactory, 
TestComponentsName, BasicDistributedZkTest, RegexBytesRefFilterTest, 
ShardSplitTest, TestMergePolicyConfig, TestIBSimilarityFactory, 
TestSimDistributedQueue, TestTolerantUpdateProcessorRandomCloud, BitVectorTest, 
TestClusterProperties, OverseerTaskQueueTest, HdfsRecoverLeaseTest, 
TestStandardQParsers, TestXmlQParser, DistribJoinFromCollectionTest, 
TermVectorComponentDistributedTest, CollectionTooManyReplicasTest, 
TestSimpleTextCodec, SolrSlf4jReporterTest, TestCloudInspectUtil, 
TestIndexSearcher, TestWriterPerf, ConfigSetsAPITest, TestOmitPositions, 
TestSchemalessBufferedUpdates, TestSkipOverseerOperations, 
WrapperMergePolicyFactoryTest, TokenizerChainTest, CdcrReplicationHandlerTest, 
OutOfBoxZkACLAndCredentialsProvidersTest, DistributedFacetPivotWhiteBoxTest, 
CollectionStateFormat2Test, TestExecutePlanAction, SpellCheckCollatorTest, 
TestClusterStateMutator, TestStressLiveNodes, AutoAddReplicasPlanActionTest, 
TestMissingGroups, TestJsonFacetRefinement, TestBackupRepositoryFactory, 
TestPHPSerializedResponseWriter, UpdateParamsTest, 
TestCollectionsAPIViaSolrCloudCluster, TestComplexPhraseQParserPlugin, 
SuggesterWFSTTest, UpdateLogTest, ZkShardTermsTest, TestInitQParser, 
TestCustomStream, TestOverriddenPrefixQueryForCustomFieldType, PluginInfoTest, 
ChaosMonkeyNothingIsSafeTest, DocValuesNotIndexedTest, SolrIndexConfigTest, 
TestDistributedSearch, OutputWriterTest, CursorPagingTest, 
TestSolrDeletionPolicy1, BJQParserTest, SolrCoreCheckLockOnStartupTest, 
TestDocSet, OpenExchangeRatesOrgProviderTest, EchoParamsTest, ZkSolrClientTest, 
TestJoin, LeaderElectionContextKeyTest, TestCollationField, 
TestReplicationHandlerBackup, SolrIndexMetricsTest, TestManagedResource, 
TestClusterStateProvider, TestPullReplicaErrorHandling, SimpleFacetsTest, 
TestSolrConfigHandlerConcurrent, TestOnReconnectListenerSupport, 
TestHighlightDedupGrouping, TestPerFieldSimilarityWithDefaultOverride, 
XmlUpdateRequestHandlerTest, TestPhraseSuggestions, 
TestWordDelimiterFilterFactory, BlockJoinFacetSimpleTest, 
ExecutePlanActionTest, TestSort, DocumentAnalysisRequestHandlerTest, 
DistributedExpandComponentTest, SolrRequestParserTest, HdfsSyncSliceTest, 
TestRebalanceLeaders, BlockCacheTest, ResponseBuilderTest, 
TestReplicationHandler, TestZkChroot, TestRandomDVFaceting, SolrCoreTest, 
SpellCheckComponentTest, TestGroupingSearch, SolrCmdDistributorTest, 
BadIndexSchemaTest, ConvertedLegacyTest, DirectUpdateHandlerTest, 
TestBadConfig, HighlighterTest, ShowFileRequestHandlerTest, 
DistributedQueryElevationComponentTest, SolrIndexSplitterTest, 
SimplePostToolTest, AnalysisAfterCoreReloadTest, SuggesterTSTTest, 
SuggesterTest, TestStressLucene, SpatialFilterTest, 
WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, 
DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, 
DocValuesMultiTest, SolrInfoBeanTest, DebugComponentTest, 
LukeRequestHandlerTest, RequestHandlersTest, TestIndexingPerformance, 
RequiredFieldsTest, FastVectorHighlighterTest, LoggingHandlerTest, 
RegexBoostProcessorTest, SolrPluginUtilsTest, TestJmxIntegration, 
TestCSVResponseWriter, BinaryUpdateRequestHandlerTest, 
TestPartialUpdateDeduplication, CSVRequestHandlerTest, HighlighterConfigTest, 
TestQuerySenderListener, TestSolrIndexConfig, TestQuerySenderNoQuery, 
ResponseLogComponentTest, BadComponentTest, MultiTermTest, TestBinaryField, 
TestSearchPerf, TestElisionMultitermQuery, ExternalFileFieldSortTest, 
DirectSolrConnectionTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, 
TestCharFilters, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, 
TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestFastWriter, 
TimeZoneUtilsTest, ResourceLoaderTest, TestFastOutputStream, 
PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, 
TestSuggestSpellingConverter, SpellingQueryConverterTest, 
RAMDirectoryFactoryTest, ClusterStateTest, SliceStateTest, UUIDFieldTest, 
CircularListTest, DistributedMLTComponentTest, 
TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, 
ConnectionReuseTest, ActionThrottleTest, AliasIntegrationTest, DeleteNodeTest, 
DeleteReplicaTest, HttpPartitionTest]
   [junit4] Completed [536/794 (5!)] on J0 in 68.71s, 1 test, 1 error <<< 
FAILURES!

[...truncated 127 lines...]
ERROR: command execution failed.
ERROR: Step ‘Archive the artifacts’ failed: no workspace for 
Lucene-Solr-BadApples-Tests-7.x #31
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for 
Lucene-Solr-BadApples-Tests-7.x #31
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org