Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/19259/
Java: 64bit/jdk1.8.0_121 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.OnlyLeaderIndexesTest.test

Error Message:
Some replicas are not in sync with leader

Stack Trace:
java.lang.AssertionError: Some replicas are not in sync with leader
        at 
__randomizedtesting.SeedInfo.seed([F585CE502594945D:7DD1F18A8B68F9A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.OnlyLeaderIndexesTest.checkShardConsistency(OnlyLeaderIndexesTest.java:368)
        at 
org.apache.solr.cloud.OnlyLeaderIndexesTest.dbiTest(OnlyLeaderIndexesTest.java:220)
        at 
org.apache.solr.cloud.OnlyLeaderIndexesTest.test(OnlyLeaderIndexesTest.java:76)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11044 lines...]
   [junit4] Suite: org.apache.solr.cloud.OnlyLeaderIndexesTest
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/init-core-data-001
   [junit4]   2> 34889 WARN  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 34889 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using TrieFields
   [junit4]   2> 34891 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 34892 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001
   [junit4]   2> 34892 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 34892 INFO  (Thread-101) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 34892 INFO  (Thread-101) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 34992 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.ZkTestServer start zk server on port:45708
   [junit4]   2> 34998 INFO  (jetty-launcher-70-thread-1) [    ] o.e.j.s.Server 
jetty-9.3.14.v20161028
   [junit4]   2> 34999 INFO  (jetty-launcher-70-thread-2) [    ] o.e.j.s.Server 
jetty-9.3.14.v20161028
   [junit4]   2> 34999 INFO  (jetty-launcher-70-thread-3) [    ] o.e.j.s.Server 
jetty-9.3.14.v20161028
   [junit4]   2> 35054 INFO  (jetty-launcher-70-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@13a24cbb{/solr,null,AVAILABLE}
   [junit4]   2> 35055 INFO  (jetty-launcher-70-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@55636a3a{/solr,null,AVAILABLE}
   [junit4]   2> 35055 INFO  (jetty-launcher-70-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@46a882cb{/solr,null,AVAILABLE}
   [junit4]   2> 35058 INFO  (jetty-launcher-70-thread-3) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@141074b0{SSL,[ssl, 
http/1.1]}{127.0.0.1:42147}
   [junit4]   2> 35058 INFO  (jetty-launcher-70-thread-1) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@5d87e4ce{SSL,[ssl, 
http/1.1]}{127.0.0.1:38900}
   [junit4]   2> 35058 INFO  (jetty-launcher-70-thread-3) [    ] o.e.j.s.Server 
Started @36810ms
   [junit4]   2> 35058 INFO  (jetty-launcher-70-thread-1) [    ] o.e.j.s.Server 
Started @36810ms
   [junit4]   2> 35058 INFO  (jetty-launcher-70-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=42147}
   [junit4]   2> 35058 INFO  (jetty-launcher-70-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=38900}
   [junit4]   2> 35059 ERROR (jetty-launcher-70-thread-3) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 35059 ERROR (jetty-launcher-70-thread-1) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-2) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@4206a67e{SSL,[ssl, 
http/1.1]}{127.0.0.1:46418}
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-2) [    ] o.e.j.s.Server 
Started @36811ms
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=46418}
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-03-25T20:57:52.880Z
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-03-25T20:57:52.880Z
   [junit4]   2> 35059 ERROR (jetty-launcher-70-thread-2) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 35059 INFO  (jetty-launcher-70-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-03-25T20:57:52.880Z
   [junit4]   2> 35062 INFO  (jetty-launcher-70-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 35063 INFO  (jetty-launcher-70-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 35063 INFO  (jetty-launcher-70-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 35074 INFO  (jetty-launcher-70-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45708/solr
   [junit4]   2> 35078 INFO  (jetty-launcher-70-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45708/solr
   [junit4]   2> 35079 INFO  (jetty-launcher-70-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45708/solr
   [junit4]   2> 35111 INFO  (jetty-launcher-70-thread-1) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 35112 INFO  (jetty-launcher-70-thread-2) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 35111 INFO  (jetty-launcher-70-thread-3) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 35112 INFO  (jetty-launcher-70-thread-1) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:38900_solr
   [junit4]   2> 35113 INFO  (jetty-launcher-70-thread-2) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:46418_solr
   [junit4]   2> 35113 INFO  (jetty-launcher-70-thread-3) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:42147_solr
   [junit4]   2> 35113 INFO  (jetty-launcher-70-thread-1) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.Overseer Overseer 
(id=97679800579850248-127.0.0.1:38900_solr-n_0000000000) starting
   [junit4]   2> 35114 INFO  
(zkCallback-85-thread-1-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 35114 INFO  
(zkCallback-86-thread-1-processing-n:127.0.0.1:42147_solr) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 35116 INFO  
(zkCallback-87-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 35122 INFO  (jetty-launcher-70-thread-1) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:38900_solr
   [junit4]   2> 35123 INFO  
(zkCallback-86-thread-1-processing-n:127.0.0.1:42147_solr) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 35123 INFO  
(zkCallback-87-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 35123 INFO  
(zkCallback-85-thread-1-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 35163 INFO  (jetty-launcher-70-thread-2) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node2/.
   [junit4]   2> 35204 INFO  (jetty-launcher-70-thread-3) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node3/.
   [junit4]   2> 35204 INFO  (jetty-launcher-70-thread-1) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node1/.
   [junit4]   2> 35227 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 35227 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45708/solr ready
   [junit4]   2> 35285 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=3&collection.configName=config&maxShardsPerNode=1&name=collection1&realtimeReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 35288 INFO  
(OverseerThreadFactory-161-thread-1-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
collection1
   [junit4]   2> 35404 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=config&newCollection=true&name=collection1_shard1_replica2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2
   [junit4]   2> 35445 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=config&newCollection=true&name=collection1_shard1_replica1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2
   [junit4]   2> 35456 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=config&newCollection=true&name=collection1_shard1_replica3&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2
   [junit4]   2> 35559 INFO  
(zkCallback-86-thread-1-processing-n:127.0.0.1:42147_solr) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 35559 INFO  
(zkCallback-87-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 35559 INFO  
(zkCallback-85-thread-2-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 36423 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 7.0.0
   [junit4]   2> 36432 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.s.IndexSchema 
[collection1_shard1_replica2] Schema name=minimal
   [junit4]   2> 36435 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.s.IndexSchema 
Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 36436 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1_shard1_replica2' using configuration from 
collection collection1
   [junit4]   2> 36436 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.SolrCore 
solr.RecoveryStrategy.Builder
   [junit4]   2> 36436 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.SolrCore 
[[collection1_shard1_replica2] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node3/collection1_shard1_replica2],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node3/./collection1_shard1_replica2/data/]
   [junit4]   2> 36459 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 7.0.0
   [junit4]   2> 36466 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 7.0.0
   [junit4]   2> 36470 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.s.IndexSchema 
[collection1_shard1_replica1] Schema name=minimal
   [junit4]   2> 36473 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.s.IndexSchema 
Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 36474 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1_shard1_replica1' using configuration from 
collection collection1
   [junit4]   2> 36475 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.c.SolrCore 
solr.RecoveryStrategy.Builder
   [junit4]   2> 36475 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.c.SolrCore 
[[collection1_shard1_replica1] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node1/collection1_shard1_replica1],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node1/./collection1_shard1_replica1/data/]
   [junit4]   2> 36476 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.s.IndexSchema 
[collection1_shard1_replica3] Schema name=minimal
   [junit4]   2> 36478 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.s.IndexSchema 
Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 36479 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1_shard1_replica3' using configuration from 
collection collection1
   [junit4]   2> 36479 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.c.SolrCore 
solr.RecoveryStrategy.Builder
   [junit4]   2> 36479 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.c.SolrCore 
[[collection1_shard1_replica3] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node2/collection1_shard1_replica3],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node2/./collection1_shard1_replica3/data/]
   [junit4]   2> 36589 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.u.UpdateHandler 
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 36590 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 36592 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.u.CommitTracker 
Hard AutoCommit: disabled
   [junit4]   2> 36592 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.u.CommitTracker 
Soft AutoCommit: disabled
   [junit4]   2> 36595 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@36a16fba[collection1_shard1_replica2] main]
   [junit4]   2> 36597 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/config
   [junit4]   2> 36597 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 36597 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 36598 INFO  
(searcherExecutor-168-thread-1-processing-n:127.0.0.1:42147_solr 
x:collection1_shard1_replica2 s:shard1 c:collection1) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.SolrCore 
[collection1_shard1_replica2] Registered new searcher 
Searcher@36a16fba[collection1_shard1_replica2] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 36598 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.u.UpdateLog Could 
not find max version in index or recent updates, using new clock 
1562876811064377344
   [junit4]   2> 36603 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for 
shard shard1: total=3 found=1 timeoutin=9999ms
   [junit4]   2> 36615 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.u.UpdateHandler 
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 36615 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 36626 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.u.CommitTracker 
Hard AutoCommit: disabled
   [junit4]   2> 36626 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.u.CommitTracker 
Soft AutoCommit: disabled
   [junit4]   2> 36628 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1856970f[collection1_shard1_replica1] main]
   [junit4]   2> 36629 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/config
   [junit4]   2> 36630 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 36631 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 36631 INFO  
(searcherExecutor-169-thread-1-processing-n:127.0.0.1:38900_solr 
x:collection1_shard1_replica1 s:shard1 c:collection1) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.c.SolrCore 
[collection1_shard1_replica1] Registered new searcher 
Searcher@1856970f[collection1_shard1_replica1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 36631 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.u.UpdateLog Could 
not find max version in index or recent updates, using new clock 
1562876811098980352
   [junit4]   2> 36707 INFO  
(zkCallback-87-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 36707 INFO  
(zkCallback-86-thread-1-processing-n:127.0.0.1:42147_solr) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 36707 INFO  
(zkCallback-85-thread-3-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 36711 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.u.UpdateHandler 
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 36711 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 36713 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.u.CommitTracker 
Hard AutoCommit: disabled
   [junit4]   2> 36713 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.u.CommitTracker 
Soft AutoCommit: disabled
   [junit4]   2> 36716 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@4a99d5f2[collection1_shard1_replica3] main]
   [junit4]   2> 36717 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/config
   [junit4]   2> 36718 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 36718 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 36718 INFO  
(searcherExecutor-170-thread-1-processing-n:127.0.0.1:46418_solr 
x:collection1_shard1_replica3 s:shard1 c:collection1) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.c.SolrCore 
[collection1_shard1_replica3] Registered new searcher 
Searcher@4a99d5f2[collection1_shard1_replica3] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 36718 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.u.UpdateLog Could 
not find max version in index or recent updates, using new clock 
1562876811190206464
   [junit4]   2> 37104 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 37104 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 37104 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.SyncStrategy 
Sync replicas to https://127.0.0.1:42147/solr/collection1_shard1_replica2/
   [junit4]   2> 37104 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.u.PeerSync 
PeerSync: core=collection1_shard1_replica2 url=https://127.0.0.1:42147/solr 
START replicas=[https://127.0.0.1:38900/solr/collection1_shard1_replica1/, 
https://127.0.0.1:46418/solr/collection1_shard1_replica3/] nUpdates=100
   [junit4]   2> 37105 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.u.PeerSync 
PeerSync: core=collection1_shard1_replica2 url=https://127.0.0.1:42147/solr 
DONE.  We have no versions.  sync failed.
   [junit4]   2> 37122 INFO  (qtp1775638443-458) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2}
 status=0 QTime=6
   [junit4]   2> 37123 INFO  (qtp1579454041-434) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2}
 status=0 QTime=7
   [junit4]   2> 37124 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.SyncStrategy 
Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 37124 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we 
can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 37124 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 37127 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:42147/solr/collection1_shard1_replica2/ shard1
   [junit4]   2> 37230 INFO  
(zkCallback-87-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37230 INFO  
(zkCallback-86-thread-1-processing-n:127.0.0.1:42147_solr) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37230 INFO  
(zkCallback-85-thread-2-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37279 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.c.ZkController I 
am the leader, no recovery necessary
   [junit4]   2> 37281 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1  x:collection1_shard1_replica2] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=config&newCollection=true&name=collection1_shard1_replica2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2}
 status=0 QTime=1877
   [junit4]   2> 37382 INFO  
(zkCallback-85-thread-3-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37382 INFO  
(zkCallback-86-thread-1-processing-n:127.0.0.1:42147_solr) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37382 INFO  
(zkCallback-87-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37650 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 37650 WARN  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.ReplicationHandler SolrCloud is enabled for core 
collection1_shard1_replica1 but so is old-style replication. Make sure you 
intend this behavior, it usually indicates a mis-configuration. Master setting 
is false and slave setting is true
   [junit4]   2> 37650 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Updated masterUrl to 
https://127.0.0.1:42147/solr/collection1_shard1_replica2/
   [junit4]   2> 37653 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=config&newCollection=true&name=collection1_shard1_replica1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2}
 status=0 QTime=2208
   [junit4]   2> 37656 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 
QTime=0
   [junit4]   2> 37657 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 37657 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 37657 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 37657 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 37723 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 37723 WARN  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.ReplicationHandler SolrCloud is enabled for core 
collection1_shard1_replica3 but so is old-style replication. Make sure you 
intend this behavior, it usually indicates a mis-configuration. Master setting 
is false and slave setting is true
   [junit4]   2> 37725 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.s.HttpSolrCall 
[admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=config&newCollection=true&name=collection1_shard1_replica3&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2}
 status=0 QTime=2269
   [junit4]   2> 37727 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr    ] 
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 37826 INFO  
(zkCallback-86-thread-1-processing-n:127.0.0.1:42147_solr) 
[n:127.0.0.1:42147_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37826 INFO  
(zkCallback-87-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 37826 INFO  
(zkCallback-85-thread-2-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [3])
   [junit4]   2> 38727 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=3&collection.configName=config&maxShardsPerNode=1&name=collection1&realtimeReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2}
 status=0 QTime=3442
   [junit4]   2> 38728 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: 
collection1 failOnTimeout:true timeout (sec):30
   [junit4]   2> 38729 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 38753 INFO  
(TEST-OnlyLeaderIndexesTest.test-seed#[F585CE502594945D]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 38780 INFO  (qtp1579454041-450) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica1]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{add=[1
 (1562876813329301504), 2 (1562876813342932992), 3 (1562876813342932993), 4 
(1562876813343981568)]} 0 1
   [junit4]   2> 38782 INFO  (qtp1775638443-456) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica3]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{add=[1
 (1562876813329301504), 2 (1562876813342932992), 3 (1562876813342932993), 4 
(1562876813343981568)]} 0 2
   [junit4]   2> 38782 INFO  (qtp1500396170-438) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]  webapp=/solr 
path=/update params={wt=javabin&version=2}{add=[1 (1562876813329301504), 2 
(1562876813342932992), 3 (1562876813342932993), 4 (1562876813343981568)]} 0 24
   [junit4]   2> 38795 INFO  (qtp1775638443-441) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 38798 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38800 INFO  (qtp1775638443-456) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38803 INFO  (qtp1775638443-458) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 38813 INFO  (qtp1500396170-438) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38818 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 38822 INFO  (qtp1500396170-518) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38827 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38836 INFO  (qtp1579454041-433) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38839 INFO  (qtp1579454041-450) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38840 INFO  (qtp1579454041-434) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38841 INFO  (qtp1579454041-434) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38847 INFO  (qtp1579454041-451) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica1]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{delete=[1
 (-1562876813418430464)]} 0 0
   [junit4]   2> 38848 INFO  (qtp1775638443-439) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica3]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{delete=[1
 (-1562876813418430464)]} 0 2
   [junit4]   2> 38849 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]  webapp=/solr 
path=/update params={wt=javabin&version=2}{delete=[1 (-1562876813418430464)]} 0 
5
   [junit4]   2> 38880 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@945d6cb[collection1_shard1_replica2] realtime]
   [junit4]   2> 38883 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@3c5b3f2f[collection1_shard1_replica2] realtime]
   [junit4]   2> 38886 INFO  (qtp1579454041-437) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica1]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&_version_=-1562876813426819072&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{deleteByQuery=id:2
 (-1562876813426819072)} 0 0
   [junit4]   2> 38889 INFO  (qtp1775638443-458) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica3]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&_version_=-1562876813426819072&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{deleteByQuery=id:2
 (-1562876813426819072)} 0 3
   [junit4]   2> 38890 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]  webapp=/solr 
path=/update params={wt=javabin&version=2}{deleteByQuery=id:2 
(-1562876813426819072)} 0 38
   [junit4]   2> 38901 INFO  (qtp1775638443-442) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38903 INFO  (qtp1775638443-442) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38904 INFO  (qtp1775638443-441) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38932 INFO  (qtp1579454041-447) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38935 INFO  (qtp1579454041-451) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38936 INFO  (qtp1579454041-437) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/get 
params={q=*:*&distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 38948 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.DirectUpdateHandler2 start 
commit{_version_=1562876813528530944,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 38950 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@2dc99b70 
commitCommandVersion:1562876813528530944
   [junit4]   2> 38951 INFO  (qtp1500396170-438) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 38956 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 39196 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@705b57e[collection1_shard1_replica2] main]
   [junit4]   2> 39218 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 39218 INFO  
(searcherExecutor-168-thread-1-processing-n:127.0.0.1:42147_solr 
x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node1) 
[n:127.0.0.1:42147_solr c:collection1 s:shard1 r:core_node1 
x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] 
Registered new searcher Searcher@705b57e[collection1_shard1_replica2] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C4/2:delGen=2)))}
   [junit4]   2> 39218 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 273
   [junit4]   2> 39290 INFO  
(OverseerCollectionConfigSetProcessor-97679800579850248-127.0.0.1:38900_solr-n_0000000000)
 [n:127.0.0.1:38900_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 39465 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 39468 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
   [junit4]   2> 39974 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 39975 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
   [junit4]   2> 40485 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 40486 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=2
   [junit4]   2> 40526 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Updated masterUrl to 
https://127.0.0.1:42147/solr/collection1_shard1_replica2/
   [junit4]   2> 40532 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 
QTime=0
   [junit4]   2> 40532 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 40532 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Master's version: 1490475476771
   [junit4]   2> 40533 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 40533 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 40533 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 40539 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} 
status=0 QTime=5
   [junit4]   2> 40540 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Number of files in latest index in master: 16
   [junit4]   2> 40554 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Starting download (fullCopy=false) to 
MMapDirectory@/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node2/collection1_shard1_replica3/data/index.20170326045758362
 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68b2dad4
   [junit4]   2> 40571 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_2.liv&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=3
   [junit4]   2> 40583 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40611 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40639 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.tim&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40646 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Direct_0.dvdd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40650 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Updated masterUrl to 
https://127.0.0.1:42147/solr/collection1_shard1_replica2/
   [junit4]   2> 40651 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 
QTime=0
   [junit4]   2> 40651 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 40651 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Master's version: 1490475476771
   [junit4]   2> 40651 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 40651 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 40651 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 40673 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.pos&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40678 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} 
status=0 QTime=26
   [junit4]   2> 40679 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40679 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Number of files in latest index in master: 16
   [junit4]   2> 40688 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Starting download (fullCopy=false) to 
MMapDirectory@/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node1/collection1_shard1_replica1/data/index.20170326045758500
 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68b2dad4
   [junit4]   2> 40689 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40705 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_2.liv&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40706 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.tip&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40715 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40721 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=2
   [junit4]   2> 40727 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Memory_0.mdvd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40728 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40765 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.tim&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40768 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Memory_0.mdvm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40778 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Direct_0.dvdd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40781 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40790 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.pos&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40793 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Direct_0.dvdm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40813 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40816 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40835 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40839 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40843 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Total time taken for download 
(fullCopy=false,bytesDownloaded=2372) : 0 secs (null bytes/sec) to 
MMapDirectory@/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node2/collection1_shard1_replica3/data/index.20170326045758362
 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68b2dad4
   [junit4]   2> 40848 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Lucene50_0.tip&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40855 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40862 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Memory_0.mdvd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40871 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Memory_0.mdvm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40877 INFO  (qtp1500396170-445) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40907 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0_Direct_0.dvdm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40931 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40939 INFO  (qtp1500396170-517) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 40943 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.h.IndexFetcher Total time taken for download 
(fullCopy=false,bytesDownloaded=2372) : 0 secs (null bytes/sec) to 
MMapDirectory@/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node1/collection1_shard1_replica1/data/index.20170326045758500
 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68b2dad4
   [junit4]   2> 40994 INFO  (qtp1500396170-544) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=2
   [junit4]   2> 40995 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
   [junit4]   2> 41502 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 41503 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=2
   [junit4]   2> 41503 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica3]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 2563
   [junit4]   2> 41509 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 41511 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@57f88fb2[collection1_shard1_replica3] main]
   [junit4]   2> 41512 INFO  
(searcherExecutor-170-thread-1-processing-n:127.0.0.1:46418_solr 
x:collection1_shard1_replica3 s:shard1 c:collection1) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1  x:collection1_shard1_replica3] o.a.s.c.SolrCore 
[collection1_shard1_replica3] Registered new searcher 
Searcher@57f88fb2[collection1_shard1_replica3] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C4/2:delGen=2)))}
   [junit4]   2> 41688 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 41690 INFO  (indexFetcher-184-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1  x:collection1_shard1_replica1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@649f0e9b[collection1_shard1_replica1] main]
   [junit4]   2> 41690 INFO  
(searcherExecutor-169-thread-1-processing-n:127.0.0.1:38900_solr 
x:collection1_shard1_replica1 s:shard1 c:collection1) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1  x:collection1_shard1_replica1] o.a.s.c.SolrCore 
[collection1_shard1_replica1] Registered new searcher 
Searcher@649f0e9b[collection1_shard1_replica1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C4/2:delGen=2)))}
   [junit4]   2> 42010 INFO  (qtp1500396170-438) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 42010 INFO  (qtp1579454041-434) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica1]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 3068
   [junit4]   2> 42011 INFO  (qtp1500396170-518) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]  webapp=/solr 
path=/update 
params={_stateVer_=collection1:5&commit=true&wt=javabin&version=2}{commit=} 0 
3073
   [junit4]   2> 42016 INFO  (qtp1775638443-436) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.c.S.Request [collection1_shard1_replica3]  webapp=/solr path=/select 
params={q=*:*&distrib=false&wt=javabin&version=2} hits=2 status=0 QTime=0
   [junit4]   2> 42023 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/select 
params={q=*:*&distrib=false&wt=javabin&version=2} hits=2 status=0 QTime=0
   [junit4]   2> 42028 INFO  (qtp1579454041-434) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.c.S.Request [collection1_shard1_replica1]  webapp=/solr path=/select 
params={q=*:*&distrib=false&wt=javabin&version=2} hits=2 status=0 QTime=0
   [junit4]   2> 42062 INFO  (qtp1579454041-450) [n:127.0.0.1:38900_solr 
c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica1]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{add=[15
 (1562876816762339328)]} 0 0
   [junit4]   2> 42063 INFO  (qtp1775638443-458) [n:127.0.0.1:46418_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica3] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica3]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&wt=javabin&version=2}{add=[15
 (1562876816762339328)]} 0 1
   [junit4]   2> 42063 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]  webapp=/solr 
path=/update params={wt=javabin&version=2}{add=[15 (1562876816762339328)]} 0 31
   [junit4]   2> 42067 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.DirectUpdateHandler2 start 
commit{_version_=1562876816799039488,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 42067 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@2dc99b70 
commitCommandVersion:1562876816799039488
   [junit4]   2> 42074 INFO  (qtp1500396170-518) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 42076 INFO  (qtp1500396170-444) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 42573 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@3b5ebfec[collection1_shard1_replica2] main]
   [junit4]   2> 42574 INFO  
(searcherExecutor-168-thread-1-processing-n:127.0.0.1:42147_solr 
x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node1) 
[n:127.0.0.1:42147_solr c:collection1 s:shard1 r:core_node1 
x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] 
Registered new searcher Searcher@3b5ebfec[collection1_shard1_replica2] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C4/2:delGen=2)
 Uninverting(_1(7.0.0):C1)))}
   [junit4]   2> 42576 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 42576 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]  webapp=/solr 
path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:42147/solr/collection1_shard1_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 509
   [junit4]   2> 42587 INFO  (qtp1500396170-454) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 42596 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=2
   [junit4]   2> 43097 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
   [junit4]   2> 43111 INFO  (qtp1500396170-518) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=2
   [junit4]   2> 43526 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Updated masterUrl to 
https://127.0.0.1:42147/solr/collection1_shard1_replica2/
   [junit4]   2> 43528 INFO  (qtp1500396170-438) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 
QTime=0
   [junit4]   2> 43529 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Master's generation: 3
   [junit4]   2> 43529 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Master's version: 1490475479888
   [junit4]   2> 43530 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Slave's generation: 2
   [junit4]   2> 43530 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Slave's version: 1490475476771
   [junit4]   2> 43530 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 43535 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=3&qt=/replication&wt=javabin&version=2&command=filelist} 
status=0 QTime=3
   [junit4]   2> 43536 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Number of files in latest index in master: 30
   [junit4]   2> 43542 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Starting download (fullCopy=false) to 
MMapDirectory@/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001/tempDir-001/node2/collection1_shard1_replica3/data/index.20170326045801357
 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68b2dad4
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_2.liv because it already exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0.si because it already exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Lucene50_0.doc because it already 
exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Lucene50_0.tim because it already 
exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Direct_0.dvdd because it already 
exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Lucene50_0.pos because it already 
exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0.nvd because it already exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0.fdx because it already exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Lucene50_0.tip because it already 
exists
   [junit4]   2> 43543 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0.fdt because it already exists
   [junit4]   2> 43544 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Memory_0.mdvd because it already 
exists
   [junit4]   2> 43544 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Memory_0.mdvm because it already 
exists
   [junit4]   2> 43544 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0.nvm because it already exists
   [junit4]   2> 43544 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0_Direct_0.dvdm because it already 
exists
   [junit4]   2> 43544 INFO  (indexFetcher-186-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1  x:collection1_shard1_replica3] 
o.a.s.h.IndexFetcher Skipping download for _0.fnm because it already exists
   [junit4]   2> 43553 INFO  (qtp1500396170-544) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=3&qt=/replication&file=_1_Direct_0.dvdd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 43562 INFO  (qtp1500396170-435) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=3&qt=/replication&file=_1.si&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 43569 INFO  (qtp1500396170-518) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=3&qt=/replication&file=_1.nvd&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 43572 INFO  (qtp1500396170-438) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=3&qt=/replication&file=_1.nvm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 43575 INFO  (qtp1500396170-448) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=3&qt=/replication&file=_1.fnm&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 43579 INFO  (qtp1500396170-544) [n:127.0.0.1:42147_solr 
c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2] 
o.a.s.c.S.Request [collection1_shard1_replica2]  webapp=/solr path=/replication 
params={generation=3&qt=/replication&file=_1_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent}
 status=0 QTime=0
   [junit4]   2> 43598 INFO  (qtp

[...truncated too long message...]

0{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 162498 INFO  (jetty-closer-71-thread-2) [    ] 
o.a.s.c.CoreContainer Shutting down CoreContainer instance=1851465106
   [junit4]   2> 162498 INFO  (jetty-closer-71-thread-2) [    ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, 
tag=null
   [junit4]   2> 162501 INFO  (jetty-closer-71-thread-2) [    ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, 
tag=null
   [junit4]   2> 162506 INFO  (coreCloseExecutor-298-thread-1) 
[n:127.0.0.1:42147_solr c:collection1 s:shard1 r:core_node1 
x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2]  
CLOSING SolrCore org.apache.solr.core.SolrCore@533989d7
   [junit4]   2> 162520 INFO  (coreCloseExecutor-298-thread-1) 
[n:127.0.0.1:42147_solr c:collection1 s:shard1 r:core_node1 
x:collection1_shard1_replica2] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.core.collection1.shard1.replica2, tag=1396279767
   [junit4]   2> 162525 INFO  (jetty-closer-71-thread-2) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@55636a3a{/solr,null,UNAVAILABLE}
   [junit4]   2> 162525 INFO  
(zkCallback-85-thread-4-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (2)
   [junit4]   2> 162526 INFO  
(zkCallback-113-thread-2-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (2)
   [junit4]   2> 162536 WARN  (indexFetcher-292-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1 r:core_node3 
x:collection1_shard1_replica3] o.a.s.h.IndexFetcher Could not log failed 
replication details
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: Already closed
   [junit4]   2>        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:336)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.loadReplicationProperties(ReplicationHandler.java:1102)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:746)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.cleanup(IndexFetcher.java:654)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:636)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:326)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:409)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$2(ReplicationHandler.java:1154)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 162537 ERROR (indexFetcher-292-thread-1) 
[n:127.0.0.1:46418_solr c:collection1 s:shard1 r:core_node3 
x:collection1_shard1_replica3] o.a.s.h.ReplicationHandler Index fetch failed 
:org.apache.solr.common.SolrException: Index fetch failed : 
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:632)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:326)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:409)
   [junit4]   2>        at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$2(ReplicationHandler.java:1154)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2>        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: 
Already closed
   [junit4]   2>        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:336)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:353)
   [junit4]   2>        at 
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:248)
   [junit4]   2>        at 
org.apache.solr.update.DefaultSolrCoreState.changeWriter(DefaultSolrCoreState.java:211)
   [junit4]   2>        at 
org.apache.solr.update.DefaultSolrCoreState.openIndexWriter(DefaultSolrCoreState.java:236)
   [junit4]   2>        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:585)
   [junit4]   2>        ... 10 more
   [junit4]   2> 
   [junit4]   2> 162548 INFO  (coreCloseExecutor-297-thread-1) 
[n:127.0.0.1:38900_solr c:collection1 s:shard1 r:core_node2 
x:collection1_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric 
reporters for registry=solr.core.collection1.shard1.replica1, tag=472398738
   [junit4]   2> 162554 INFO  (jetty-closer-71-thread-3) [    ] 
o.a.s.c.Overseer Overseer 
(id=97679800579850248-127.0.0.1:38900_solr-n_0000000000) closing
   [junit4]   2> 162554 INFO  
(OverseerStateUpdate-97679800579850248-127.0.0.1:38900_solr-n_0000000000) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 
127.0.0.1:38900_solr
   [junit4]   2> 162557 INFO  
(zkCallback-113-thread-1-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:46418_solr
   [junit4]   2> 162559 INFO  
(zkCallback-113-thread-2-processing-n:127.0.0.1:46418_solr) 
[n:127.0.0.1:46418_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (1)
   [junit4]   2> 162591 INFO  (jetty-closer-71-thread-1) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@bcfd36d{/solr,null,UNAVAILABLE}
   [junit4]   2> 164056 WARN  
(zkCallback-85-thread-4-processing-n:127.0.0.1:38900_solr) 
[n:127.0.0.1:38900_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, 
but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 164058 INFO  (jetty-closer-71-thread-3) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@13a24cbb{/solr,null,UNAVAILABLE}
   [junit4]   2> 164059 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:45708 45708
   [junit4]   2> 164081 INFO  (Thread-101) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:45708 45708
   [junit4]   2> 164082 WARN  (Thread-101) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        7       /solr/aliases.json
   [junit4]   2>        6       /solr/security.json
   [junit4]   2>        6       /solr/configs/config
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        7       /solr/clusterstate.json
   [junit4]   2>        7       /solr/clusterprops.json
   [junit4]   2>        6       /solr/collections/collection1/state.json
   [junit4]   2>        4       
/solr/collections/collection1/leader_elect/shard1/election/97679800579850248-core_node2-n_0000000001
   [junit4]   2>        3       
/solr/overseer_elect/election/97679800579850247-127.0.0.1:42147_solr-n_0000000002
   [junit4]   2>        3       
/solr/overseer_elect/election/97679800579850248-127.0.0.1:38900_solr-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        125     /solr/overseer/collection-queue-work
   [junit4]   2>        27      /solr/overseer/queue
   [junit4]   2>        9       /solr/overseer/queue-work
   [junit4]   2>        7       /solr/collections
   [junit4]   2>        4       /solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 164082 INFO  
(SUITE-OnlyLeaderIndexesTest-seed#[F585CE502594945D]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.OnlyLeaderIndexesTest_F585CE502594945D-001
   [junit4]   2> Mar 25, 2017 9:00:01 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{id=Lucene50(blocksize=128)}, 
docValues:{_version_=DocValuesFormat(name=Direct), 
id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=49, 
maxMBSortInHeap=5.90302170784274, sim=RandomSimilarity(queryNorm=true): {}, 
locale=es-CU, timezone=Asia/Hovd
   [junit4]   2> NOTE: Linux 4.4.0-66-generic amd64/Oracle Corporation 
1.8.0_121 (64-bit)/cpus=12,threads=1,free=330505504,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [DateFieldTest, 
PrimUtilsTest, HdfsWriteToMultipleCollectionsTest, 
TriLevelCompositeIdRoutingTest, PrimitiveFieldTypeTest, 
SharedFSAutoReplicaFailoverTest, HighlighterConfigTest, 
TestConfigSetsAPIExclusivity, TestManagedSchemaAPI, TestBinaryField, 
OnlyLeaderIndexesTest]
   [junit4] Completed [60/701 (1!)] on J2 in 129.46s, 1 test, 1 failure <<< 
FAILURES!

[...truncated 45315 lines...]

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to