Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1868/
Java: 32bit/jdk1.8.0_102 -client -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.cloud.CdcrVersionReplicationTest.testCdcrDocVersions

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space


FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
        at 
__randomizedtesting.SeedInfo.seed([50CCC30F4CBE9AA2:A7BF2D578A563544]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1331)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11080 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrVersionReplicationTest
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/init-core-data-001
   [junit4]   2> 172696 INFO  
(SUITE-CdcrVersionReplicationTest-seed#[50CCC30F4CBE9AA2]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 172697 INFO  
(SUITE-CdcrVersionReplicationTest-seed#[50CCC30F4CBE9AA2]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_q/l
   [junit4]   2> 172698 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 172699 INFO  (Thread-364) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 172699 INFO  (Thread-364) [    ] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 172799 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.ZkTestServer start zk server on port:39068
   [junit4]   2> 172809 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 172810 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 172811 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 172812 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 172814 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 172815 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 172816 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 172817 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 172818 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 172819 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 172819 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.AbstractZkTestCase put 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 172823 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.SolrTestCaseJ4 ###Starting testCdcrDocVersions
   [junit4]   2> 172914 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-001/cores/collection1
   [junit4]   2> 172916 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 172917 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@c9753{/_q/l,null,AVAILABLE}
   [junit4]   2> 172921 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.ServerConnector Started ServerConnector@62bdd8{SSL,[ssl, 
http/1.1]}{127.0.0.1:43252}
   [junit4]   2> 172921 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server Started @174316ms
   [junit4]   2> 172921 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, 
hostContext=/_q/l, hostPort=43252, 
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-001/cores}
   [junit4]   2> 172922 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ 
version 6.3.0
   [junit4]   2> 172922 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on 
port null
   [junit4]   2> 172922 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 172922 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-10-04T08:22:54.901Z
   [junit4]   2> 172925 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 172925 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.SolrXmlConfig Loading container configuration from 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-001/solr.xml
   [junit4]   2> 172932 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with 
params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 172933 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39068/solr
   [junit4]   2> 172950 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:43252__q%2Fl
   [junit4]   2> 172951 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.Overseer Overseer 
(id=96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000) starting
   [junit4]   2> 172955 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:43252__q%2Fl
   [junit4]   2> 172956 INFO  
(zkCallback-153-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 172992 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-001/cores
   [junit4]   2> 172992 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 172994 INFO  
(OverseerStateUpdate-96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard2
   [junit4]   2> 173995 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection   x:collection1] 
o.a.s.c.ZkController Only one config set found in zk - using it:conf1
   [junit4]   2> 174006 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 174019 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 174107 WARN  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 174109 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 174125 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection tmp_collection
   [junit4]   2> 174125 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-001/cores/collection1],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-001/cores/collection1/data/]
   [junit4]   2> 174125 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fee053
   [junit4]   2> 174127 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@1a3634a
   [junit4]   2> 174172 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 174172 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 174173 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 174173 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 174174 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=20, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=10.2685546875, 
floorSegmentMB=1.20703125, forceMergeDeletesPctAllowed=17.322828593338066, 
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 174175 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@13fa155[collection1] main]
   [junit4]   2> 174176 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 174176 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 174176 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 174180 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.h.CdcrBufferStateManager Created znode 
/collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 174183 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.h.CdcrProcessStateManager Created znode 
/collections/tmp_collection/cdcr/state/process
   [junit4]   2> 174194 INFO  
(searcherExecutor-527-thread-1-processing-n:127.0.0.1:43252__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@13fa155[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 174195 INFO  
(coreLoadExecutor-526-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1547246634191028224
   [junit4]   2> 174200 INFO  
(coreZkRegister-519-thread-1-processing-n:127.0.0.1:43252__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 174200 INFO  
(coreZkRegister-519-thread-1-processing-n:127.0.0.1:43252__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 174200 INFO  
(coreZkRegister-519-thread-1-processing-n:127.0.0.1:43252__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync 
replicas to https://127.0.0.1:43252/_q/l/collection1/
   [junit4]   2> 174200 INFO  
(coreZkRegister-519-thread-1-processing-n:127.0.0.1:43252__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync 
Success - now sync replicas to me
   [junit4]   2> 174200 INFO  
(coreZkRegister-519-thread-1-processing-n:127.0.0.1:43252__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
https://127.0.0.1:43252/_q/l/collection1/ has no replicas
   [junit4]   2> 174202 INFO  
(zkCallback-153-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.h.CdcrLeaderStateManager Received new 
leader state @ tmp_collection:shard2
   [junit4]   2> 174203 INFO  
(coreZkRegister-519-thread-1-processing-n:127.0.0.1:43252__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:43252/_q/l/collection1/ shard2
   [junit4]   2> 174353 INFO  
(coreZkRegister-519-thread-1-processing-n:127.0.0.1:43252__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43252__q%2Fl 
c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am 
the leader, no recovery necessary
   [junit4]   2> 174568 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-002/cores/collection1
   [junit4]   2> 174570 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 174572 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@c7651a{/_q/l,null,AVAILABLE}
   [junit4]   2> 174575 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.ServerConnector Started ServerConnector@1fb8640{SSL,[ssl, 
http/1.1]}{127.0.0.1:41597}
   [junit4]   2> 174575 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server Started @175971ms
   [junit4]   2> 174575 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, 
hostContext=/_q/l, hostPort=41597, 
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-002/cores}
   [junit4]   2> 174575 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ 
version 6.3.0
   [junit4]   2> 174575 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on 
port null
   [junit4]   2> 174575 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 174575 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-10-04T08:22:56.554Z
   [junit4]   2> 174579 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 174580 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.SolrXmlConfig Loading container configuration from 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-002/solr.xml
   [junit4]   2> 174586 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with 
params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 174587 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39068/solr
   [junit4]   2> 174599 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 174602 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:41597__q%2Fl
   [junit4]   2> 174603 INFO  
(zkCallback-159-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 174603 INFO  
(zkCallback-153-thread-3-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (2)
   [junit4]   2> 174649 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-002/cores
   [junit4]   2> 174649 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 174650 INFO  
(OverseerStateUpdate-96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 175660 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 175672 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 175754 WARN  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 175756 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 175770 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection tmp_collection
   [junit4]   2> 175770 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-002/cores/collection1],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-002/cores/collection1/data/]
   [junit4]   2> 175770 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fee053
   [junit4]   2> 175772 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@1af6549
   [junit4]   2> 175797 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 175797 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 175798 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 175798 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 175799 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=20, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=10.2685546875, 
floorSegmentMB=1.20703125, forceMergeDeletesPctAllowed=17.322828593338066, 
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 175800 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@c5fb1d[collection1] main]
   [junit4]   2> 175800 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 175801 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 175801 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 175803 INFO  
(searcherExecutor-539-thread-1-processing-n:127.0.0.1:41597__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@c5fb1d[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 175803 INFO  
(coreLoadExecutor-538-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1547246635877138432
   [junit4]   2> 175808 INFO  
(coreZkRegister-533-thread-1-processing-n:127.0.0.1:41597__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 175808 INFO  
(coreZkRegister-533-thread-1-processing-n:127.0.0.1:41597__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 175808 INFO  
(coreZkRegister-533-thread-1-processing-n:127.0.0.1:41597__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync 
replicas to https://127.0.0.1:41597/_q/l/collection1/
   [junit4]   2> 175808 INFO  
(coreZkRegister-533-thread-1-processing-n:127.0.0.1:41597__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync 
Success - now sync replicas to me
   [junit4]   2> 175808 INFO  
(coreZkRegister-533-thread-1-processing-n:127.0.0.1:41597__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy 
https://127.0.0.1:41597/_q/l/collection1/ has no replicas
   [junit4]   2> 175810 INFO  
(zkCallback-159-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.h.CdcrLeaderStateManager Received new 
leader state @ tmp_collection:shard1
   [junit4]   2> 175811 INFO  
(coreZkRegister-533-thread-1-processing-n:127.0.0.1:41597__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:41597/_q/l/collection1/ shard1
   [junit4]   2> 175962 INFO  
(coreZkRegister-533-thread-1-processing-n:127.0.0.1:41597__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:41597__q%2Fl 
c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am 
the leader, no recovery necessary
   [junit4]   2> 176226 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-003/cores/collection1
   [junit4]   2> 176228 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 176229 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@14d3fa7{/_q/l,null,AVAILABLE}
   [junit4]   2> 176231 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.ServerConnector Started ServerConnector@9b3ef7{SSL,[ssl, 
http/1.1]}{127.0.0.1:34096}
   [junit4]   2> 176231 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server Started @177627ms
   [junit4]   2> 176231 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard3, 
hostContext=/_q/l, hostPort=34096, 
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-003/cores}
   [junit4]   2> 176232 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ 
version 6.3.0
   [junit4]   2> 176232 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on 
port null
   [junit4]   2> 176232 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 176232 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-10-04T08:22:58.211Z
   [junit4]   2> 176236 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 176236 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.SolrXmlConfig Loading container configuration from 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-003/solr.xml
   [junit4]   2> 176242 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with 
params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 176243 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39068/solr
   [junit4]   2> 176251 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 176253 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:34096__q%2Fl
   [junit4]   2> 176254 INFO  
(zkCallback-153-thread-3-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 176254 INFO  
(zkCallback-165-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 176254 INFO  
(zkCallback-159-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 176288 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-003/cores
   [junit4]   2> 176288 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 176290 INFO  
(OverseerStateUpdate-96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard2
   [junit4]   2> 177300 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 177312 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 177392 WARN  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 177394 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 177406 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection tmp_collection
   [junit4]   2> 177407 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-003/cores/collection1],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-003/cores/collection1/data/]
   [junit4]   2> 177407 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fee053
   [junit4]   2> 177408 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@100e5d8
   [junit4]   2> 177431 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 177431 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 177432 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 177432 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 177433 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=20, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=10.2685546875, 
floorSegmentMB=1.20703125, forceMergeDeletesPctAllowed=17.322828593338066, 
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 177434 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@3f5d06[collection1] main]
   [junit4]   2> 177435 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 177435 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 177435 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 177437 INFO  
(searcherExecutor-551-thread-1-processing-n:127.0.0.1:34096__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@3f5d06[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 177437 INFO  
(coreLoadExecutor-550-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1547246637590511616
   [junit4]   2> 177440 INFO  
(coreZkRegister-545-thread-1-processing-n:127.0.0.1:34096__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core 
needs to recover:collection1
   [junit4]   2> 177441 INFO  
(updateExecutor-162-thread-1-processing-n:127.0.0.1:34096__q%2Fl x:collection1 
s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] 
o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 177441 INFO  
(recoveryExecutor-163-thread-1-processing-n:127.0.0.1:34096__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 177441 INFO  
(recoveryExecutor-163-thread-1-processing-n:127.0.0.1:34096__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
###### startupVersions=[[]]
   [junit4]   2> 177441 INFO  
(recoveryExecutor-163-thread-1-processing-n:127.0.0.1:34096__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Begin buffering updates. core=[collection1]
   [junit4]   2> 177441 INFO  
(recoveryExecutor-163-thread-1-processing-n:127.0.0.1:34096__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog 
Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 177441 INFO  
(recoveryExecutor-163-thread-1-processing-n:127.0.0.1:34096__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Publishing state of core [collection1] as recovering, leader is 
[https://127.0.0.1:43252/_q/l/collection1/] and I am 
[https://127.0.0.1:34096/_q/l/collection1/]
   [junit4]   2> 177443 INFO  
(recoveryExecutor-163-thread-1-processing-n:127.0.0.1:34096__q%2Fl 
x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:34096__q%2Fl 
c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Sending prep recovery command to [https://127.0.0.1:43252/_q/l]; [WaitForState: 
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:34096__q%252Fl&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 177543 INFO  (qtp8155725-1126) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: 
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 177543 INFO  (qtp8155725-1126) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 
(shard2 of tmp_collection) have state: recovering
   [junit4]   2> 177543 INFO  (qtp8155725-1126) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): 
collection=tmp_collection, shard=shard2, thisCore=collection1, 
leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, 
currentState=down, localState=active, nodeName=127.0.0.1:34096__q%2Fl, 
coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: 
core_node3:{"core":"collection1","base_url":"https://127.0.0.1:34096/_q/l","node_name":"127.0.0.1:34096__q%2Fl","state":"down"}
   [junit4]   2> 177877 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-004/cores/collection1
   [junit4]   2> 177879 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 177880 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@bf684d{/_q/l,null,AVAILABLE}
   [junit4]   2> 177883 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.ServerConnector Started ServerConnector@c4f0c4{SSL,[ssl, 
http/1.1]}{127.0.0.1:32820}
   [junit4]   2> 177883 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.Server Started @179279ms
   [junit4]   2> 177883 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard4, 
hostContext=/_q/l, hostPort=32820, 
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-004/cores}
   [junit4]   2> 177883 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ 
version 6.3.0
   [junit4]   2> 177883 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on 
port null
   [junit4]   2> 177884 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 177884 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-10-04T08:22:59.863Z
   [junit4]   2> 177887 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 177887 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.SolrXmlConfig Loading container configuration from 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-004/solr.xml
   [junit4]   2> 177897 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with 
params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 177898 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39068/solr
   [junit4]   2> 177907 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:32820__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 177910 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:32820__q%2Fl    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:32820__q%2Fl
   [junit4]   2> 177911 INFO  
(zkCallback-159-thread-1-processing-n:127.0.0.1:41597__q%2Fl) 
[n:127.0.0.1:41597__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 177911 INFO  
(zkCallback-172-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 177911 INFO  
(zkCallback-153-thread-3-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 177911 INFO  
(zkCallback-165-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 177949 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:32820__q%2Fl    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-004/cores
   [junit4]   2> 177949 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) 
[n:127.0.0.1:32820__q%2Fl    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 177951 INFO  
(OverseerStateUpdate-96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 178543 INFO  (qtp8155725-1126) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): 
collection=tmp_collection, shard=shard2, thisCore=collection1, 
leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, 
currentState=recovering, localState=active, nodeName=127.0.0.1:34096__q%2Fl, 
coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: 
core_node3:{"core":"collection1","base_url":"https://127.0.0.1:34096/_q/l","node_name":"127.0.0.1:34096__q%2Fl","state":"recovering"}
   [junit4]   2> 178564 INFO  (qtp8155725-1126) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, 
checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 178564 INFO  (qtp8155725-1126) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={nodeName=127.0.0.1:34096__q%252Fl&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
 status=0 QTime=1021
   [junit4]   2> 178961 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 178972 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 179055 WARN  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 179057 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 179072 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection tmp_collection
   [junit4]   2> 179073 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-004/cores/collection1],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrVersionReplicationTest_50CCC30F4CBE9AA2-001/jetty-004/cores/collection1/data/]
   [junit4]   2> 179073 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@fee053
   [junit4]   2> 179074 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@1b74147
   [junit4]   2> 179097 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 179097 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 179098 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 179098 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 179099 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=20, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=10.2685546875, 
floorSegmentMB=1.20703125, forceMergeDeletesPctAllowed=17.322828593338066, 
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 179099 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@8921d5[collection1] main]
   [junit4]   2> 179100 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 179100 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 179101 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 179103 INFO  
(searcherExecutor-562-thread-1-processing-n:127.0.0.1:32820__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@8921d5[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 179103 INFO  
(coreLoadExecutor-561-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1547246639337439232
   [junit4]   2> 179107 INFO  
(coreZkRegister-556-thread-1-processing-n:127.0.0.1:32820__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.ZkController Core 
needs to recover:collection1
   [junit4]   2> 179107 INFO  
(updateExecutor-169-thread-1-processing-n:127.0.0.1:32820__q%2Fl x:collection1 
s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 179107 INFO  
(recoveryExecutor-170-thread-1-processing-n:127.0.0.1:32820__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy 
Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 179108 INFO  
(recoveryExecutor-170-thread-1-processing-n:127.0.0.1:32820__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy 
###### startupVersions=[[]]
   [junit4]   2> 179108 INFO  
(recoveryExecutor-170-thread-1-processing-n:127.0.0.1:32820__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy 
Begin buffering updates. core=[collection1]
   [junit4]   2> 179108 INFO  
(recoveryExecutor-170-thread-1-processing-n:127.0.0.1:32820__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateLog 
Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 179108 INFO  
(recoveryExecutor-170-thread-1-processing-n:127.0.0.1:32820__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy 
Publishing state of core [collection1] as recovering, leader is 
[https://127.0.0.1:41597/_q/l/collection1/] and I am 
[https://127.0.0.1:32820/_q/l/collection1/]
   [junit4]   2> 179110 INFO  
(recoveryExecutor-170-thread-1-processing-n:127.0.0.1:32820__q%2Fl 
x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:32820__q%2Fl 
c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy 
Sending prep recovery command to [https://127.0.0.1:41597/_q/l]; [WaitForState: 
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:32820__q%252Fl&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 179213 INFO  (qtp8914697-1159) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: 
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 179213 INFO  (qtp8914697-1159) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 
(shard1 of tmp_collection) have state: recovering
   [junit4]   2> 179213 INFO  (qtp8914697-1159) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): 
collection=tmp_collection, shard=shard1, thisCore=collection1, 
leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, 
currentState=recovering, localState=active, nodeName=127.0.0.1:32820__q%2Fl, 
coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: 
core_node4:{"core":"collection1","base_url":"https://127.0.0.1:32820/_q/l","node_name":"127.0.0.1:32820__q%2Fl","state":"recovering"}
   [junit4]   2> 179213 INFO  (qtp8914697-1159) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node4, state: recovering, 
checkLive: true, onlyIfLeader: true for: 0 seconds.
   [junit4]   2> 179214 INFO  (qtp8914697-1159) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={nodeName=127.0.0.1:32820__q%252Fl&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   1> / (2)
   [junit4]   1> DATA:
   [junit4]   1>     
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1>  /solr (9)
   [junit4]   1>   /solr/configs (1)
   [junit4]   1>    /solr/configs/conf1 (11)
   [junit4]   1>     /solr/configs/conf1/currency.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/enumsConfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/protwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/old_synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/solrconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/stopwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/schema.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     
/solr/configs/conf1/solrconfig.snippet.randomindexconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/open-exchange-rates.json (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {
   [junit4]   1>           "disclaimer": "This data is not real, it was 
synthetically created to match currency.xml.  It is modeled after the data 
format available from openexchangerates.org.  See 
https://openexchangerates.org/documentation for details
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           IMPORTANT NOTE: In order for tests to work, this 
data must be kept in sync with ./currency.xml",
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           "license": 
"http://www.apache.org/licenses/LICENSE-2.0";,
   [junit4]   1>           "timestamp": 1332070464,
   [junit4]   1>           "base": "USD",
   [junit4]   1>           "rates": {
   [junit4]   1>             "USD": 1,
   [junit4]   1>             "JPY": 81.29,
   [junit4]   1>             "EUR": 2.5,
   [junit4]   1>             "GBP": 0.5,
   [junit4]   1>             "MXN": 2.0
   [junit4]   1>           }
   [junit4]   1>         }
   [junit4]   1>         
   [junit4]   1>   /solr/overseer (6)
   [junit4]   1>   DATA:
   [junit4]   1>       
   [junit4]   1>    /solr/overseer/collection-queue-work (0)
   [junit4]   1>    /solr/overseer/queue-work (0)
   [junit4]   1>    /solr/overseer/collection-map-failure (0)
   [junit4]   1>    /solr/overseer/collection-map-completed (0)
   [junit4]   1>    /solr/overseer/queue (0)
   [junit4]   1>    /solr/overseer/collection-map-running (0)
   [junit4]   1>   /solr/aliases.json (0)
   [junit4]   1>   /solr/collections (3)
   [junit4]   1>    /solr/collections/collection1 (1)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/collection1/shards (0)
   [junit4]   1>    /solr/collections/tmp_collection (3)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/tmp_collection/leaders (2)
   [junit4]   1>      /solr/collections/tmp_collection/leaders/shard2 (1)
   [junit4]   1>       /solr/collections/tmp_collection/leaders/shard2/leader 
(0)
   [junit4]   1>       DATA:
   [junit4]   1>           {
   [junit4]   1>             "core":"collection1",
   [junit4]   1>             "core_node_name":"core_node1",
   [junit4]   1>             "base_url":"https://127.0.0.1:43252/_q/l";,
   [junit4]   1>             "node_name":"127.0.0.1:43252__q%2Fl"}
   [junit4]   1>      /solr/collections/tmp_collection/leaders/shard1 (1)
   [junit4]   1>       /solr/collections/tmp_collection/leaders/shard1/leader 
(0)
   [junit4]   1>       DATA:
   [junit4]   1>           {
   [junit4]   1>             "core":"collection1",
   [junit4]   1>             "core_node_name":"core_node2",
   [junit4]   1>             "base_url":"https://127.0.0.1:41597/_q/l";,
   [junit4]   1>             "node_name":"127.0.0.1:41597__q%2Fl"}
   [junit4]   1>     /solr/collections/tmp_collection/cdcr (1)
   [junit4]   1>      /solr/collections/tmp_collection/cdcr/state (2)
   [junit4]   1>       /solr/collections/tmp_collection/cdcr/state/process (0)
   [junit4]   1>       DATA:
   [junit4]   1>           stopped
   [junit4]   1>       /solr/collections/tmp_collection/cdcr/state/buffer (0)
   [junit4]   1>       DATA:
   [junit4]   1>           enabled
   [junit4]   1>     /solr/collections/tmp_collection/leader_elect (2)
   [junit4]   1>      /solr/collections/tmp_collection/leader_elect/shard2 (1)
   [junit4]   1>       
/solr/collections/tmp_collection/leader_elect/shard2/election (2)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard2/election/96702914538962955-core_node3-n_0000000001
 (0)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard2/election/96702914538962949-core_node1-n_0000000000
 (0)
   [junit4]   1>      /solr/collections/tmp_collection/leader_elect/shard1 (1)
   [junit4]   1>       
/solr/collections/tmp_collection/leader_elect/shard1/election (2)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard1/election/96702914538962952-core_node2-n_0000000000
 (0)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard1/election/96702914538962958-core_node4-n_0000000001
 (0)
   [junit4]   1>    /solr/collections/control_collection (1)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/control_collection/shards (0)
   [junit4]   1>   /solr/live_nodes (4)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:34096__q%2Fl (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:41597__q%2Fl (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:32820__q%2Fl (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:43252__q%2Fl (0)
   [junit4]   1>   /solr/overseer_elect (2)
   [junit4]   1>    /solr/overseer_elect/leader (0)
   [junit4]   1>    DATA:
   [junit4]   1>        
{"id":"96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000"}
   [junit4]   1>    /solr/overseer_elect/election (4)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962958-127.0.0.1:32820__q%2Fl-n_0000000003
 (0)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962955-127.0.0.1:34096__q%2Fl-n_0000000002
 (0)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000
 (0)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962952-127.0.0.1:41597__q%2Fl-n_0000000001
 (0)
   [junit4]   1>   /solr/security.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {}
   [junit4]   1>   /solr/clusterstate.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {"tmp_collection":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "autoAddReplicas":"false",
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "shards":{
   [junit4]   1>             "shard1":{
   [junit4]   1>               "range":"80000000-ffffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:41597/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:41597__q%2Fl",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node4":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:32820/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:32820__q%2Fl",
   [junit4]   1>                   "state":"recovering"}}},
   [junit4]   1>             "shard2":{
   [junit4]   1>               "range":"0-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:43252/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:43252__q%2Fl",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node3":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:34096/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:34096__q%2Fl",
   [junit4]   1>                   "state":"recovering"}}}}}}
   [junit4]   1>   /solr/clusterprops.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {"urlScheme":"https"}
   [junit4]   1> 
   [junit4]   1> / (2)
   [junit4]   1> DATA:
   [junit4]   1>     
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1>  /solr (9)
   [junit4]   1>   /solr/configs (1)
   [junit4]   1>    /solr/configs/conf1 (11)
   [junit4]   1>     /solr/configs/conf1/currency.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/enumsConfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/protwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/old_synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/solrconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/stopwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/schema.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     
/solr/configs/conf1/solrconfig.snippet.randomindexconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/open-exchange-rates.json (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {
   [junit4]   1>           "disclaimer": "This data is not real, it was 
synthetically created to match currency.xml.  It is modeled after the data 
format available from openexchangerates.org.  See 
https://openexchangerates.org/documentation for details
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           IMPORTANT NOTE: In order for tests to work, this 
data must be kept in sync with ./currency.xml",
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           "license": 
"http://www.apache.org/licenses/LICENSE-2.0";,
   [junit4]   1>           "timestamp": 1332070464,
   [junit4]   1>           "base": "USD",
   [junit4]   1>           "rates": {
   [junit4]   1>             "USD": 1,
   [junit4]   1>             "JPY": 81.29,
   [junit4]   1>             "EUR": 2.5,
   [junit4]   1>             "GBP": 0.5,
   [junit4]   1>             "MXN": 2.0
   [junit4]   1>           }
   [junit4]   1>         }
   [junit4]   1>         
   [junit4]   1>   /solr/overseer (6)
   [junit4]   1>   DATA:
   [junit4]   1>       
   [junit4]   1>    /solr/overseer/collection-queue-work (0)
   [junit4]   1>    /solr/overseer/queue-work (0)
   [junit4]   1>    /solr/overseer/collection-map-failure (0)
   [junit4]   1>    /solr/overseer/collection-map-completed (0)
   [junit4]   1>    /solr/overseer/queue (0)
   [junit4]   1>    /solr/overseer/collection-map-running (0)
   [junit4]   1>   /solr/aliases.json (0)
   [junit4]   1>   /solr/collections (3)
   [junit4]   1>    /solr/collections/collection1 (1)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/collection1/shards (0)
   [junit4]   1>    /solr/collections/tmp_collection (3)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/tmp_collection/leaders (2)
   [junit4]   1>      /solr/collections/tmp_collection/leaders/shard2 (1)
   [junit4]   1>       /solr/collections/tmp_collection/leaders/shard2/leader 
(0)
   [junit4]   1>       DATA:
   [junit4]   1>           {
   [junit4]   1>             "core":"collection1",
   [junit4]   1>             "core_node_name":"core_node1",
   [junit4]   1>             "base_url":"https://127.0.0.1:43252/_q/l";,
   [junit4]   1>             "node_name":"127.0.0.1:43252__q%2Fl"}
   [junit4]   1>      /solr/collections/tmp_collection/leaders/shard1 (1)
   [junit4]   1>       /solr/collections/tmp_collection/leaders/shard1/leader 
(0)
   [junit4]   1>       DATA:
   [junit4]   1>           {
   [junit4]   1>             "core":"collection1",
   [junit4]   1>             "core_node_name":"core_node2",
   [junit4]   1>             "base_url":"https://127.0.0.1:41597/_q/l";,
   [junit4]   1>             "node_name":"127.0.0.1:41597__q%2Fl"}
   [junit4]   1>     /solr/collections/tmp_collection/cdcr (1)
   [junit4]   1>      /solr/collections/tmp_collection/cdcr/state (2)
   [junit4]   1>       /solr/collections/tmp_collection/cdcr/state/process (0)
   [junit4]   1>       DATA:
   [junit4]   1>           stopped
   [junit4]   1>       /solr/collections/tmp_collection/cdcr/state/buffer (0)
   [junit4]   1>       DATA:
   [junit4]   1>           enabled
   [junit4]   1>     /solr/collections/tmp_collection/leader_elect (2)
   [junit4]   1>      /solr/collections/tmp_collection/leader_elect/shard2 (1)
   [junit4]   1>       
/solr/collections/tmp_collection/leader_elect/shard2/election (2)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard2/election/96702914538962955-core_node3-n_0000000001
 (0)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard2/election/96702914538962949-core_node1-n_0000000000
 (0)
   [junit4]   1>      /solr/collections/tmp_collection/leader_elect/shard1 (1)
   [junit4]   1>       
/solr/collections/tmp_collection/leader_elect/shard1/election (2)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard1/election/96702914538962952-core_node2-n_0000000000
 (0)
   [junit4]   1>        
/solr/collections/tmp_collection/leader_elect/shard1/election/96702914538962958-core_node4-n_0000000001
 (0)
   [junit4]   1>    /solr/collections/control_collection (1)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/control_collection/shards (0)
   [junit4]   1>   /solr/live_nodes (4)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:34096__q%2Fl (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:41597__q%2Fl (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:32820__q%2Fl (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:43252__q%2Fl (0)
   [junit4]   1>   /solr/overseer_elect (2)
   [junit4]   1>    /solr/overseer_elect/leader (0)
   [junit4]   1>    DATA:
   [junit4]   1>        
{"id":"96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000"}
   [junit4]   1>    /solr/overseer_elect/election (4)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962958-127.0.0.1:32820__q%2Fl-n_0000000003
 (0)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962955-127.0.0.1:34096__q%2Fl-n_0000000002
 (0)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962949-127.0.0.1:43252__q%2Fl-n_0000000000
 (0)
   [junit4]   1>     
/solr/overseer_elect/election/96702914538962952-127.0.0.1:41597__q%2Fl-n_0000000001
 (0)
   [junit4]   1>   /solr/security.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {}
   [junit4]   1>   /solr/clusterstate.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {"tmp_collection":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "autoAddReplicas":"false",
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "shards":{
   [junit4]   1>             "shard1":{
   [junit4]   1>               "range":"80000000-ffffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:41597/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:41597__q%2Fl",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node4":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:32820/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:32820__q%2Fl",
   [junit4]   1>                   "state":"recovering"}}},
   [junit4]   1>             "shard2":{
   [junit4]   1>               "range":"0-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:43252/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:43252__q%2Fl",
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node3":{
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "base_url":"https://127.0.0.1:34096/_q/l";,
   [junit4]   1>                   "node_name":"127.0.0.1:34096__q%2Fl",
   [junit4]   1>                   "state":"recovering"}}}}}}
   [junit4]   1>   /solr/clusterprops.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {"urlScheme":"https"}
   [junit4]   1> 
   [junit4]   2> 179486 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 179588 INFO  (qtp24435380-1189) [n:127.0.0.1:34096__q%2Fl    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params 
name=tmp_collection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 179590 INFO  
(OverseerThreadFactory-524-thread-1-processing-n:127.0.0.1:43252__q%2Fl) 
[n:127.0.0.1:43252__q%2Fl    ] o.a.s.c.OverseerCollectionMessageHandler 
Executing Collection Cmd : 
action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 179604 WARN  (qtp24435380-1190) [n:127.0.0.1:34096__q%2Fl    ] 
o.a.s.c.RecoveryStrategy Stopping recovery for core=[collection1] 
coreNodeName=[core_node3]
   [junit4]   2> 179604 INFO  (qtp8155725-1132) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.c.SolrCore [collection1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@145613e
   [junit4]   2> 179604 INFO  (qtp24435380-1190) [n:127.0.0.1:34096__q%2Fl    ] 
o.a.s.c.SolrCore Core collection1 is not yet closed, waiting 100 ms before 
checking again.
   [junit4]   2> 179604 INFO  (qtp8914697-1160) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.c.SolrCore [collection1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@65c9d7
   [junit4]   2> 179604 INFO  (qtp8914697-1160) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR 
handler @ tmp_collection:shard1
   [junit4]   2> 179604 INFO  (qtp8155725-1132) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR 
handler @ tmp_collection:shard2
   [junit4]   2> 179633 INFO  
(zkCallback-172-thread-2-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl    ] o.a.s.h.CdcrLeaderStateManager Received new 
leader state @ tmp_collection:shard1
   [junit4]   2> 179634 WARN  
(zkCallback-165-thread-1-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.c.LeaderElector 
   [junit4]   2> org.apache.solr.common.SolrException: SolrCore not 
found:collection1 in []
   [junit4]   2>        at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:296)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:348)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:268)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 179634 INFO  
(zkCallback-165-thread-2-processing-n:127.0.0.1:34096__q%2Fl) 
[n:127.0.0.1:34096__q%2Fl    ] o.a.s.h.CdcrLeaderStateManager Received new 
leader state @ tmp_collection:shard2
   [junit4]   2> 179634 INFO  
(zkCallback-172-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.c.ShardLeaderElectionContext There may be a better leader candidate than 
us - going back into recovery
   [junit4]   2> 179634 INFO  (qtp8155725-1132) [n:127.0.0.1:43252__q%2Fl    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={deleteInstanceDir=true&core=collection1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2}
 status=0 QTime=31
   [junit4]   2> 179634 INFO  (qtp8914697-1160) [n:127.0.0.1:41597__q%2Fl    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={deleteInstanceDir=true&core=collection1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2}
 status=0 QTime=32
   [junit4]   2> 179635 INFO  
(zkCallback-172-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader 
parent node, won't remove previous leader registration.
   [junit4]   2> 179635 INFO  
(zkCallback-172-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.c.ActionThrottle The last leader attempt started 2ms ago.
   [junit4]   2> 179635 INFO  
(zkCallback-172-thread-1-processing-n:127.0.0.1:32820__q%2Fl) 
[n:127.0.0.1:32820__q%2Fl c:tmp_collection s:shard1 r:core_node4 x:collection1] 
o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 4997ms
   [junit4]   2> 179704 INFO  (qtp24435380-1190) [n:127.0.0.1:34096__q%2Fl    ] 
o.a.s.c.SolrCore Core collection1 is not yet closed, waiting 100 ms be

[...truncated too long message...]

rsion=2} hits=829 status=0 QTime=2
   [junit4]   2> 1444649 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 829 docs
   [junit4]   2> 1444652 INFO  (qtp5796238-87755) [    x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication 
params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} 
status=0 QTime=1
   [junit4]   2> 1444655 INFO  (qtp5796238-87748) [    x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication 
params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} 
status=0 QTime=0
   [junit4]   2> 1444658 INFO  (qtp13509271-87732) [    x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication 
params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} 
status=0 QTime=0
   [junit4]   2> 1444660 INFO  (qtp5796238-87753) [    x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication 
params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion}
 status=0 QTime=0
   [junit4]   2> 1444662 INFO  (qtp13509271-87736) [    x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication 
params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion}
 status=0 QTime=0
   [junit4]   1> [snapshot_metadata, index]
   [junit4]   1> [snapshot_metadata, index.20161004014406147, 
replication.properties, index.properties]
   [junit4]   2> 1444666 INFO  (qtp13509271-87735) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[841]} 0 1
   [junit4]   2> 1444669 INFO  (qtp13509271-87733) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[842]} 0 0
   [junit4]   2> 1444671 INFO  (qtp13509271-87732) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={wt=javabin&version=2}{add=[843]} 0 0
   [junit4]   2> 1444672 INFO  (qtp13509271-87736) [    x:collection1] 
o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1444673 INFO  (qtp13509271-87736) [    x:collection1] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@1557113
   [junit4]   2> 1444793 INFO  (qtp13509271-87736) [    x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@1fe92e2[collection1] main]
   [junit4]   2> 1444794 INFO  (qtp13509271-87736) [    x:collection1] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1444794 INFO  
(searcherExecutor-8612-thread-1-processing-x:collection1) [    x:collection1] 
o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@1fe92e2[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.3.0):c49)
 Uninverting(_1(6.3.0):c19) Uninverting(_2(6.3.0):c142) 
Uninverting(_3(6.3.0):c145) Uninverting(_4(6.3.0):c103) 
Uninverting(_5(6.3.0):c96) Uninverting(_6(6.3.0):c101) 
Uninverting(_7(6.3.0):c174) Uninverting(_8(6.3.0):c3)))}
   [junit4]   2> 1444794 INFO  (qtp13509271-87736) [    x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update 
params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
 0 121
   [junit4]   2> 1444795 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.SolrTestCaseJ4 ###Ending doTestStressReplication
   [junit4]   2> 1444795 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.ServerConnector Stopped 
ServerConnector@133f872{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1444796 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=21555209
   [junit4]   2> 1444796 INFO  (coreCloseExecutor-8616-thread-1) [    
x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@1ad8284
   [junit4]   2> 1444797 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@a86b9{/solr,null,UNAVAILABLE}
   [junit4]   2> 1444798 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.ServerConnector Stopped 
ServerConnector@1bdc3cf{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1444799 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=26182783
   [junit4]   2> 1444799 INFO  (coreCloseExecutor-8618-thread-1) [    
x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@8e0be1
   [junit4]   2> 1444800 INFO  
(TEST-TestReplicationHandler.doTestStressReplication-seed#[50CCC30F4CBE9AA2]) [ 
   ] o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@b9f810{/solr,null,UNAVAILABLE}
   [junit4]   2> 1444804 INFO  
(SUITE-TestReplicationHandler-seed#[50CCC30F4CBE9AA2]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_50CCC30F4CBE9AA2-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{name=PostingsFormat(name=LuceneVarGapDocFreqInterval), 
id=PostingsFormat(name=LuceneVarGapDocFreqInterval), 
newname=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{}, 
maxPointsInLeafNode=673, maxMBSortInHeap=5.448850345890045, 
sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=ms-MY, 
timezone=America/Creston
   [junit4]   2> NOTE: Linux 4.4.0-36-generic i386/Oracle Corporation 1.8.0_102 
(32-bit)/cpus=12,threads=1,free=200069368,total=388485120
   [junit4]   2> NOTE: All tests run in this JVM: 
[TestPHPSerializedResponseWriter, TestQuerySenderNoQuery, 
TestPerFieldSimilarityWithDefaultOverride, TestStressRecovery, 
TestDFISimilarityFactory, BitVectorTest, MultiThreadedOCPTest, 
SolrCLIZkUtilsTest, TestBadConfig, BlobRepositoryCloudTest, TestFunctionQuery, 
TestTestInjection, SharedFSAutoReplicaFailoverTest, ResourceLoaderTest, 
QueryElevationComponentTest, TestRealTimeGet, UUIDFieldTest, 
TestComponentsName, DeleteInactiveReplicaTest, 
TestSha256AuthenticationProvider, TestFilteredDocIdSet, 
DistributedFacetPivotLongTailTest, DistributedExpandComponentTest, 
TestRandomFaceting, TestHighlightDedupGrouping, CloudMLTQParserTest, 
MBeansHandlerTest, AnalyticsMergeStrategyTest, MergeStrategyTest, 
TestSolrDynamicMBean, JavabinLoaderTest, FastVectorHighlighterTest, 
HdfsRecoverLeaseTest, HdfsDirectoryFactoryTest, TestScoreJoinQPNoScore, 
RequestLoggingTest, TestCodecSupport, TestExceedMaxTermLength, 
TestMacroExpander, TestConfigSetsAPI, DefaultValueUpdateProcessorTest, 
ReplicationFactorTest, TestSerializedLuceneMatchVersion, 
TestSolrFieldCacheMBean, BadCopyFieldTest, TestJmxIntegration, TestDocSet, 
CleanupOldIndexTest, TestFastWriter, DocumentAnalysisRequestHandlerTest, 
NumericFieldsTest, AlternateDirectoryTest, 
DistributedQueryElevationComponentTest, TestValueSourceCache, 
ChaosMonkeyNothingIsSafeTest, TestAuthenticationFramework, TestConfigSets, 
CdcrBootstrapTest, CoreMergeIndexesAdminHandlerTest, TestStressLiveNodes, 
TestFoldingMultitermQuery, CollectionsAPISolrJTest, TestOmitPositions, 
NotRequiredUniqueKeyTest, CollectionStateFormat2Test, 
TestAuthorizationFramework, TestFieldCacheSortRandom, 
TestFieldTypeCollectionResource, TestSchemaManager, 
RecoveryAfterSoftCommitTest, ZkNodePropsTest, TestFastLRUCache, 
ExternalFileFieldSortTest, TestCopyFieldCollectionResource, 
LeaderElectionIntegrationTest, SegmentsInfoRequestHandlerTest, TestRestoreCore, 
TestRangeQuery, TestSolrConfigHandlerConcurrent, TestFaceting, SyncSliceTest, 
CdcrUpdateLogTest, TestReloadAndDeleteDocs, 
UniqFieldsUpdateProcessorFactoryTest, TestClusterStateMutator, TestCSVLoader, 
WrapperMergePolicyFactoryTest, SmileWriterTest, TestLazyCores, 
TestAtomicUpdateErrorCases, TestDynamicFieldResource, 
CurrencyFieldOpenExchangeTest, BasicDistributedZkTest, SmileWriterTest, 
HdfsUnloadDistributedZkTest, SolrCoreTest, DateFieldTest, 
TestReplicaProperties, TestTrieFacet, CoreSorterTest, 
PathHierarchyTokenizerFactoryTest, ParsingFieldUpdateProcessorsTest, 
ConnectionReuseTest, TestAddFieldRealTimeGet, BasicDistributedZk2Test, 
UnloadDistributedZkTest, OpenCloseCoreStressTest, BasicZkTest, RecoveryZkTest, 
TestReplicationHandler]
   [junit4] Completed [326/638 (2!)] on J0 in 111.46s, 15 tests, 1 failure, 1 
skipped <<< FAILURES!

[...truncated 988 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/temp/junit4-J1-20161004_082000_546.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/heapdumps/java_pid10981.hprof ...
   [junit4] Heap dump file created [558313237 bytes in 3.037 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 11051 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/build.xml:763: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/build.xml:715: Some of the tests 
produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? 
Dumps created:
* java_pid10981.hprof

Total time: 80 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to