Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1090/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.PeerSyncReplicationTest.test

Error Message:
timeout waiting to see all nodes active

Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
        at 
__randomizedtesting.SeedInfo.seed([9070C66A508CF174:1824F9B0FE709C8C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
        at 
org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
        at 
org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
        at 
org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12284 lines...]
   [junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
   [junit4]   2> Creating dataDir: 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/init-core-data-001
   [junit4]   2> 2620928 INFO  
(SUITE-PeerSyncReplicationTest-seed#[9070C66A508CF174]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 2620930 INFO  
(SUITE-PeerSyncReplicationTest-seed#[9070C66A508CF174]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2620930 INFO  
(SUITE-PeerSyncReplicationTest-seed#[9070C66A508CF174]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /rcpl/k
   [junit4]   2> 2620934 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2620934 INFO  (Thread-6141) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2620934 INFO  (Thread-6141) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 2621034 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:48387
   [junit4]   2> 2621052 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 2621056 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 2621059 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2621061 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 2621063 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 2621066 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 2621069 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2621071 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2621074 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2621077 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2621080 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 2621230 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/control-001/cores/collection1
   [junit4]   2> 2621231 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2621233 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@339ca386{/rcpl/k,null,AVAILABLE}
   [junit4]   2> 2621234 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@330bd700{HTTP/1.1,[http/1.1]}{127.0.0.1:38838}
   [junit4]   2> 2621234 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server Started @2626041ms
   [junit4]   2> 2621234 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/tempDir-001/control/data,
 hostContext=/rcpl/k, hostPort=38838, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/control-001/cores}
   [junit4]   2> 2621234 ERROR 
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2621234 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.0.0
   [junit4]   2> 2621234 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2621234 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2621235 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-01-22T12:28:28.667Z
   [junit4]   2> 2621240 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 2621240 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/control-001/solr.xml
   [junit4]   2> 2621248 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48387/solr
   [junit4]   2> 2621287 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.OverseerElectionContext I am going to 
be the leader 127.0.0.1:38838_rcpl%2Fk
   [junit4]   2> 2621289 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.Overseer Overseer 
(id=97326734270791684-127.0.0.1:38838_rcpl%2Fk-n_0000000000) starting
   [junit4]   2> 2621301 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:38838_rcpl%2Fk
   [junit4]   2> 2621304 INFO  
(OverseerStateUpdate-97326734270791684-127.0.0.1:38838_rcpl%2Fk-n_0000000000) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 2621446 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/control-001/cores
   [junit4]   2> 2621446 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 2621452 INFO  
(OverseerStateUpdate-97326734270791684-127.0.0.1:38838_rcpl%2Fk-n_0000000000) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 2622468 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2622484 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2622595 WARN  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] default search field in schema is text. 
WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2622599 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2622635 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 2622635 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/control-001/cores/collection1],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/control-001/cores/collection1/data/]
   [junit4]   2> 2622635 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@556c5fad
   [junit4]   2> 2622638 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=25, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=66.572265625, 
floorSegmentMB=1.0556640625, forceMergeDeletesPctAllowed=20.05078628936581, 
segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.3936373592860306
   [junit4]   2> 2622648 WARN  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2622660 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2622660 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2622663 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2622663 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2622663 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@2f68a1cf
   [junit4]   2> 2622664 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@2864296c[collection1] main]
   [junit4]   2> 2622666 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 2622667 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2622667 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2622668 INFO  
(searcherExecutor-8594-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection   x:collection1] o.a.s.c.SolrCore [collection1] Registered 
new searcher Searcher@2864296c[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2622668 INFO  
(coreLoadExecutor-8593-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection   x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1557227750136217600
   [junit4]   2> 2622679 INFO  
(coreZkRegister-8586-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2622680 INFO  
(coreZkRegister-8586-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2622680 INFO  
(coreZkRegister-8586-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:38838/rcpl/k/collection1/
   [junit4]   2> 2622680 INFO  
(coreZkRegister-8586-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 2622680 INFO  
(coreZkRegister-8586-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:38838/rcpl/k/collection1/ has no replicas
   [junit4]   2> 2622687 INFO  
(coreZkRegister-8586-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:38838/rcpl/k/collection1/ shard1
   [junit4]   2> 2622842 INFO  
(coreZkRegister-8586-thread-1-processing-n:127.0.0.1:38838_rcpl%2Fk 
x:collection1 c:control_collection) [n:127.0.0.1:38838_rcpl%2Fk 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 2623077 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2623079 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48387/solr ready
   [junit4]   2> 2623079 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 2623227 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-1-001/cores/collection1
   [junit4]   2> 2623227 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-1-001
   [junit4]   2> 2623228 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2623231 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@4f9f2c7b{/rcpl/k,null,AVAILABLE}
   [junit4]   2> 2623231 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@3afb700e{HTTP/1.1,[http/1.1]}{127.0.0.1:44084}
   [junit4]   2> 2623231 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server Started @2628039ms
   [junit4]   2> 2623232 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/tempDir-001/jetty1,
 solrconfig=solrconfig.xml, hostContext=/rcpl/k, hostPort=44084, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-1-001/cores}
   [junit4]   2> 2623232 ERROR 
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2623232 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.0.0
   [junit4]   2> 2623232 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2623232 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2623232 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-01-22T12:28:30.664Z
   [junit4]   2> 2623236 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 2623236 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-1-001/solr.xml
   [junit4]   2> 2623263 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48387/solr
   [junit4]   2> 2623279 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:44084_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 2623287 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:44084_rcpl%2Fk    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:44084_rcpl%2Fk
   [junit4]   2> 2623290 INFO  
(zkCallback-3561-thread-3-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (1) -> (2)
   [junit4]   2> 2623290 INFO  (zkCallback-3565-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2623292 INFO  
(zkCallback-3570-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (1) -> (2)
   [junit4]   2> 2623402 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:44084_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-1-001/cores
   [junit4]   2> 2623402 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:44084_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 2623406 INFO  
(OverseerStateUpdate-97326734270791684-127.0.0.1:38838_rcpl%2Fk-n_0000000000) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 2624435 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2624449 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 2624580 WARN  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 2624583 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2624603 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection collection1
   [junit4]   2> 2624604 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] o.a.s.c.SolrCore 
[[collection1] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-1-001/cores/collection1],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 2624604 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@556c5fad
   [junit4]   2> 2624607 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=25, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=66.572265625, 
floorSegmentMB=1.0556640625, forceMergeDeletesPctAllowed=20.05078628936581, 
segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.3936373592860306
   [junit4]   2> 2624620 WARN  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2624640 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2624640 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2624642 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2624642 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2624643 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@abd9b4c
   [junit4]   2> 2624644 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@14ff9d3c[collection1] main]
   [junit4]   2> 2624646 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 2624647 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2624647 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2624648 INFO  
(searcherExecutor-8605-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1   
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@14ff9d3c[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2624648 INFO  
(coreLoadExecutor-8604-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk c:collection1   x:collection1] o.a.s.u.UpdateLog 
Could not find max version in index or recent updates, using new clock 
1557227752212398080
   [junit4]   2> 2624660 INFO  
(coreZkRegister-8599-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1 s:shard1 
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas 
found to continue.
   [junit4]   2> 2624661 INFO  
(coreZkRegister-8599-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1 s:shard1 
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new 
leader - try and sync
   [junit4]   2> 2624661 INFO  
(coreZkRegister-8599-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1 s:shard1 
r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:44084/rcpl/k/collection1/
   [junit4]   2> 2624661 INFO  
(coreZkRegister-8599-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1 s:shard1 
r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync 
replicas to me
   [junit4]   2> 2624662 INFO  
(coreZkRegister-8599-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1 s:shard1 
r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:44084/rcpl/k/collection1/ has no replicas
   [junit4]   2> 2624668 INFO  
(coreZkRegister-8599-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1 s:shard1 
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new 
leader: http://127.0.0.1:44084/rcpl/k/collection1/ shard1
   [junit4]   2> 2624821 INFO  
(coreZkRegister-8599-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:44084_rcpl%2Fk c:collection1 s:shard1 
r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery 
necessary
   [junit4]   2> 2625102 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-2-001/cores/collection1
   [junit4]   2> 2625102 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-2-001
   [junit4]   2> 2625103 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2625106 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1b126b86{/rcpl/k,null,AVAILABLE}
   [junit4]   2> 2625106 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@2522ff4c{HTTP/1.1,[http/1.1]}{127.0.0.1:33030}
   [junit4]   2> 2625106 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server Started @2629914ms
   [junit4]   2> 2625106 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/tempDir-001/jetty2,
 solrconfig=solrconfig.xml, hostContext=/rcpl/k, hostPort=33030, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-2-001/cores}
   [junit4]   2> 2625106 ERROR 
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2625107 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.0.0
   [junit4]   2> 2625107 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2625107 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2625107 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-01-22T12:28:32.539Z
   [junit4]   2> 2625112 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 2625112 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-2-001/solr.xml
   [junit4]   2> 2625117 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48387/solr
   [junit4]   2> 2625133 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:33030_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (2)
   [junit4]   2> 2625141 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:33030_rcpl%2Fk    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:33030_rcpl%2Fk
   [junit4]   2> 2625143 INFO  (zkCallback-3565-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2625143 INFO  
(zkCallback-3561-thread-3-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (2) -> (3)
   [junit4]   2> 2625143 INFO  
(zkCallback-3570-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (2) -> (3)
   [junit4]   2> 2625143 INFO  
(zkCallback-3576-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (2) -> (3)
   [junit4]   2> 2625258 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:33030_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-2-001/cores
   [junit4]   2> 2625258 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:33030_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 2625261 INFO  
(OverseerStateUpdate-97326734270791684-127.0.0.1:38838_rcpl%2Fk-n_0000000000) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 2626277 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2626290 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 2626398 WARN  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 2626400 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2626419 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection collection1
   [junit4]   2> 2626420 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] o.a.s.c.SolrCore 
[[collection1] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-2-001/cores/collection1],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-2-001/cores/collection1/data/]
   [junit4]   2> 2626420 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@556c5fad
   [junit4]   2> 2626422 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=25, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=66.572265625, 
floorSegmentMB=1.0556640625, forceMergeDeletesPctAllowed=20.05078628936581, 
segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.3936373592860306
   [junit4]   2> 2626432 WARN  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2626445 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2626445 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2626447 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2626447 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2626447 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@16c68c3b
   [junit4]   2> 2626448 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@62175f53[collection1] main]
   [junit4]   2> 2626449 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 2626450 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2626450 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2626451 INFO  
(searcherExecutor-8616-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:33030_rcpl%2Fk c:collection1   
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@62175f53[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2626452 INFO  
(coreLoadExecutor-8615-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1   x:collection1] o.a.s.u.UpdateLog 
Could not find max version in index or recent updates, using new clock 
1557227754104029184
   [junit4]   2> 2626457 INFO  
(coreZkRegister-8610-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.ZkController Core needs to 
recover:collection1
   [junit4]   2> 2626458 INFO  
(updateExecutor-3573-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState 
Running recovery
   [junit4]   2> 2626459 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 2626459 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
###### startupVersions=[[]]
   [junit4]   2> 2626459 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
Begin buffering updates. core=[collection1]
   [junit4]   2> 2626459 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting 
to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 2626459 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
Publishing state of core [collection1] as recovering, leader is 
[http://127.0.0.1:44084/rcpl/k/collection1/] and I am 
[http://127.0.0.1:33030/rcpl/k/collection1/]
   [junit4]   2> 2626462 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
Sending prep recovery command to [http://127.0.0.1:44084/rcpl/k]; 
[WaitForState: 
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:33030_rcpl%252Fk&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 2626464 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, 
state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 2626465 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 
(shard1 of collection1) have state: recovering
   [junit4]   2> 2626465 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): 
collection=collection1, shard=shard1, thisCore=collection1, 
leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, 
currentState=down, localState=active, nodeName=127.0.0.1:33030_rcpl%2Fk, 
coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: 
core_node2:{"core":"collection1","base_url":"http://127.0.0.1:33030/rcpl/k","node_name":"127.0.0.1:33030_rcpl%2Fk","state":"down"}
   [junit4]   2> 2627027 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-3-001/cores/collection1
   [junit4]   2> 2627028 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-3-001
   [junit4]   2> 2627029 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2627030 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@5e798942{/rcpl/k,null,AVAILABLE}
   [junit4]   2> 2627031 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@68aaae26{HTTP/1.1,[http/1.1]}{127.0.0.1:47058}
   [junit4]   2> 2627031 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.e.j.s.Server Started @2631838ms
   [junit4]   2> 2627031 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/tempDir-001/jetty3,
 solrconfig=solrconfig.xml, hostContext=/rcpl/k, hostPort=47058, 
coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-3-001/cores}
   [junit4]   2> 2627031 ERROR 
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 2627031 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.0.0
   [junit4]   2> 2627031 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2627031 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2627031 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2017-01-22T12:28:34.463Z
   [junit4]   2> 2627035 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 2627035 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-3-001/solr.xml
   [junit4]   2> 2627042 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48387/solr
   [junit4]   2> 2627057 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:47058_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (3)
   [junit4]   2> 2627064 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:47058_rcpl%2Fk    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:47058_rcpl%2Fk
   [junit4]   2> 2627066 INFO  
(zkCallback-3561-thread-2-processing-n:127.0.0.1:38838_rcpl%2Fk) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (3) -> (4)
   [junit4]   2> 2627066 INFO  (zkCallback-3565-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2627067 INFO  
(zkCallback-3570-thread-1-processing-n:127.0.0.1:44084_rcpl%2Fk) 
[n:127.0.0.1:44084_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (3) -> (4)
   [junit4]   2> 2627067 INFO  
(zkCallback-3576-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk) 
[n:127.0.0.1:33030_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (3) -> (4)
   [junit4]   2> 2627068 INFO  
(zkCallback-3583-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (3) -> (4)
   [junit4]   2> 2627109 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:47058_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-3-001/cores
   [junit4]   2> 2627109 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) 
[n:127.0.0.1:47058_rcpl%2Fk    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 2627113 INFO  
(OverseerStateUpdate-97326734270791684-127.0.0.1:38838_rcpl%2Fk-n_0000000000) 
[n:127.0.0.1:38838_rcpl%2Fk    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 2627467 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): 
collection=collection1, shard=shard1, thisCore=collection1, 
leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, 
currentState=recovering, localState=active, nodeName=127.0.0.1:33030_rcpl%2Fk, 
coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: 
core_node2:{"core":"collection1","base_url":"http://127.0.0.1:33030/rcpl/k","node_name":"127.0.0.1:33030_rcpl%2Fk","state":"recovering"}
   [junit4]   2> 2627467 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: 
recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 2627467 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={nodeName=127.0.0.1:33030_rcpl%252Fk&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
 status=0 QTime=1003
   [junit4]   2> 2628129 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2628150 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 2628254 WARN  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 2628255 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2628273 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection collection1
   [junit4]   2> 2628273 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] o.a.s.c.SolrCore 
[[collection1] ] Opening new SolrCore at 
[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-3-001/cores/collection1],
 
dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_9070C66A508CF174-001/shard-3-001/cores/collection1/data/]
   [junit4]   2> 2628273 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@556c5fad
   [junit4]   2> 2628275 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=25, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=66.572265625, 
floorSegmentMB=1.0556640625, forceMergeDeletesPctAllowed=20.05078628936581, 
segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.3936373592860306
   [junit4]   2> 2628284 WARN  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,attributes = 
{initParams=a, name=/dump, class=DumpRequestHandler},args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2628298 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2628298 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] o.a.s.u.UpdateLog 
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2628299 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2628299 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2628300 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@28216065
   [junit4]   2> 2628301 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@1ccdf6c8[collection1] main]
   [junit4]   2> 2628326 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 2628329 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2628334 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2628339 INFO  
(searcherExecutor-8627-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:47058_rcpl%2Fk c:collection1   
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@1ccdf6c8[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2628341 INFO  
(coreLoadExecutor-8626-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk) 
[n:127.0.0.1:47058_rcpl%2Fk c:collection1   x:collection1] o.a.s.u.UpdateLog 
Could not find max version in index or recent updates, using new clock 
1557227756084789248
   [junit4]   2> 2628355 INFO  
(coreZkRegister-8621-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 c:collection1) [n:127.0.0.1:47058_rcpl%2Fk c:collection1 s:shard1 
r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to 
recover:collection1
   [junit4]   2> 2628355 INFO  
(updateExecutor-3580-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState 
Running recovery
   [junit4]   2> 2628356 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 2628356 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
###### startupVersions=[[]]
   [junit4]   2> 2628356 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Begin buffering updates. core=[collection1]
   [junit4]   2> 2628356 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting 
to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 2628356 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Publishing state of core [collection1] as recovering, leader is 
[http://127.0.0.1:44084/rcpl/k/collection1/] and I am 
[http://127.0.0.1:47058/rcpl/k/collection1/]
   [junit4]   2> 2628361 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Sending prep recovery command to [http://127.0.0.1:44084/rcpl/k]; 
[WaitForState: 
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:47058_rcpl%252Fk&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 2628363 INFO  (qtp184447031-22629) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, 
state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 2628363 INFO  (qtp184447031-22629) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 
(shard1 of collection1) have state: recovering
   [junit4]   2> 2628363 INFO  (qtp184447031-22629) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): 
collection=collection1, shard=shard1, thisCore=collection1, 
leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, 
currentState=down, localState=active, nodeName=127.0.0.1:47058_rcpl%2Fk, 
coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: 
core_node3:{"core":"collection1","base_url":"http://127.0.0.1:47058/rcpl/k","node_name":"127.0.0.1:47058_rcpl%2Fk","state":"down"}
   [junit4]   2> 2628663 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2628663 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30 
for each attempt
   [junit4]   2> 2628663 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: 
collection1 failOnTimeout:true timeout (sec):30
   [junit4]   2> 2629371 INFO  (qtp184447031-22629) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): 
collection=collection1, shard=shard1, thisCore=collection1, 
leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, 
currentState=recovering, localState=active, nodeName=127.0.0.1:47058_rcpl%2Fk, 
coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: 
core_node3:{"core":"collection1","base_url":"http://127.0.0.1:47058/rcpl/k","node_name":"127.0.0.1:47058_rcpl%2Fk","state":"recovering"}
   [junit4]   2> 2629371 INFO  (qtp184447031-22629) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: 
recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 2629371 INFO  (qtp184447031-22629) [n:127.0.0.1:44084_rcpl%2Fk 
   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={nodeName=127.0.0.1:47058_rcpl%252Fk&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
 status=0 QTime=1008
   [junit4]   2> 2634470 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
Attempting to PeerSync from [http://127.0.0.1:44084/rcpl/k/collection1/] - 
recoveringAfterStartup=[true]
   [junit4]   2> 2634470 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync PeerSync: 
core=collection1 url=http://127.0.0.1:33030/rcpl/k START 
replicas=[http://127.0.0.1:44084/rcpl/k/collection1/] nUpdates=1000
   [junit4]   2> 2634472 INFO  (qtp184447031-22626) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint 
IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 2634472 INFO  (qtp184447031-22626) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp=/rcpl/k path=/get 
params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   2> 2634473 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.IndexFingerprint 
IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 2634473 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync We are 
already in sync. No need to do a PeerSync 
   [junit4]   2> 2634473 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 
start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2634473 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 
No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2634474 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 
end_commit_flush
   [junit4]   2> 2634474 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
PeerSync stage of recovery was successful.
   [junit4]   2> 2634474 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
Replaying updates buffered during PeerSync.
   [junit4]   2> 2634474 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy No 
replay needed.
   [junit4]   2> 2634474 INFO  
(recoveryExecutor-3574-thread-1-processing-n:127.0.0.1:33030_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:33030_rcpl%2Fk 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy 
Registering as Active after recovery.
   [junit4]   2> 2636373 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Attempting to PeerSync from [http://127.0.0.1:44084/rcpl/k/collection1/] - 
recoveringAfterStartup=[true]
   [junit4]   2> 2636374 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: 
core=collection1 url=http://127.0.0.1:47058/rcpl/k START 
replicas=[http://127.0.0.1:44084/rcpl/k/collection1/] nUpdates=1000
   [junit4]   2> 2636376 INFO  (qtp184447031-22627) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint 
IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 2636376 INFO  (qtp184447031-22627) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp=/rcpl/k path=/get 
params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   2> 2636377 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint 
IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 2636377 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync We are 
already in sync. No need to do a PeerSync 
   [junit4]   2> 2636377 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 
start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2636377 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 
No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2636378 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 
end_commit_flush
   [junit4]   2> 2636378 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
PeerSync stage of recovery was successful.
   [junit4]   2> 2636378 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Replaying updates buffered during PeerSync.
   [junit4]   2> 2636378 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No 
replay needed.
   [junit4]   2> 2636378 INFO  
(recoveryExecutor-3581-thread-1-processing-n:127.0.0.1:47058_rcpl%2Fk 
x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy 
Registering as Active after recovery.
   [junit4]   2> 2636691 INFO  
(TEST-PeerSyncReplicationTest.test-seed#[9070C66A508CF174]) [    ] 
o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 2636692 INFO  (qtp2028176514-22589) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2636692 INFO  (qtp2028176514-22589) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping 
IW.commit.
   [junit4]   2> 2636693 INFO  (qtp2028176514-22589) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2636693 INFO  (qtp2028176514-22589) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  
webapp=/rcpl/k path=/update 
params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
 0 1
   [junit4]   2> 2636696 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 
start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2636696 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 
No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2636697 INFO  (qtp1542813611-22651) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2636697 INFO  (qtp1542813611-22651) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2636698 INFO  (qtp1542813611-22651) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2636698 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 
end_commit_flush
   [junit4]   2> 2636698 INFO  (qtp1542813611-22651) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 1
   [junit4]   2> 2636698 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 1
   [junit4]   2> 2636701 INFO  (qtp103098067-22688) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 
start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2636701 INFO  (qtp103098067-22688) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 
No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2636701 INFO  (qtp103098067-22688) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 
end_commit_flush
   [junit4]   2> 2636701 INFO  (qtp103098067-22688) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 0
   [junit4]   2> 2636701 INFO  (qtp184447031-22630) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
 0 7
   [junit4]   2> 2636703 INFO  (qtp184447031-22626) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp=/rcpl/k path=/select 
params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2}
 hits=0 status=0 QTime=0
   [junit4]   2> 2636704 INFO  (qtp1542813611-22656) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.c.S.Request [collection1]  webapp=/rcpl/k path=/select 
params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2}
 hits=0 status=0 QTime=0
   [junit4]   2> 2636705 INFO  (qtp103098067-22690) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp=/rcpl/k path=/select 
params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2}
 hits=0 status=0 QTime=0
   [junit4]   2> 2638708 INFO  (qtp2028176514-22587) 
[n:127.0.0.1:38838_rcpl%2Fk c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  
webapp=/rcpl/k path=/update params={wt=javabin&version=2}{deleteByQuery=*:* 
(-1557227766954328064)} 0 1
   [junit4]   2> 2638713 INFO  (qtp103098067-22691) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&_version_=-1557227766956425216&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{deleteByQuery=*:*
 (-1557227766956425216)} 0 2
   [junit4]   2> 2638713 INFO  (qtp1542813611-22654) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&_version_=-1557227766956425216&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{deleteByQuery=*:*
 (-1557227766956425216)} 0 2
   [junit4]   2> 2638714 INFO  (qtp184447031-22627) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={wt=javabin&version=2}{deleteByQuery=*:* (-1557227766956425216)} 0 5
   [junit4]   2> 2638717 INFO  (qtp1542813611-22657) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[0
 (1557227766962716672)]} 0 1
   [junit4]   2> 2638718 INFO  (qtp103098067-22692) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[0
 (1557227766962716672)]} 0 1
   [junit4]   2> 2638718 INFO  (qtp184447031-22629) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={wt=javabin&version=2}{add=[0 (1557227766962716672)]} 0 2
   [junit4]   2> 2638719 INFO  (qtp1542813611-22658) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[1
 (1557227766966910976)]} 0 0
   [junit4]   2> 2638720 INFO  (qtp103098067-22693) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[1
 (1557227766966910976)]} 0 0
   [junit4]   2> 2638720 INFO  (qtp184447031-22623) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={wt=javabin&version=2}{add=[1 (1557227766966910976)]} 0 1
   [junit4]   2> 2638722 INFO  (qtp1542813611-22652) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[2
 (1557227766969008128)]} 0 0
   [junit4]   2> 2638722 INFO  (qtp103098067-22686) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[2
 (1557227766969008128)]} 0 0
   [junit4]   2> 2638722 INFO  (qtp184447031-22625) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={wt=javabin&version=2}{add=[2 (1557227766969008128)]} 0 1
   [junit4]   2> 2638724 INFO  (qtp1542813611-22651) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[3
 (1557227766971105280)]} 0 0
   [junit4]   2> 2638724 INFO  (qtp103098067-22688) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[3
 (1557227766971105280)]} 0 0
   [junit4]   2> 2638724 INFO  (qtp184447031-22630) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={wt=javabin&version=2}{add=[3 (1557227766971105280)]} 0 1
   [junit4]   2> 2638726 INFO  (qtp1542813611-22655) 
[n:127.0.0.1:33030_rcpl%2Fk c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[4
 (1557227766973202432)]} 0 0
   [junit4]   2> 2638727 INFO  (qtp103098067-22689) [n:127.0.0.1:47058_rcpl%2Fk 
c:collection1 s:shard1 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/rcpl/k path=/update 
params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:44084/rcpl/k/collection1/&wt=javabin&version=2}{add=[4
 (1557227766973202432)]} 0 0
   [junit4]   2> 2638727 INFO  (qtp184447031-22628) [n:127.0.0.1:44084_rcpl%2Fk 
c:collection1 

[...truncated too long message...]

search.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: 
codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY,
 chunkSize=11855, maxDocsPerChunk=786, blockSize=7), 
termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, 
chunkSize=11855, blockSize=7)), sim=RandomSimilarity(queryNorm=true): {}, 
locale=es-CU, timezone=America/Tijuana
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_121 
(64-bit)/cpus=3,threads=1,free=303464776,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolrDeletionPolicy1, 
TestConfigOverlay, BooleanFieldTest, TestXIncludeConfig, 
TestDFISimilarityFactory, BlockJoinFacetSimpleTest, JavabinLoaderTest, 
SampleTest, TestMinMaxOnMultiValuedField, TestHighFrequencyDictionaryFactory, 
TestCharFilters, TestChildDocTransformer, MigrateRouteKeyTest, EnumFieldTest, 
OverseerModifyCollectionTest, PrimitiveFieldTypeTest, 
TestSchemalessBufferedUpdates, TestDownShardTolerantSearch, 
TestClusterProperties, UnloadDistributedZkTest, ClusterStateUpdateTest, 
TestSweetSpotSimilarityFactory, DistributedMLTComponentTest, 
TestShortCircuitedRequests, TestMissingGroups, TestNRTOpen, 
TestRequestForwarding, TestDynamicFieldResource, 
TestDistributedStatsComponentCardinality, CollectionStateFormat2Test, 
SolrCoreMetricManagerTest, TestFieldTypeResource, TestShardHandlerFactory, 
DistributedQueueTest, TestRuleBasedAuthorizationPlugin, 
HdfsCollectionsAPIDistributedZkTest, TestRemoteStreaming, 
TestStressUserVersions, TestReplicationHandlerBackup, StressHdfsTest, 
RollingRestartTest, TestSubQueryTransformerCrossCore, 
DistributedQueryComponentOptimizationTest, TestSolrCloudWithHadoopAuthPlugin, 
TestWordDelimiterFilterFactory, TestElisionMultitermQuery, 
TestHdfsCloudBackupRestore, SearchHandlerTest, TestDefaultSearchFieldResource, 
PingRequestHandlerTest, TestFieldCacheReopen, TestDistributedSearch, 
TestReplicationHandler, DOMUtilTest, AddSchemaFieldsUpdateProcessorFactoryTest, 
TestRTGBase, TestFunctionQuery, AddBlockUpdateTest, TestSQLHandlerNonCloud, 
RequiredFieldsTest, TestSortByMinMaxFunction, ParsingFieldUpdateProcessorsTest, 
TestHdfsUpdateLog, TestPivotHelperCode, NumericFieldsTest, TestStressLiveNodes, 
TestPartialUpdateDeduplication, RecoveryZkTest, SimpleFacetsTest, 
TestCloudInspectUtil, TestSolr4Spatial2, TestQuerySenderNoQuery, 
TestComplexPhraseLeadingWildcard, TestRestoreCore, DocumentBuilderTest, 
TestPerFieldSimilarity, TestLFUCache, DeleteReplicaTest, 
TestReRankQParserPlugin, TestSSLRandomization, TestJsonRequest, 
TestFieldCacheVsDocValues, UUIDUpdateProcessorFallbackTest, 
TestRandomCollapseQParserPlugin, TestManagedSynonymFilterFactory, 
TestManagedResource, TestBulkSchemaConcurrent, TestHashPartitioner, 
UtilsToolTest, TestExtendedDismaxParser, SynonymTokenizerTest, CoreSorterTest, 
DateFieldTest, TestPseudoReturnFields, TestUpdate, CurrencyFieldXmlFileTest, 
TestPostingsSolrHighlighter, TestSolrCloudSnapshots, 
TestPerFieldSimilarityWithDefaultOverride, 
TestOverriddenPrefixQueryForCustomFieldType, HdfsChaosMonkeyNothingIsSafeTest, 
TimeZoneUtilsTest, TestStressReorder, ExplicitHLLTest, MultiThreadedOCPTest, 
XmlUpdateRequestHandlerTest, TestJettySolrRunner, TestSolrFieldCacheMBean, 
HighlighterTest, HdfsRecoverLeaseTest, TermsComponentTest, 
TestSimpleQParserPlugin, TestAnalyzeInfixSuggestions, TestIBSimilarityFactory, 
HdfsRestartWhileUpdatingTest, CollectionsAPIAsyncDistributedZkTest, 
HdfsDirectoryTest, PluginInfoTest, BlockDirectoryTest, 
HdfsBasicDistributedZk2Test, TestMiniSolrCloudClusterSSL, SolrPluginUtilsTest, 
TestSurroundQueryParser, IndexBasedSpellCheckerTest, 
TestTolerantUpdateProcessorRandomCloud, 
PeerSyncWithIndexFingerprintCachingTest, SystemInfoHandlerTest, 
TestUnifiedSolrHighlighter, TestJmxIntegration, ActionThrottleTest, 
OverriddenZkACLAndCredentialsProvidersTest, TestCloudPseudoReturnFields, 
TestPhraseSuggestions, TestBinaryResponseWriter, TestMiniSolrCloudCluster, 
AnalyticsQueryTest, TestHashQParserPlugin, CdcrRequestHandlerTest, 
TestInfoStreamLogging, SharedFSAutoReplicaFailoverTest, SOLR749Test, 
DistributedSpellCheckComponentTest, FileUtilsTest, TestConfigSetProperties, 
TestFileDictionaryLookup, TestRandomDVFaceting, 
BigEndianAscendingWordSerializerTest, RequestLoggingTest, 
TestLocalFSCloudBackupRestore, ZkCLITest, HdfsThreadLeakTest, 
ExitableDirectoryReaderTest, ClassificationUpdateProcessorTest, 
DirectUpdateHandlerTest, DocValuesNotIndexedTest, TestLeaderElectionZkExpiry, 
TestDocBasedVersionConstraints, PathHierarchyTokenizerFactoryTest, 
DistributedFacetPivotLargeTest, TestGroupingSearch, BasicAuthStandaloneTest, 
HdfsUnloadDistributedZkTest, CdcrReplicationDistributedZkTest, 
BasicDistributedZkTest, BasicDistributedZk2Test, SyncSliceTest, OverseerTest, 
LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, 
LeaderElectionTest, ZkSolrClientTest, ShardRoutingCustomTest, TestFaceting, 
TermVectorComponentDistributedTest, ZkControllerTest, TestReload, 
HardAutoCommitTest, TestCoreContainer, SolrCoreTest, PeerSyncTest, 
BasicFunctionalityTest, TestLazyCores, ShowFileRequestHandlerTest, 
CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, 
SimplePostToolTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, 
SuggesterFSTTest, DocValuesTest, DisMaxRequestHandlerTest, 
DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, 
MoreLikeThisHandlerTest, FastVectorHighlighterTest, 
IndexSchemaRuntimeFieldTest, TestCSVResponseWriter, TestAnalyzedSuggestions, 
BinaryUpdateRequestHandlerTest, HighlighterConfigTest, TestSolrIndexConfig, 
SolrIndexConfigTest, TestSolrDeletionPolicy2, MultiTermTest, TestBinaryField, 
TestFuzzyAnalyzedSuggestions, DirectSolrConnectionTest, 
NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, 
TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, 
TestFastWriter, TestFastOutputStream, ScriptEngineTest, TestFastLRUCache, 
ChaosMonkeyNothingIsSafeTest, DistributedIntervalFacetingTest, 
SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, 
TestSimpleTrackingShardHandler, CdcrReplicationHandlerTest, 
CdcrVersionReplicationTest, CleanupOldIndexTest, CollectionTooManyReplicasTest, 
ConnectionManagerTest, CustomCollectionTest, DeleteStatusTest, 
DistributedVersionInfoTest, LeaderFailoverAfterPartitionTest, 
LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnCommitTest, 
PeerSyncReplicationTest]
   [junit4] Completed [500/680 (1!)] on J0 in 210.85s, 1 test, 1 failure <<< 
FAILURES!

[...truncated 43219 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.lucene...
  [javadoc] Loading source files for package org.apache.lucene.analysis...
  [javadoc] Loading source files for package 
org.apache.lucene.analysis.standard...
  [javadoc] Loading source files for package 
org.apache.lucene.analysis.tokenattributes...
  [javadoc] Loading source files for package org.apache.lucene.codecs...
  [javadoc] Loading source files for package 
org.apache.lucene.codecs.blocktree...
  [javadoc] Loading source files for package 
org.apache.lucene.codecs.compressing...
  [javadoc] Loading source files for package 
org.apache.lucene.codecs.lucene50...
  [javadoc] Loading source files for package 
org.apache.lucene.codecs.lucene60...
  [javadoc] Loading source files for package 
org.apache.lucene.codecs.lucene62...
  [javadoc] Loading source files for package 
org.apache.lucene.codecs.lucene70...
  [javadoc] Loading source files for package 
org.apache.lucene.codecs.perfield...
  [javadoc] Loading source files for package org.apache.lucene.document...
  [javadoc] Loading source files for package org.apache.lucene.geo...
  [javadoc] Loading source files for package org.apache.lucene.index...
  [javadoc] Loading source files for package org.apache.lucene.search...
  [javadoc] Loading source files for package 
org.apache.lucene.search.similarities...
  [javadoc] Loading source files for package org.apache.lucene.search.spans...
  [javadoc] Loading source files for package org.apache.lucene.store...
  [javadoc] Loading source files for package org.apache.lucene.util...
  [javadoc] Loading source files for package org.apache.lucene.util.automaton...
  [javadoc] Loading source files for package org.apache.lucene.util.bkd...
  [javadoc] Loading source files for package org.apache.lucene.util.fst...
  [javadoc] Loading source files for package org.apache.lucene.util.graph...
  [javadoc] Loading source files for package org.apache.lucene.util.mutable...
  [javadoc] Loading source files for package org.apache.lucene.util.packed...
  [javadoc] Constructing Javadoc information...
  [javadoc] javadoc: error - Argument for -bottom contains JavaScript.
  [javadoc] Use --allow-script-in-comments to allow use of JavaScript.
  [javadoc] 1 error

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:775: The 
following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:101: The 
following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/build.xml:138: 
The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:746:
 The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/core/build.xml:54:
 The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:2101:
 Javadoc returned 1

Total time: 91 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to