Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1468/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([D5CB3A81DD306DB5]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([D5CB3A81DD306DB5]:0)




Build Log:
[...truncated 12812 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/init-core-data-001
   [junit4]   2> 826870 INFO  
(SUITE-BasicDistributedZkTest-seed#[D5CB3A81DD306DB5]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 826870 INFO  
(SUITE-BasicDistributedZkTest-seed#[D5CB3A81DD306DB5]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 826895 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 826895 INFO  (Thread-1642) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 826895 INFO  (Thread-1642) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 826995 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:37485
   [junit4]   2> 827015 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 827017 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 827018 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 827019 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 827021 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 827022 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 827023 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 827024 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 827025 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 827027 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 827028 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 827166 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/control-001/cores/collection1
   [junit4]   2> 827167 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 827175 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@74006c40{/,null,AVAILABLE}
   [junit4]   2> 827176 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@26356e1d{HTTP/1.1,[http/1.1]}{127.0.0.1:55848}
   [junit4]   2> 827176 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server Started @832242ms
   [junit4]   2> 827176 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/tempDir-001/control/data,
 hostContext=/, hostPort=55848, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/control-001/cores}
   [junit4]   2> 827176 ERROR 
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 827177 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 827177 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 827177 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 827177 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-09T05:22:00.618Z
   [junit4]   2> 827205 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 827206 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/control-001/solr.xml
   [junit4]   2> 827235 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37485/solr
   [junit4]   2> 827278 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:55848_  
  ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55848_
   [junit4]   2> 827279 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:55848_  
  ] o.a.s.c.Overseer Overseer 
(id=96906046363271172-127.0.0.1:55848_-n_0000000000) starting
   [junit4]   2> 827295 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:55848_  
  ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:55848_
   [junit4]   2> 827302 INFO  
(OverseerStateUpdate-96906046363271172-127.0.0.1:55848_-n_0000000000) 
[n:127.0.0.1:55848_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 827316 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:55848_  
  ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/control-001/cores
   [junit4]   2> 827317 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:55848_  
  ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 827338 INFO  
(OverseerStateUpdate-96906046363271172-127.0.0.1:55848_-n_0000000000) 
[n:127.0.0.1:55848_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 828349 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 828364 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 828495 WARN  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 828497 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 828510 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection 
control_collection
   [junit4]   2> 828511 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/control-001/cores/collection1],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/control-001/cores/collection1/data/]
   [junit4]   2> 828511 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bb98918
   [junit4]   2> 828512 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=3.302734375, 
floorSegmentMB=0.2421875, forceMergeDeletesPctAllowed=16.244544546560203, 
segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 828515 WARN  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 828529 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 828529 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 828530 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 828530 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 828530 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=49, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6973758522552682]
   [junit4]   2> 828531 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@2a482426[collection1] main]
   [junit4]   2> 828531 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 828532 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 828532 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 828532 INFO  
(coreLoadExecutor-3053-thread-1-processing-n:127.0.0.1:55848_) 
[n:127.0.0.1:55848_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1550496743526760448
   [junit4]   2> 828540 INFO  
(coreZkRegister-3046-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 828540 INFO  
(coreZkRegister-3046-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 828540 INFO  
(coreZkRegister-3046-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:55848/collection1/
   [junit4]   2> 828540 INFO  
(coreZkRegister-3046-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 828540 INFO  
(coreZkRegister-3046-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:55848/collection1/ has no replicas
   [junit4]   2> 828548 INFO  
(searcherExecutor-3054-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@2a482426[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 828549 INFO  
(coreZkRegister-3046-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:55848/collection1/ shard1
   [junit4]   2> 828700 INFO  
(coreZkRegister-3046-thread-1-processing-n:127.0.0.1:55848_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 828844 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 828845 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 828955 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-1-001/cores/collection1
   [junit4]   2> 828956 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-1-001
   [junit4]   2> 828957 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 828958 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@105b9447{/,null,AVAILABLE}
   [junit4]   2> 828958 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@3a1aed40{HTTP/1.1,[http/1.1]}{127.0.0.1:59540}
   [junit4]   2> 828958 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server Started @834024ms
   [junit4]   2> 828958 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/tempDir-001/jetty1,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=59540, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-1-001/cores}
   [junit4]   2> 828959 ERROR 
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 828959 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 828959 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 828959 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 828959 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-09T05:22:02.400Z
   [junit4]   2> 828963 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 828963 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-1-001/solr.xml
   [junit4]   2> 828971 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37485/solr
   [junit4]   2> 828983 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:59540_  
  ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 828987 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:59540_  
  ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:59540_
   [junit4]   2> 828992 INFO  (zkCallback-1147-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 828992 INFO  
(zkCallback-1143-thread-2-processing-n:127.0.0.1:55848_) [n:127.0.0.1:55848_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 828999 INFO  
(zkCallback-1152-thread-1-processing-n:127.0.0.1:59540_) [n:127.0.0.1:59540_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 829025 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:59540_  
  ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-1-001/cores
   [junit4]   2> 829025 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:59540_  
  ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 829027 INFO  
(OverseerStateUpdate-96906046363271172-127.0.0.1:55848_-n_0000000000) 
[n:127.0.0.1:55848_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard2
   [junit4]   2> 830051 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 7.0.0
   [junit4]   2> 830067 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 830232 WARN  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 830235 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded 
schema test/1.0 with uniqueid field id
   [junit4]   2> 830249 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 830250 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-1-001/cores/collection1],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 830250 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bb98918
   [junit4]   2> 830251 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=3.302734375, 
floorSegmentMB=0.2421875, forceMergeDeletesPctAllowed=16.244544546560203, 
segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 830261 WARN  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 830302 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 830302 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 830303 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 830303 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 830303 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=49, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6973758522552682]
   [junit4]   2> 830304 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@efe996b[collection1] main]
   [junit4]   2> 830305 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 830305 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 830305 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 830306 INFO  
(coreLoadExecutor-3064-thread-1-processing-n:127.0.0.1:59540_) 
[n:127.0.0.1:59540_ c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1550496745386934272
   [junit4]   2> 830313 INFO  
(searcherExecutor-3065-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new 
searcher Searcher@efe996b[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 830318 INFO  
(coreZkRegister-3059-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas 
found to continue.
   [junit4]   2> 830318 INFO  
(coreZkRegister-3059-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new 
leader - try and sync
   [junit4]   2> 830319 INFO  
(coreZkRegister-3059-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:59540/collection1/
   [junit4]   2> 830319 INFO  
(coreZkRegister-3059-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync 
replicas to me
   [junit4]   2> 830319 INFO  
(coreZkRegister-3059-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:59540/collection1/ has no replicas
   [junit4]   2> 830323 INFO  
(coreZkRegister-3059-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new 
leader: http://127.0.0.1:59540/collection1/ shard2
   [junit4]   2> 830474 INFO  
(coreZkRegister-3059-thread-1-processing-n:127.0.0.1:59540_ x:collection1 
s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:59540_ c:collection1 s:shard2 
r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery 
necessary
   [junit4]   2> 830714 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-2-001/cores/collection1
   [junit4]   2> 830715 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-2-001
   [junit4]   2> 830716 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 830717 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@c7071f6{/,null,AVAILABLE}
   [junit4]   2> 830718 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@218e4aa1{HTTP/1.1,[http/1.1]}{127.0.0.1:40982}
   [junit4]   2> 830718 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server Started @835784ms
   [junit4]   2> 830718 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/tempDir-001/jetty2,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=40982, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-2-001/cores}
   [junit4]   2> 830718 ERROR 
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 830718 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 830718 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 830718 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 830718 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-09T05:22:04.159Z
   [junit4]   2> 830738 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 830739 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-2-001/solr.xml
   [junit4]   2> 830748 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37485/solr
   [junit4]   2> 830765 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x15847892612000a, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 830774 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:40982_  
  ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 830778 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:40982_  
  ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:40982_
   [junit4]   2> 830780 INFO  (zkCallback-1147-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 830780 INFO  
(zkCallback-1143-thread-1-processing-n:127.0.0.1:55848_) [n:127.0.0.1:55848_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 830780 INFO  
(zkCallback-1152-thread-1-processing-n:127.0.0.1:59540_) [n:127.0.0.1:59540_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 830794 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:40982_  
  ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-2-001/cores
   [junit4]   2> 830794 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:40982_  
  ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 830795 INFO  
(zkCallback-1158-thread-1-processing-n:127.0.0.1:40982_) [n:127.0.0.1:40982_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 830796 INFO  
(OverseerStateUpdate-96906046363271172-127.0.0.1:55848_-n_0000000000) 
[n:127.0.0.1:55848_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 831811 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 7.0.0
   [junit4]   2> 831826 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 831946 WARN  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 831949 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded 
schema test/1.0 with uniqueid field id
   [junit4]   2> 831962 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 831962 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-2-001/cores/collection1],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-2-001/cores/collection1/data/]
   [junit4]   2> 831962 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bb98918
   [junit4]   2> 831964 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=3.302734375, 
floorSegmentMB=0.2421875, forceMergeDeletesPctAllowed=16.244544546560203, 
segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 831968 WARN  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 831977 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 831977 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 831978 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 831978 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 831978 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=49, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6973758522552682]
   [junit4]   2> 831979 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@5892d527[collection1] main]
   [junit4]   2> 831980 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 831980 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 831980 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 831981 INFO  
(searcherExecutor-3076-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new 
searcher Searcher@5892d527[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 831981 INFO  
(coreLoadExecutor-3075-thread-1-processing-n:127.0.0.1:40982_) 
[n:127.0.0.1:40982_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1550496747143299072
   [junit4]   2> 831989 INFO  
(coreZkRegister-3070-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas 
found to continue.
   [junit4]   2> 831989 INFO  
(coreZkRegister-3070-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new 
leader - try and sync
   [junit4]   2> 831989 INFO  
(coreZkRegister-3070-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:40982/collection1/
   [junit4]   2> 831989 INFO  
(coreZkRegister-3070-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync 
replicas to me
   [junit4]   2> 831989 INFO  
(coreZkRegister-3070-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:40982/collection1/ has no replicas
   [junit4]   2> 831993 INFO  
(coreZkRegister-3070-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new 
leader: http://127.0.0.1:40982/collection1/ shard1
   [junit4]   2> 832144 INFO  
(coreZkRegister-3070-thread-1-processing-n:127.0.0.1:40982_ x:collection1 
s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:40982_ c:collection1 s:shard1 
r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery 
necessary
   [junit4]   2> 832404 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-3-001/cores/collection1
   [junit4]   2> 832404 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-3-001
   [junit4]   2> 832405 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 832406 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@717af74e{/,null,AVAILABLE}
   [junit4]   2> 832407 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@6a67a699{HTTP/1.1,[http/1.1]}{127.0.0.1:58677}
   [junit4]   2> 832407 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server Started @837473ms
   [junit4]   2> 832407 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/tempDir-001/jetty3,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=58677, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-3-001/cores}
   [junit4]   2> 832408 ERROR 
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 832408 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 832408 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 832408 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 832408 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-09T05:22:05.849Z
   [junit4]   2> 832411 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 832411 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-3-001/solr.xml
   [junit4]   2> 832419 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37485/solr
   [junit4]   2> 832429 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:58677_  
  ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 832434 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:58677_  
  ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:58677_
   [junit4]   2> 832436 INFO  
(zkCallback-1152-thread-1-processing-n:127.0.0.1:59540_) [n:127.0.0.1:59540_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 832436 INFO  
(zkCallback-1158-thread-1-processing-n:127.0.0.1:40982_) [n:127.0.0.1:40982_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 832436 INFO  (zkCallback-1147-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 832436 INFO  
(zkCallback-1143-thread-1-processing-n:127.0.0.1:55848_) [n:127.0.0.1:55848_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 832438 INFO  
(zkCallback-1164-thread-1-processing-n:127.0.0.1:58677_) [n:127.0.0.1:58677_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 832447 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:58677_  
  ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-3-001/cores
   [junit4]   2> 832447 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:58677_  
  ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 832450 INFO  
(OverseerStateUpdate-96906046363271172-127.0.0.1:55848_-n_0000000000) 
[n:127.0.0.1:55848_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard2
   [junit4]   2> 833464 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 7.0.0
   [junit4]   2> 833490 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 833588 WARN  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 833591 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded 
schema test/1.0 with uniqueid field id
   [junit4]   2> 833602 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 833603 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-3-001/cores/collection1],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-3-001/cores/collection1/data/]
   [junit4]   2> 833603 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bb98918
   [junit4]   2> 833604 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=3.302734375, 
floorSegmentMB=0.2421875, forceMergeDeletesPctAllowed=16.244544546560203, 
segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 833608 WARN  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 833617 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 833617 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 833618 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 833618 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 833618 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=49, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6973758522552682]
   [junit4]   2> 833618 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@350118ce[collection1] main]
   [junit4]   2> 833620 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 833620 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 833620 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 833621 INFO  
(searcherExecutor-3087-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.SolrCore [collection1] Registered new 
searcher Searcher@350118ce[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 833621 INFO  
(coreLoadExecutor-3086-thread-1-processing-n:127.0.0.1:58677_) 
[n:127.0.0.1:58677_ c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1550496748862963712
   [junit4]   2> 833625 INFO  
(coreZkRegister-3081-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to 
recover:collection1
   [junit4]   2> 833626 INFO  
(updateExecutor-1161-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 833626 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. 
recoveringAfterStartup=true
   [junit4]   2> 833626 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 833626 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. 
core=[collection1]
   [junit4]   2> 833626 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. 
FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 833626 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core 
[collection1] as recovering, leader is [http://127.0.0.1:59540/collection1/] 
and I am [http://127.0.0.1:58677/collection1/]
   [junit4]   2> 833628 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery 
command to [http://127.0.0.1:59540]; [WaitForState: 
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:58677_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 833630 INFO  (qtp1822426758-6743) [n:127.0.0.1:59540_    ] 
o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: 
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 833631 INFO  (qtp1822426758-6743) [n:127.0.0.1:59540_    ] 
o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 
(shard2 of collection1) have state: recovering
   [junit4]   2> 833631 INFO  (qtp1822426758-6743) [n:127.0.0.1:59540_    ] 
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, 
shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? 
true, live=true, checkLive=true, currentState=down, localState=active, 
nodeName=127.0.0.1:58677_, coreNodeName=core_node3, 
onlyIfActiveCheckResult=false, nodeProps: 
core_node3:{"core":"collection1","base_url":"http://127.0.0.1:58677","node_name":"127.0.0.1:58677_","state":"down"}
   [junit4]   2> 834054 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-4-001/cores/collection1
   [junit4]   2> 834055 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-4-001
   [junit4]   2> 834056 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 834057 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@33dfa92{/,null,AVAILABLE}
   [junit4]   2> 834058 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@3d5490af{HTTP/1.1,[http/1.1]}{127.0.0.1:49822}
   [junit4]   2> 834058 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.e.j.s.Server Started @839123ms
   [junit4]   2> 834059 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/tempDir-001/jetty4,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=49822, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-4-001/cores}
   [junit4]   2> 834059 ERROR 
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 834059 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
7.0.0
   [junit4]   2> 834059 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 834059 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 834059 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-09T05:22:07.500Z
   [junit4]   2> 834062 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 834062 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-4-001/solr.xml
   [junit4]   2> 834069 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37485/solr
   [junit4]   2> 834079 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:49822_  
  ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 834083 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:49822_  
  ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:49822_
   [junit4]   2> 834091 INFO  
(zkCallback-1158-thread-1-processing-n:127.0.0.1:40982_) [n:127.0.0.1:40982_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 834091 INFO  
(zkCallback-1164-thread-1-processing-n:127.0.0.1:58677_) [n:127.0.0.1:58677_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 834092 INFO  (zkCallback-1147-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 834092 INFO  
(zkCallback-1152-thread-1-processing-n:127.0.0.1:59540_) [n:127.0.0.1:59540_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 834092 INFO  
(zkCallback-1143-thread-1-processing-n:127.0.0.1:55848_) [n:127.0.0.1:55848_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 834106 INFO  
(zkCallback-1171-thread-1-processing-n:127.0.0.1:49822_) [n:127.0.0.1:49822_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 834107 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:49822_  
  ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-4-001/cores
   [junit4]   2> 834107 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [n:127.0.0.1:49822_  
  ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 834109 INFO  
(OverseerStateUpdate-96906046363271172-127.0.0.1:55848_-n_0000000000) 
[n:127.0.0.1:55848_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 834631 INFO  (qtp1822426758-6743) [n:127.0.0.1:59540_    ] 
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, 
shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? 
true, live=true, checkLive=true, currentState=recovering, localState=active, 
nodeName=127.0.0.1:58677_, coreNodeName=core_node3, 
onlyIfActiveCheckResult=false, nodeProps: 
core_node3:{"core":"collection1","base_url":"http://127.0.0.1:58677","node_name":"127.0.0.1:58677_","state":"recovering"}
   [junit4]   2> 834631 INFO  (qtp1822426758-6743) [n:127.0.0.1:59540_    ] 
o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, 
checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 834631 INFO  (qtp1822426758-6743) [n:127.0.0.1:59540_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={nodeName=127.0.0.1:58677_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
 status=0 QTime=1000
   [junit4]   2> 835128 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 7.0.0
   [junit4]   2> 835143 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 835247 WARN  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1   x:collection1] o.a.s.s.IndexSchema 
[collection1] default search field in schema is text. WARNING: Deprecated, 
please use 'df' on request instead.
   [junit4]   2> 835249 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded 
schema test/1.0 with uniqueid field id
   [junit4]   2> 835265 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 835265 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-4-001/cores/collection1],
 
dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.BasicDistributedZkTest_D5CB3A81DD306DB5-001/shard-4-001/cores/collection1/data/]
   [junit4]   2> 835265 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bb98918
   [junit4]   2> 835266 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=13, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=3.302734375, 
floorSegmentMB=0.2421875, forceMergeDeletesPctAllowed=16.244544546560203, 
segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 835270 WARN  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 835279 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 835280 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 835280 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 835280 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 835281 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=49, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6973758522552682]
   [junit4]   2> 835281 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@3effe2ab[collection1] main]
   [junit4]   2> 835282 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 835282 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 835282 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 835283 INFO  
(searcherExecutor-3098-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.SolrCore [collection1] Registered new 
searcher Searcher@3effe2ab[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 835283 INFO  
(coreLoadExecutor-3097-thread-1-processing-n:127.0.0.1:49822_) 
[n:127.0.0.1:49822_ c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1550496750605697024
   [junit4]   2> 835288 INFO  
(coreZkRegister-3092-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.ZkController Core needs to 
recover:collection1
   [junit4]   2> 835288 INFO  
(updateExecutor-1168-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 835289 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. 
recoveringAfterStartup=true
   [junit4]   2> 835289 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 835289 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. 
core=[collection1]
   [junit4]   2> 835289 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. 
FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 835289 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core 
[collection1] as recovering, leader is [http://127.0.0.1:40982/collection1/] 
and I am [http://127.0.0.1:49822/collection1/]
   [junit4]   2> 835292 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery 
command to [http://127.0.0.1:40982]; [WaitForState: 
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:49822_&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 835293 INFO  (qtp1117654708-6771) [n:127.0.0.1:40982_    ] 
o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: 
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 835294 INFO  (qtp1117654708-6771) [n:127.0.0.1:40982_    ] 
o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 
(shard1 of collection1) have state: recovering
   [junit4]   2> 835294 INFO  (qtp1117654708-6771) [n:127.0.0.1:40982_    ] 
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, 
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? 
true, live=true, checkLive=true, currentState=down, localState=active, 
nodeName=127.0.0.1:49822_, coreNodeName=core_node4, 
onlyIfActiveCheckResult=false, nodeProps: 
core_node4:{"core":"collection1","base_url":"http://127.0.0.1:49822","node_name":"127.0.0.1:49822_","state":"down"}
   [junit4]   2> 835611 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 835612 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: 
collection1 failOnTimeout:true timeout (sec):330
   [junit4]   2> 836294 INFO  (qtp1117654708-6771) [n:127.0.0.1:40982_    ] 
o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, 
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? 
true, live=true, checkLive=true, currentState=recovering, localState=active, 
nodeName=127.0.0.1:49822_, coreNodeName=core_node4, 
onlyIfActiveCheckResult=false, nodeProps: 
core_node4:{"core":"collection1","base_url":"http://127.0.0.1:49822","node_name":"127.0.0.1:49822_","state":"recovering"}
   [junit4]   2> 836294 INFO  (qtp1117654708-6771) [n:127.0.0.1:40982_    ] 
o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node4, state: recovering, 
checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 836294 INFO  (qtp1117654708-6771) [n:127.0.0.1:40982_    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={nodeName=127.0.0.1:49822_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
 status=0 QTime=1000
   [junit4]   2> 841632 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync 
from [http://127.0.0.1:59540/collection1/] - recoveringAfterStartup=[true]
   [junit4]   2> 841633 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 
url=http://127.0.0.1:58677 START replicas=[http://127.0.0.1:59540/collection1/] 
nUpdates=100
   [junit4]   2> 841645 INFO  (qtp1822426758-6741) [n:127.0.0.1:59540_ 
c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint 
IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 841645 INFO  (qtp1822426758-6741) [n:127.0.0.1:59540_ 
c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp= path=/get 
params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint 
millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to 
do a PeerSync 
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted 
changes. Skipping IW.commit.
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery 
was successful.
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered 
during PeerSync.
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
   [junit4]   2> 841646 INFO  
(recoveryExecutor-1162-thread-1-processing-n:127.0.0.1:58677_ x:collection1 
s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:58677_ c:collection1 s:shard2 
r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active 
after recovery.
   [junit4]   2> 843295 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync 
from [http://127.0.0.1:40982/collection1/] - recoveringAfterStartup=[true]
   [junit4]   2> 843295 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 
url=http://127.0.0.1:49822 START replicas=[http://127.0.0.1:40982/collection1/] 
nUpdates=100
   [junit4]   2> 843309 INFO  (qtp1117654708-6769) [n:127.0.0.1:40982_ 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.IndexFingerprint 
IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 843310 INFO  (qtp1117654708-6769) [n:127.0.0.1:40982_ 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp= path=/get 
params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   2> 843315 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint 
millis:0.0 result:{maxVersionSpecified=9223372036854775807, 
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, 
maxDoc=0}
   [junit4]   2> 843315 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to 
do a PeerSync 
   [junit4]   2> 843315 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 843315 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted 
changes. Skipping IW.commit.
   [junit4]   2> 843315 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 843318 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery 
was successful.
   [junit4]   2> 843318 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered 
during PeerSync.
   [junit4]   2> 843318 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
   [junit4]   2> 843318 INFO  
(recoveryExecutor-1169-thread-1-processing-n:127.0.0.1:49822_ x:collection1 
s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:49822_ c:collection1 s:shard1 
r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active 
after recovery.
   [junit4]   2> 843614 INFO  
(TEST-BasicDistributedZkTest.test-seed#[D5CB3A81DD306DB5]) [    ] 
o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 843633 INFO  (qtp789967037-6706) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp= path=/update 
params={wt=javabin&version=2}{deleteByQuery=*:* (-1550496759359209472)} 0 2
   [junit4]   2> 843696 INFO  (qtp153755947-6799) [n:127.0.0.1:58677_ 
c:collection1 s:shard2 r:core_node3 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp= path=/update 
params={update.distrib=FROMLEADER&_version_=-1550496759377035264&distrib.from=http://127.0.0.1:59540/collection1/&wt=javabin&version=2}{deleteByQuery=*:*
 (-1550496759377035264)} 0 12
   [junit4]   2> 843696 INFO  (qtp1067304453-6834) [n:127.0.0.1:49822_ 
c:collection1 s:shard1 r:core_node4 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp= path=/update 
params={update.distrib=FROMLEADER&_version_=-1550496759379132416&distrib.from=http://127.0.0.1:40982/collection1/&wt=javabin&version=2}{deleteByQuery=*:*
 (-1550496759379132416)} 0 2
   [junit4]   2> 843697 INFO  (qtp1117654708-6768) [n:127.0.0.1:40982_ 
c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp= path=/update 
params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:59540/collection1/&wt=javabin&version=2}{deleteByQuery=*:*
 (-1550496759379132416)} 0 46
   [junit4]   2> 843697 INFO  (qtp1822426758-6743) [n:127.0.0.1:59540_ 
c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp= path=/update 
params={wt=javabin&version=2}{deleteByQuery=*:* (-1550496759377035264)} 0 63
   [junit4]   2> 843700 INFO  (qtp1822426758-6741) [n:127.0.0.1:59540_ 
c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp= path=/select 
params={q=*:*&distrib=false&sanity_check=is_empty&wt=javabin&version=2} hits=0 
status=0 QTime=0
   [junit4]   2> 843701 INFO  (qtp153755947-6797) [n:127.0.0.1:58677_ 
c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp= path=/select 
params={q=*:*&distrib=false&sanity_check=is_empty&wt=javabin&version=2} hits=0 
status=0 QTime=0
   [junit4]   2> 843703 INFO  (qtp1117654708-6772) [n:127.0.0.1:40982_ 
c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp= path=/select 
params={q=*:*&distrib=false&sanity_check=is_empty&wt=javabin&version=2} hits=0 
status=0 QTime=0
   [junit4]   2> 843704 INFO  (qtp1067304453-6836) [n:127.0.0.1:49822_ 
c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.S.Request 
[collection1]  webapp= path=/select 
params={q=*:*&distrib=false&sanity_check=is_empty&wt=javabin&version=2} hits=0 
status=0 QTime=0
   [junit4]   2> 843706 INFO  (qtp789967037-6707) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 843706 INFO  (qtp789967037-6707) [n:127.0.0.1:55848_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.

[...truncated too long message...]

 Method)
   [junit4]   2>         at 
sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
   [junit4]   2>         at 
sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207)
   [junit4]   2>         at 
java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
   [junit4]   2>         at 
java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
   [junit4]   2>         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
   [junit4]   2>         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2>         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2>         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
   [junit4]   2>         at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
   [junit4]   2>         at 
org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>   21) Thread[id=6839, 
name=org.eclipse.jetty.server.session.HashSessionManager@719cafe1Timer, 
state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{regex_dup_A_s=PostingsFormat(name=LuceneVarGapFixedInterval), 
other_tl1=PostingsFormat(name=LuceneFixedGap), 
regex_dup_B_s=BlockTreeOrds(blocksize=128), 
range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 SubjectTerms_mfacet=PostingsFormat(name=LuceneFixedGap), 
n_l1=BlockTreeOrds(blocksize=128), multiDefault=BlockTreeOrds(blocksize=128), 
intDefault=PostingsFormat(name=LuceneFixedGap), 
genre_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 n_td1=PostingsFormat(name=LuceneFixedGap), n_d1=BlockTreeOrds(blocksize=128), 
range_facet_l=BlockTreeOrds(blocksize=128), 
n_f1=PostingsFormat(name=LuceneFixedGap), 
series_t=PostingsFormat(name=LuceneFixedGap), 
n_tl1=PostingsFormat(name=LuceneFixedGap), 
oddField_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 n_tf1=BlockTreeOrds(blocksize=128), 
price=PostingsFormat(name=LuceneVarGapFixedInterval), 
a_t=BlockTreeOrds(blocksize=128), cat=BlockTreeOrds(blocksize=128), 
inStock=PostingsFormat(name=LuceneVarGapFixedInterval), 
sequence_i=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 
id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 text=PostingsFormat(name=LuceneVarGapFixedInterval), 
timestamp=BlockTreeOrds(blocksize=128), 
foo_i=PostingsFormat(name=LuceneFixedGap), 
val_i=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 n_dt1=PostingsFormat(name=LuceneFixedGap), 
author_t=PostingsFormat(name=LuceneFixedGap), 
a_i1=BlockTreeOrds(blocksize=128), 
n_ti1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 
rnd_b=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 _version_=PostingsFormat(name=LuceneFixedGap), 
foo_b=PostingsFormat(name=LuceneVarGapFixedInterval), 
n_tdt1=PostingsFormat(name=LuceneFixedGap), 
name=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 
foo_d=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 range_facet_i_dv=BlockTreeOrds(blocksize=128), 
foo_f=PostingsFormat(name=LuceneVarGapFixedInterval)}, 
docValues:{regex_dup_A_s=DocValuesFormat(name=Lucene70), 
other_tl1=DocValuesFormat(name=Direct), 
regex_dup_B_s=DocValuesFormat(name=Memory), 
range_facet_l_dv=DocValuesFormat(name=Asserting), 
SubjectTerms_mfacet=DocValuesFormat(name=Direct), 
n_l1=DocValuesFormat(name=Memory), multiDefault=DocValuesFormat(name=Memory), 
intDefault=DocValuesFormat(name=Direct), 
genre_s=DocValuesFormat(name=Asserting), n_td1=DocValuesFormat(name=Direct), 
n_d1=DocValuesFormat(name=Memory), range_facet_l=DocValuesFormat(name=Memory), 
n_f1=DocValuesFormat(name=Direct), series_t=DocValuesFormat(name=Direct), 
n_tl1=DocValuesFormat(name=Direct), oddField_s=DocValuesFormat(name=Asserting), 
n_tf1=DocValuesFormat(name=Memory), price=DocValuesFormat(name=Lucene70), 
a_t=DocValuesFormat(name=Memory), cat=DocValuesFormat(name=Memory), 
inStock=DocValuesFormat(name=Lucene70), 
sequence_i=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Asserting), 
text=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Memory), 
foo_i=DocValuesFormat(name=Direct), val_i=DocValuesFormat(name=Asserting), 
n_dt1=DocValuesFormat(name=Direct), author_t=DocValuesFormat(name=Direct), 
a_i1=DocValuesFormat(name=Memory), n_ti1=DocValuesFormat(name=Asserting), 
rnd_b=DocValuesFormat(name=Asserting), _version_=DocValuesFormat(name=Direct), 
foo_b=DocValuesFormat(name=Lucene70), n_tdt1=DocValuesFormat(name=Direct), 
name=DocValuesFormat(name=Asserting), foo_d=DocValuesFormat(name=Asserting), 
range_facet_i_dv=DocValuesFormat(name=Memory), 
foo_f=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1047, 
maxMBSortInHeap=5.252828090717796, sim=ClassicSimilarity, locale=de-GR, 
timezone=Asia/Dili
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=36,free=329060688,total=486539264
   [junit4]   2> NOTE: All tests run in this JVM: [TestFastLRUCache, 
AnalyticsMergeStrategyTest, CollectionReloadTest, ConfigSetsAPITest, 
TestNRTOpen, SpellCheckComponentTest, IndexBasedSpellCheckerTest, 
TestSerializedLuceneMatchVersion, TestDistribDocBasedVersion, HLLUtilTest, 
TestComplexPhraseQParserPlugin, ChaosMonkeyNothingIsSafeTest, 
TestStressRecovery, TimeZoneUtilsTest, TestHdfsCloudBackupRestore, 
TestCharFilters, TestFilteredDocIdSet, FileUtilsTest, 
ChaosMonkeySafeLeaderTest, DateRangeFieldTest, TestSizeLimitedDistributedMap, 
TestCursorMarkWithoutUniqueKey, SolrInfoMBeanTest, TestSolrIndexConfig, 
OpenExchangeRatesOrgProviderTest, TestXIncludeConfig, 
DistributedSuggestComponentTest, DocumentBuilderTest, TestRestManager, 
OverseerModifyCollectionTest, TestReRankQParserPlugin, TestCoreContainer, 
TestPartialUpdateDeduplication, TestNumericTerms32, 
CoreMergeIndexesAdminHandlerTest, BaseCdcrDistributedZkTest, 
TestSolrCLIRunExample, TestCollectionAPI, TestSystemIdResolver, 
SoftAutoCommitTest, TestMissingGroups, BigEndianAscendingWordDeserializerTest, 
TestElisionMultitermQuery, TestExpandComponent, RestartWhileUpdatingTest, 
LoggingHandlerTest, TestDynamicFieldResource, AliasIntegrationTest, 
QueryResultKeyTest, HdfsBasicDistributedZkTest, ZkCLITest, 
DataDrivenBlockJoinTest, TestCloudPseudoReturnFields, 
DistributedTermsComponentTest, AlternateDirectoryTest, 
TestPostingsSolrHighlighter, TestUpdate, TestCorePropertiesReload, 
AddSchemaFieldsUpdateProcessorFactoryTest, TestRawResponseWriter, 
DistributedFacetPivotWhiteBoxTest, BadCopyFieldTest, TestSQLHandler, 
TestMinMaxOnMultiValuedField, TestConfigSetsAPIZkFailure, 
TestWordDelimiterFilterFactory, SchemaVersionSpecificBehaviorTest, 
BlockJoinFacetSimpleTest, TestTolerantUpdateProcessorCloud, ResponseHeaderTest, 
TestFieldTypeCollectionResource, SpellCheckCollatorTest, 
TestGeoJSONResponseWriter, HighlighterTest, BufferStoreTest, 
CustomCollectionTest, TestImplicitCoreProperties, HdfsThreadLeakTest, 
QueryParsingTest, TestSimpleTrackingShardHandler, BasicFunctionalityTest, 
JsonLoaderTest, TestScoreJoinQPScore, MergeStrategyTest, 
TestNamedUpdateProcessors, SuggestComponentContextFilterQueryTest, TestMacros, 
TestXmlQParserPlugin, TestFiltering, MoreLikeThisHandlerTest, 
TestStressVersions, TestUniqueKeyFieldResource, JavabinLoaderTest, 
SolrIndexSplitterTest, TestFieldCacheSanityChecker, SolrCloudExampleTest, 
TestQueryTypes, TestSolr4Spatial2, TestConfigSetImmutable, BJQParserTest, 
DirectoryFactoryTest, CloneFieldUpdateProcessorFactoryTest, 
RemoteQueryErrorTest, TestStressUserVersions, BasicDistributedZkTest]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=BasicDistributedZkTest -Dtests.seed=D5CB3A81DD306DB5 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=de-GR 
-Dtests.timezone=Asia/Dili -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | BasicDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 
7200000 msec).
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([D5CB3A81DD306DB5]:0)
   [junit4] Completed [648/648 (1!)] on J1 in 7223.96s, 1 test, 2 errors <<< 
FAILURES!

[...truncated 62228 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to