Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20519/
Java: 32bit/jdk-9-ea+181 -client -XX:+UseG1GC --illegal-access=deny
1 tests failed.
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitWithChaosMonkey
Error Message:
There are still nodes recoverying - waited for 330 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330
seconds
at
__randomizedtesting.SeedInfo.seed([D8504D6784902EB3:53779EB6C5968537]:0)
at org.junit.Assert.fail(Assert.java:93)
at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:908)
at
org.apache.solr.cloud.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:436)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 11817 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> Creating dataDir:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/init-core-data-001
[junit4] 2> 68492 WARN
(SUITE-ShardSplitTest-seed#[D8504D6784902EB3]-worker) [ ]
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=27 numCloses=27
[junit4] 2> 68493 INFO
(SUITE-ShardSplitTest-seed#[D8504D6784902EB3]-worker) [ ]
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 68494 INFO
(SUITE-ShardSplitTest-seed#[D8504D6784902EB3]-worker) [ ]
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via:
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776";)
[junit4] 2> 68494 INFO
(SUITE-ShardSplitTest-seed#[D8504D6784902EB3]-worker) [ ]
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 68494 INFO
(SUITE-ShardSplitTest-seed#[D8504D6784902EB3]-worker) [ ]
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 68495 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 68496 INFO (Thread-157) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 68496 INFO (Thread-157) [ ] o.a.s.c.ZkTestServer Starting
server
[junit4] 2> 68513 ERROR (Thread-157) [ ] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 68596 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkTestServer start zk server on port:37935
[junit4] 2> 68599 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from
client sessionid 0x15ea93a0d2c0000, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 68603 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
to /configs/conf1/solrconfig.xml
[junit4] 2> 68604 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml
to /configs/conf1/schema.xml
[junit4] 2> 68604 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 68605 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
to /configs/conf1/stopwords.txt
[junit4] 2> 68606 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
to /configs/conf1/protwords.txt
[junit4] 2> 68606 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
to /configs/conf1/currency.xml
[junit4] 2> 68607 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
to /configs/conf1/enumsConfig.xml
[junit4] 2> 68608 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
to /configs/conf1/open-exchange-rates.json
[junit4] 2> 68609 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 68609 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
to /configs/conf1/old_synonyms.txt
[junit4] 2> 68610 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractZkTestCase put
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
to /configs/conf1/synonyms.txt
[junit4] 2> 68612 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly
asked otherwise
[junit4] 2> 68692 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 68692 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@1cfb82e{/,null,AVAILABLE}
[junit4] 2> 68693 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@fa1b01{HTTP/1.1,[http/1.1]}{127.0.0.1:45853}
[junit4] 2> 68694 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server Started @70736ms
[junit4] 2> 68694 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/,
solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/tempDir-001/control/data,
hostPort=45853,
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/control-001/cores}
[junit4] 2> 68694 ERROR
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 68694 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version
8.0.0
[junit4] 2> 68694 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 68694 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config
dir: null
[junit4] 2> 68694 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2017-09-22T10:55:07.762925Z
[junit4] 2> 68698 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 68698 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/control-001/solr.xml
[junit4] 2> 68701 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig MBean server found:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac, but no JMX reporters were
configured - adding default JMX reporter.
[junit4] 2> 68705 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37935/solr
[junit4] 2> 68753 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 68753 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45853_
[junit4] 2> 68753 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.Overseer Overseer (id=98702308642914308-127.0.0.1:45853_-n_0000000000)
starting
[junit4] 2> 68757 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:45853_
[junit4] 2> 68757 INFO (zkCallback-76-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 68812 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 68818 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 68819 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 68820 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/control-001/cores
[junit4] 2> 68837 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 68838 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37935/solr ready
[junit4] 2> 68840 INFO (qtp32331028-555) [ ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params
replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45853_&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 68842 INFO (OverseerThreadFactory-257-thread-1) [ ]
o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 68950 INFO (qtp32331028-551) [ ]
o.a.s.h.a.CoreAdminOperation core create command
qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 68950 INFO (qtp32331028-551) [ ]
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4
transient cores
[junit4] 2> 69053 INFO (zkCallback-76-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/control_collection/state.json] for collection
[control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 69975 INFO (qtp32331028-551) [ ] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 8.0.0
[junit4] 2> 69985 INFO (qtp32331028-551) [ ] o.a.s.s.IndexSchema
[control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 70046 INFO (qtp32331028-551) [ ] o.a.s.s.IndexSchema
Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 70063 INFO (qtp32331028-551) [ ] o.a.s.c.CoreContainer
Creating SolrCore 'control_collection_shard1_replica_n1' using configuration
from collection control_collection, trusted=true
[junit4] 2> 70063 INFO (qtp32331028-551) [ ] o.a.s.m.r.SolrJmxReporter
JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry
'solr.core.control_collection.shard1.replica_n1') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 70066 INFO (qtp32331028-551) [ ] o.a.s.c.SolrCore
solr.RecoveryStrategy.Builder
[junit4] 2> 70066 INFO (qtp32331028-551) [ ] o.a.s.c.SolrCore
[[control_collection_shard1_replica_n1] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/control-001/cores/control_collection_shard1_replica_n1],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 70069 INFO (qtp32331028-551) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=44,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.0]
[junit4] 2> 70090 WARN (qtp32331028-551) [ ] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,attributes = {initParams=a, name=/dump,
class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 70132 INFO (qtp32331028-551) [ ] o.a.s.u.UpdateHandler
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 70132 INFO (qtp32331028-551) [ ] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 70133 INFO (qtp32331028-551) [ ] o.a.s.u.CommitTracker
Hard AutoCommit: disabled
[junit4] 2> 70133 INFO (qtp32331028-551) [ ] o.a.s.u.CommitTracker
Soft AutoCommit: disabled
[junit4] 2> 70134 INFO (qtp32331028-551) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=36,
maxMergedSegmentMB=33.3642578125, floorSegmentMB=0.4033203125,
forceMergeDeletesPctAllowed=3.6241769498282306, segmentsPerTier=43.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 70135 INFO (qtp32331028-551) [ ] o.a.s.s.SolrIndexSearcher
Opening [Searcher@13fc35f[control_collection_shard1_replica_n1] main]
[junit4] 2> 70136 INFO (qtp32331028-551) [ ]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 70136 INFO (qtp32331028-551) [ ]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 70137 INFO (qtp32331028-551) [ ]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 70138 INFO (searcherExecutor-260-thread-1) [ ]
o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher
Searcher@13fc35f[control_collection_shard1_replica_n1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 70138 INFO (qtp32331028-551) [ ] o.a.s.u.UpdateLog Could
not find max version in index or recent updates, using new clock
1579236940008390656
[junit4] 2> 70141 INFO (qtp32331028-551) [ ]
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 70141 INFO (qtp32331028-551) [ ]
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 70141 INFO (qtp32331028-551) [ ] o.a.s.c.SyncStrategy Sync
replicas to http://127.0.0.1:45853/control_collection_shard1_replica_n1/
[junit4] 2> 70141 INFO (qtp32331028-551) [ ] o.a.s.c.SyncStrategy Sync
Success - now sync replicas to me
[junit4] 2> 70141 INFO (qtp32331028-551) [ ] o.a.s.c.SyncStrategy
http://127.0.0.1:45853/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 70142 INFO (qtp32331028-551) [ ]
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
election, clear LIR
[junit4] 2> 70143 INFO (qtp32331028-551) [ ]
o.a.s.c.ShardLeaderElectionContext I am the new leader:
http://127.0.0.1:45853/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 70246 INFO (zkCallback-76-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/control_collection/state.json] for collection
[control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 70295 INFO (qtp32331028-551) [ ] o.a.s.c.ZkController I am
the leader, no recovery necessary
[junit4] 2> 70296 INFO (qtp32331028-551) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT}
status=0 QTime=1346
[junit4] 2> 70299 INFO (qtp32331028-555) [ ]
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most
30 seconds. Check all shard replicas
[junit4] 2> 70399 INFO (zkCallback-76-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/control_collection/state.json] for collection
[control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 70847 INFO
(OverseerCollectionConfigSetProcessor-98702308642914308-127.0.0.1:45853_-n_0000000000)
[ ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may
have disconnected from ZooKeeper
[junit4] 2> 71299 INFO (qtp32331028-555) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/collections
params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45853_&wt=javabin&version=2}
status=0 QTime=2458
[junit4] 2> 71304 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 71304 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37935/solr ready
[junit4] 2> 71304 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection
loss:false
[junit4] 2> 71309 INFO (qtp32331028-555) [ ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params
replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 71312 INFO (OverseerThreadFactory-257-thread-2) [ ]
o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 71312 WARN (OverseerThreadFactory-257-thread-2) [ ]
o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1)
without cores.
[junit4] 2> 71515 INFO (qtp32331028-555) [ ]
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most
30 seconds. Check all shard replicas
[junit4] 2> 71516 INFO (qtp32331028-555) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/collections
params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2}
status=0 QTime=206
[junit4] 2> 71614 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-1-001
of type NRT
[junit4] 2> 71615 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 71621 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@dba16b{/,null,AVAILABLE}
[junit4] 2> 71621 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@7027b6{HTTP/1.1,[http/1.1]}{127.0.0.1:41787}
[junit4] 2> 71621 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server Started @73663ms
[junit4] 2> 71621 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/,
solrconfig=solrconfig.xml,
solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/tempDir-001/jetty1,
hostPort=41787,
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-1-001/cores,
replicaType=NRT}
[junit4] 2> 71621 ERROR
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 71621 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version
8.0.0
[junit4] 2> 71622 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 71622 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config
dir: null
[junit4] 2> 71622 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2017-09-22T10:55:10.690091Z
[junit4] 2> 71626 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 71627 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-1-001/solr.xml
[junit4] 2> 71630 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig MBean server found:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac, but no JMX reporters were
configured - adding default JMX reporter.
[junit4] 2> 71633 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37935/solr
[junit4] 2> 71640 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 71641 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 71642 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:41787_
[junit4] 2> 71648 INFO (zkCallback-83-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 71648 INFO (zkCallback-76-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 71649 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 71838 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 71845 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 71845 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 71847 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-1-001/cores
[junit4] 2> 71863 INFO (qtp32331028-556) [ ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params
node=127.0.0.1:41787_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 71865 INFO
(OverseerCollectionConfigSetProcessor-98702308642914308-127.0.0.1:45853_-n_0000000000)
[ ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may
have disconnected from ZooKeeper
[junit4] 2> 71865 INFO (OverseerThreadFactory-257-thread-3) [ ]
o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:41787_ for creating new replica
[junit4] 2> 71869 INFO (qtp21558926-597) [ ]
o.a.s.h.a.CoreAdminOperation core create command
qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 71869 INFO (qtp21558926-597) [ ]
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4
transient cores
[junit4] 2> 71973 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 72878 INFO (qtp21558926-597) [ ] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 8.0.0
[junit4] 2> 72885 INFO (qtp21558926-597) [ ] o.a.s.s.IndexSchema
[collection1_shard2_replica_n41] Schema name=test
[junit4] 2> 72971 INFO (qtp21558926-597) [ ] o.a.s.s.IndexSchema
Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 72989 INFO (qtp21558926-597) [ ] o.a.s.c.CoreContainer
Creating SolrCore 'collection1_shard2_replica_n41' using configuration from
collection collection1, trusted=true
[junit4] 2> 72990 INFO (qtp21558926-597) [ ] o.a.s.m.r.SolrJmxReporter
JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry
'solr.core.collection1.shard2.replica_n41') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 72990 INFO (qtp21558926-597) [ ] o.a.s.c.SolrCore
solr.RecoveryStrategy.Builder
[junit4] 2> 72990 INFO (qtp21558926-597) [ ] o.a.s.c.SolrCore
[[collection1_shard2_replica_n41] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-1-001/cores/collection1_shard2_replica_n41],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
[junit4] 2> 72995 INFO (qtp21558926-597) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=44,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.0]
[junit4] 2> 73015 WARN (qtp21558926-597) [ ] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,attributes = {initParams=a, name=/dump,
class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 73066 INFO (qtp21558926-597) [ ] o.a.s.u.UpdateHandler
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 73066 INFO (qtp21558926-597) [ ] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 73067 INFO (qtp21558926-597) [ ] o.a.s.u.CommitTracker
Hard AutoCommit: disabled
[junit4] 2> 73067 INFO (qtp21558926-597) [ ] o.a.s.u.CommitTracker
Soft AutoCommit: disabled
[junit4] 2> 73069 INFO (qtp21558926-597) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=36,
maxMergedSegmentMB=33.3642578125, floorSegmentMB=0.4033203125,
forceMergeDeletesPctAllowed=3.6241769498282306, segmentsPerTier=43.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 73069 INFO (qtp21558926-597) [ ] o.a.s.s.SolrIndexSearcher
Opening [Searcher@613fb2[collection1_shard2_replica_n41] main]
[junit4] 2> 73070 INFO (qtp21558926-597) [ ]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 73070 INFO (qtp21558926-597) [ ]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 73071 INFO (qtp21558926-597) [ ]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 73072 INFO (searcherExecutor-271-thread-1) [ ]
o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher
Searcher@613fb2[collection1_shard2_replica_n41]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 73072 INFO (qtp21558926-597) [ ] o.a.s.u.UpdateLog Could
not find max version in index or recent updates, using new clock
1579236943084912640
[junit4] 2> 73076 INFO (qtp21558926-597) [ ]
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 73076 INFO (qtp21558926-597) [ ]
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 73076 INFO (qtp21558926-597) [ ] o.a.s.c.SyncStrategy Sync
replicas to http://127.0.0.1:41787/collection1_shard2_replica_n41/
[junit4] 2> 73076 INFO (qtp21558926-597) [ ] o.a.s.c.SyncStrategy Sync
Success - now sync replicas to me
[junit4] 2> 73076 INFO (qtp21558926-597) [ ] o.a.s.c.SyncStrategy
http://127.0.0.1:41787/collection1_shard2_replica_n41/ has no replicas
[junit4] 2> 73076 INFO (qtp21558926-597) [ ]
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
election, clear LIR
[junit4] 2> 73078 INFO (qtp21558926-597) [ ]
o.a.s.c.ShardLeaderElectionContext I am the new leader:
http://127.0.0.1:41787/collection1_shard2_replica_n41/ shard2
[junit4] 2> 73179 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 73228 INFO (qtp21558926-597) [ ] o.a.s.c.ZkController I am
the leader, no recovery necessary
[junit4] 2> 73229 INFO (qtp21558926-597) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT}
status=0 QTime=1360
[junit4] 2> 73234 INFO (qtp32331028-556) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/collections
params={node=127.0.0.1:41787_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2}
status=0 QTime=1370
[junit4] 2> 73318 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-2-001
of type NRT
[junit4] 2> 73318 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 73319 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@10ec4e6{/,null,AVAILABLE}
[junit4] 2> 73319 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@1406328{HTTP/1.1,[http/1.1]}{127.0.0.1:43017}
[junit4] 2> 73319 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server Started @75361ms
[junit4] 2> 73319 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/,
solrconfig=solrconfig.xml,
solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/tempDir-001/jetty2,
hostPort=43017,
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-2-001/cores}
[junit4] 2> 73320 ERROR
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 73320 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version
8.0.0
[junit4] 2> 73320 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 73320 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config
dir: null
[junit4] 2> 73320 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2017-09-22T10:55:12.388338Z
[junit4] 2> 73322 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 73322 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-2-001/solr.xml
[junit4] 2> 73325 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig MBean server found:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac, but no JMX reporters were
configured - adding default JMX reporter.
[junit4] 2> 73327 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37935/solr
[junit4] 2> 73335 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [2])
[junit4] 2> 73335 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 73336 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 73337 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:43017_
[junit4] 2> 73339 INFO (zkCallback-76-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 73339 INFO (zkCallback-83-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 73339 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 73340 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 73405 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 73413 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 73413 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 73414 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-2-001/cores
[junit4] 2> 73438 INFO (qtp14852282-626) [ ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params
node=127.0.0.1:43017_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 73443 INFO
(OverseerCollectionConfigSetProcessor-98702308642914308-127.0.0.1:45853_-n_0000000000)
[ ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may
have disconnected from ZooKeeper
[junit4] 2> 73443 INFO (OverseerThreadFactory-257-thread-4) [ ]
o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43017_ for creating new replica
[junit4] 2> 73445 INFO (qtp14852282-622) [ ]
o.a.s.h.a.CoreAdminOperation core create command
qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 73445 INFO (qtp14852282-622) [ ]
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4
transient cores
[junit4] 2> 73553 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 73553 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 74454 INFO (qtp14852282-622) [ ] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 8.0.0
[junit4] 2> 74461 INFO (qtp14852282-622) [ ] o.a.s.s.IndexSchema
[collection1_shard1_replica_n43] Schema name=test
[junit4] 2> 74507 INFO (qtp14852282-622) [ ] o.a.s.s.IndexSchema
Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 74519 INFO (qtp14852282-622) [ ] o.a.s.c.CoreContainer
Creating SolrCore 'collection1_shard1_replica_n43' using configuration from
collection collection1, trusted=true
[junit4] 2> 74519 INFO (qtp14852282-622) [ ] o.a.s.m.r.SolrJmxReporter
JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry
'solr.core.collection1.shard1.replica_n43') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 74519 INFO (qtp14852282-622) [ ] o.a.s.c.SolrCore
solr.RecoveryStrategy.Builder
[junit4] 2> 74519 INFO (qtp14852282-622) [ ] o.a.s.c.SolrCore
[[collection1_shard1_replica_n43] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-2-001/cores/collection1_shard1_replica_n43],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
[junit4] 2> 74522 INFO (qtp14852282-622) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=44,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.0]
[junit4] 2> 74542 WARN (qtp14852282-622) [ ] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,attributes = {initParams=a, name=/dump,
class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 74593 INFO (qtp14852282-622) [ ] o.a.s.u.UpdateHandler
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 74593 INFO (qtp14852282-622) [ ] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 74608 INFO (qtp14852282-622) [ ] o.a.s.u.CommitTracker
Hard AutoCommit: disabled
[junit4] 2> 74608 INFO (qtp14852282-622) [ ] o.a.s.u.CommitTracker
Soft AutoCommit: disabled
[junit4] 2> 74610 INFO (qtp14852282-622) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=36,
maxMergedSegmentMB=33.3642578125, floorSegmentMB=0.4033203125,
forceMergeDeletesPctAllowed=3.6241769498282306, segmentsPerTier=43.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 74610 INFO (qtp14852282-622) [ ] o.a.s.s.SolrIndexSearcher
Opening [Searcher@c2bfab[collection1_shard1_replica_n43] main]
[junit4] 2> 74611 INFO (qtp14852282-622) [ ]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 74612 INFO (qtp14852282-622) [ ]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 74613 INFO (qtp14852282-622) [ ]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 74615 INFO (searcherExecutor-282-thread-1) [ ]
o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher
Searcher@c2bfab[collection1_shard1_replica_n43]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 74615 INFO (qtp14852282-622) [ ] o.a.s.u.UpdateLog Could
not find max version in index or recent updates, using new clock
1579236944702865408
[junit4] 2> 74618 INFO (qtp14852282-622) [ ]
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 74619 INFO (qtp14852282-622) [ ]
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 74619 INFO (qtp14852282-622) [ ] o.a.s.c.SyncStrategy Sync
replicas to http://127.0.0.1:43017/collection1_shard1_replica_n43/
[junit4] 2> 74619 INFO (qtp14852282-622) [ ] o.a.s.c.SyncStrategy Sync
Success - now sync replicas to me
[junit4] 2> 74619 INFO (qtp14852282-622) [ ] o.a.s.c.SyncStrategy
http://127.0.0.1:43017/collection1_shard1_replica_n43/ has no replicas
[junit4] 2> 74619 INFO (qtp14852282-622) [ ]
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in
election, clear LIR
[junit4] 2> 74620 INFO (qtp14852282-622) [ ]
o.a.s.c.ShardLeaderElectionContext I am the new leader:
http://127.0.0.1:43017/collection1_shard1_replica_n43/ shard1
[junit4] 2> 74721 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 74721 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [3])
[junit4] 2> 74771 INFO (qtp14852282-622) [ ] o.a.s.c.ZkController I am
the leader, no recovery necessary
[junit4] 2> 74772 INFO (qtp14852282-622) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT}
status=0 QTime=1327
[junit4] 2> 74774 INFO (qtp14852282-626) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/collections
params={node=127.0.0.1:43017_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2}
status=0 QTime=1335
[junit4] 2> 74839 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-3-001
of type NRT
[junit4] 2> 74840 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 74840 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@e3af85{/,null,AVAILABLE}
[junit4] 2> 74841 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@1a5ee3e{HTTP/1.1,[http/1.1]}{127.0.0.1:42993}
[junit4] 2> 74841 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server Started @76883ms
[junit4] 2> 74841 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/,
solrconfig=solrconfig.xml,
solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/tempDir-001/jetty3,
hostPort=42993,
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-3-001/cores}
[junit4] 2> 74841 ERROR
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 74841 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version
8.0.0
[junit4] 2> 74841 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 74841 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config
dir: null
[junit4] 2> 74841 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2017-09-22T10:55:13.909957Z
[junit4] 2> 74844 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 74844 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-3-001/solr.xml
[junit4] 2> 74846 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig MBean server found:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac, but no JMX reporters were
configured - adding default JMX reporter.
[junit4] 2> 74848 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37935/solr
[junit4] 2> 74854 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 74855 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 74856 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:42993_
[junit4] 2> 74857 INFO (zkCallback-76-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 74861 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 74857 INFO (zkCallback-83-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 74857 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 74861 INFO (zkCallback-100-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 74954 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 74957 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 74957 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 74966 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 74966 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 74968 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-3-001/cores
[junit4] 2> 75015 INFO (qtp14852282-627) [ ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params
node=127.0.0.1:42993_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 75016 INFO
(OverseerCollectionConfigSetProcessor-98702308642914308-127.0.0.1:45853_-n_0000000000)
[ ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may
have disconnected from ZooKeeper
[junit4] 2> 75017 INFO (OverseerThreadFactory-257-thread-5) [ ]
o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:42993_ for creating new replica
[junit4] 2> 75018 INFO (qtp30155223-655) [ ]
o.a.s.h.a.CoreAdminOperation core create command
qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 75019 INFO (qtp30155223-655) [ ]
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4
transient cores
[junit4] 2> 75121 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 75121 INFO (zkCallback-100-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 75121 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 76031 INFO (qtp30155223-655) [ ] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 8.0.0
[junit4] 2> 76040 INFO (qtp30155223-655) [ ] o.a.s.s.IndexSchema
[collection1_shard2_replica_n45] Schema name=test
[junit4] 2> 76087 INFO (qtp30155223-655) [ ] o.a.s.s.IndexSchema
Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 76097 INFO (qtp30155223-655) [ ] o.a.s.c.CoreContainer
Creating SolrCore 'collection1_shard2_replica_n45' using configuration from
collection collection1, trusted=true
[junit4] 2> 76097 INFO (qtp30155223-655) [ ] o.a.s.m.r.SolrJmxReporter
JMX monitoring for 'solr.core.collection1.shard2.replica_n45' (registry
'solr.core.collection1.shard2.replica_n45') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 76098 INFO (qtp30155223-655) [ ] o.a.s.c.SolrCore
solr.RecoveryStrategy.Builder
[junit4] 2> 76098 INFO (qtp30155223-655) [ ] o.a.s.c.SolrCore
[[collection1_shard2_replica_n45] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-3-001/cores/collection1_shard2_replica_n45],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-3-001/cores/collection1_shard2_replica_n45/data/]
[junit4] 2> 76100 INFO (qtp30155223-655) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=44,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.0]
[junit4] 2> 76120 WARN (qtp30155223-655) [ ] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,attributes = {initParams=a, name=/dump,
class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 76168 INFO (qtp30155223-655) [ ] o.a.s.u.UpdateHandler
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 76168 INFO (qtp30155223-655) [ ] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 76169 INFO (qtp30155223-655) [ ] o.a.s.u.CommitTracker
Hard AutoCommit: disabled
[junit4] 2> 76170 INFO (qtp30155223-655) [ ] o.a.s.u.CommitTracker
Soft AutoCommit: disabled
[junit4] 2> 76171 INFO (qtp30155223-655) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=36,
maxMergedSegmentMB=33.3642578125, floorSegmentMB=0.4033203125,
forceMergeDeletesPctAllowed=3.6241769498282306, segmentsPerTier=43.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 76172 INFO (qtp30155223-655) [ ] o.a.s.s.SolrIndexSearcher
Opening [Searcher@cd2b02[collection1_shard2_replica_n45] main]
[junit4] 2> 76173 INFO (qtp30155223-655) [ ]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 76173 INFO (qtp30155223-655) [ ]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 76174 INFO (qtp30155223-655) [ ]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 76174 INFO (qtp30155223-655) [ ] o.a.s.u.UpdateLog Could
not find max version in index or recent updates, using new clock
1579236946337595392
[junit4] 2> 76178 INFO (qtp30155223-655) [ ] o.a.s.c.ZkController Core
needs to recover:collection1_shard2_replica_n45
[junit4] 2> 76180 INFO (searcherExecutor-293-thread-1) [ ]
o.a.s.c.SolrCore [collection1_shard2_replica_n45] Registered new searcher
Searcher@cd2b02[collection1_shard2_replica_n45]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 76181 INFO (updateExecutor-97-thread-1) [ ]
o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 76181 INFO (qtp30155223-655) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT}
status=0 QTime=1162
[junit4] 2> 76183 INFO (qtp14852282-627) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/collections
params={node=127.0.0.1:42993_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2}
status=0 QTime=1167
[junit4] 2> 76185 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 76190 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 76191 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy Begin buffering updates.
core=[collection1_shard2_replica_n45]
[junit4] 2> 76191 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE,
tlog=null}
[junit4] 2> 76191 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy Publishing state of core
[collection1_shard2_replica_n45] as recovering, leader is
[http://127.0.0.1:41787/collection1_shard2_replica_n41/] and I am
[http://127.0.0.1:42993/collection1_shard2_replica_n45/]
[junit4] 2> 76213 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy Sending prep recovery command to
[http://127.0.0.1:41787]; [WaitForState:
action=PREPRECOVERY&core=collection1_shard2_replica_n41&nodeName=127.0.0.1:42993_&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 76214 INFO (qtp21558926-593) [ ] o.a.s.h.a.PrepRecoveryOp
Going to wait for coreNodeName: core_node46, state: recovering, checkLive:
true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 76214 INFO (qtp21558926-593) [ ] o.a.s.h.a.PrepRecoveryOp
In WaitForState(recovering): collection=collection1, shard=shard2,
thisCore=collection1_shard2_replica_n41, leaderDoesNotNeedRecovery=false,
isLeader? true, live=true, checkLive=true, currentState=down,
localState=active, nodeName=127.0.0.1:42993_, coreNodeName=core_node46,
onlyIfActiveCheckResult=false, nodeProps:
core_node46:{"core":"collection1_shard2_replica_n45","base_url":"http://127.0.0.1:42993","node_name":"127.0.0.1:42993_","state":"down","type":"NRT"}
[junit4] 2> 76295 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 76295 INFO (zkCallback-100-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 76295 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [4])
[junit4] 2> 76327 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-4-001
of type NRT
[junit4] 2> 76328 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 76330 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@51721c{/,null,AVAILABLE}
[junit4] 2> 76331 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.AbstractConnector Started
ServerConnector@1ab7a51{HTTP/1.1,[http/1.1]}{127.0.0.1:37095}
[junit4] 2> 76331 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.Server Started @78373ms
[junit4] 2> 76331 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/,
solrconfig=solrconfig.xml,
solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/tempDir-001/jetty4,
hostPort=37095,
coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-4-001/cores}
[junit4] 2> 76331 ERROR
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be
missing or incomplete.
[junit4] 2> 76331 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version
8.0.0
[junit4] 2> 76332 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 76332 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config
dir: null
[junit4] 2> 76332 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
2017-09-22T10:55:15.400124Z
[junit4] 2> 76348 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 76348 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-4-001/solr.xml
[junit4] 2> 76351 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.SolrXmlConfig MBean server found:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac, but no JMX reporters were
configured - adding default JMX reporter.
[junit4] 2> 76354 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37935/solr
[junit4] 2> 76389 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 76390 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 76392 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkController Register node as live in
ZooKeeper:/live_nodes/127.0.0.1:37095_
[junit4] 2> 76393 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 76394 INFO (zkCallback-100-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 76394 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 76395 INFO (zkCallback-76-thread-2) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 76394 INFO (zkCallback-107-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 76400 INFO (zkCallback-83-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 76455 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 76464 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm')
enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 76465 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry
'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 76466 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-4-001/cores
[junit4] 2> 76516 INFO (qtp14852282-621) [ ]
o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params
node=127.0.0.1:37095_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2
and sendToOCPQueue=true
[junit4] 2> 76522 INFO
(OverseerCollectionConfigSetProcessor-98702308642914308-127.0.0.1:45853_-n_0000000000)
[ ] o.a.s.c.OverseerTaskQueue Response ZK path:
/overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may
have disconnected from ZooKeeper
[junit4] 2> 76522 INFO (OverseerThreadFactory-257-thread-5) [ ]
o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:37095_ for creating new replica
[junit4] 2> 76527 INFO (qtp21222562-688) [ ]
o.a.s.h.a.CoreAdminOperation core create command
qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n47&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 76528 INFO (qtp21222562-688) [ ]
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4
transient cores
[junit4] 2> 76673 INFO (zkCallback-107-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 76673 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 76673 INFO (zkCallback-100-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 76676 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 77214 INFO (qtp21558926-593) [ ] o.a.s.h.a.PrepRecoveryOp
In WaitForState(recovering): collection=collection1, shard=shard2,
thisCore=collection1_shard2_replica_n41, leaderDoesNotNeedRecovery=false,
isLeader? true, live=true, checkLive=true, currentState=recovering,
localState=active, nodeName=127.0.0.1:42993_, coreNodeName=core_node46,
onlyIfActiveCheckResult=false, nodeProps:
core_node46:{"core":"collection1_shard2_replica_n45","base_url":"http://127.0.0.1:42993","node_name":"127.0.0.1:42993_","state":"recovering","type":"NRT"}
[junit4] 2> 77214 INFO (qtp21558926-593) [ ] o.a.s.h.a.PrepRecoveryOp
Waited coreNodeName: core_node46, state: recovering, checkLive: true,
onlyIfLeader: true for: 1 seconds.
[junit4] 2> 77214 INFO (qtp21558926-593) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/cores
params={nodeName=127.0.0.1:42993_&onlyIfLeaderActive=true&core=collection1_shard2_replica_n41&coreNodeName=core_node46&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
status=0 QTime=1001
[junit4] 2> 77559 INFO (qtp21222562-688) [ ] o.a.s.c.SolrConfig Using
Lucene MatchVersion: 8.0.0
[junit4] 2> 77570 INFO (qtp21222562-688) [ ] o.a.s.s.IndexSchema
[collection1_shard1_replica_n47] Schema name=test
[junit4] 2> 77625 INFO (qtp21222562-688) [ ] o.a.s.s.IndexSchema
Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 77636 INFO (qtp21222562-688) [ ] o.a.s.c.CoreContainer
Creating SolrCore 'collection1_shard1_replica_n47' using configuration from
collection collection1, trusted=true
[junit4] 2> 77637 INFO (qtp21222562-688) [ ] o.a.s.m.r.SolrJmxReporter
JMX monitoring for 'solr.core.collection1.shard1.replica_n47' (registry
'solr.core.collection1.shard1.replica_n47') enabled at server:
com.sun.jmx.mbeanserver.JmxMBeanServer@155cdac
[junit4] 2> 77637 INFO (qtp21222562-688) [ ] o.a.s.c.SolrCore
solr.RecoveryStrategy.Builder
[junit4] 2> 77637 INFO (qtp21222562-688) [ ] o.a.s.c.SolrCore
[[collection1_shard1_replica_n47] ] Opening new SolrCore at
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-4-001/cores/collection1_shard1_replica_n47],
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001/shard-4-001/cores/collection1_shard1_replica_n47/data/]
[junit4] 2> 77640 INFO (qtp21222562-688) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
[LogDocMergePolicy: minMergeSize=1000, mergeFactor=44,
maxMergeSize=9223372036854775807,
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false,
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.0]
[junit4] 2> 77665 WARN (qtp21222562-688) [ ] o.a.s.c.RequestHandlers
INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class
= DumpRequestHandler,attributes = {initParams=a, name=/dump,
class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 77698 INFO (qtp21222562-688) [ ] o.a.s.u.UpdateHandler
Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 77698 INFO (qtp21222562-688) [ ] o.a.s.u.UpdateLog
Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100
maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 77699 INFO (qtp21222562-688) [ ] o.a.s.u.CommitTracker
Hard AutoCommit: disabled
[junit4] 2> 77699 INFO (qtp21222562-688) [ ] o.a.s.u.CommitTracker
Soft AutoCommit: disabled
[junit4] 2> 77700 INFO (qtp21222562-688) [ ] o.a.s.u.RandomMergePolicy
RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
[TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=36,
maxMergedSegmentMB=33.3642578125, floorSegmentMB=0.4033203125,
forceMergeDeletesPctAllowed=3.6241769498282306, segmentsPerTier=43.0,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 77700 INFO (qtp21222562-688) [ ] o.a.s.s.SolrIndexSearcher
Opening [Searcher@ea4672[collection1_shard1_replica_n47] main]
[junit4] 2> 77701 INFO (qtp21222562-688) [ ]
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase:
/configs/conf1
[junit4] 2> 77701 INFO (qtp21222562-688) [ ]
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 77702 INFO (qtp21222562-688) [ ]
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 77703 INFO (searcherExecutor-304-thread-1) [ ]
o.a.s.c.SolrCore [collection1_shard1_replica_n47] Registered new searcher
Searcher@ea4672[collection1_shard1_replica_n47]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 77703 INFO (qtp21222562-688) [ ] o.a.s.u.UpdateLog Could
not find max version in index or recent updates, using new clock
1579236947940868096
[junit4] 2> 77705 INFO (qtp21222562-688) [ ] o.a.s.c.ZkController Core
needs to recover:collection1_shard1_replica_n47
[junit4] 2> 77706 INFO (qtp21222562-688) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/cores
params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n47&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT}
status=0 QTime=1178
[junit4] 2> 77710 INFO (qtp14852282-621) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/collections
params={node=127.0.0.1:37095_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2}
status=0 QTime=1194
[junit4] 2> 77715 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testSplitWithChaosMonkey
[junit4] 2> 77715 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 15
for each attempt
[junit4] 2> 77715 INFO
(TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection:
collection1 failOnTimeout:true timeout (sec):15
[junit4] 2> 77705 INFO (updateExecutor-104-thread-1) [ ]
o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 77720 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy Attempting to PeerSync from
[http://127.0.0.1:41787/collection1_shard2_replica_n41/] -
recoveringAfterStartup=[true]
[junit4] 2> 77721 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n45
url=http://127.0.0.1:42993 START
replicas=[http://127.0.0.1:41787/collection1_shard2_replica_n41/] nUpdates=100
[junit4] 2> 77721 INFO (recoveryExecutor-105-thread-1) [ ]
o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 77721 INFO (recoveryExecutor-105-thread-1) [ ]
o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 77721 INFO (recoveryExecutor-105-thread-1) [ ]
o.a.s.c.RecoveryStrategy Begin buffering updates.
core=[collection1_shard1_replica_n47]
[junit4] 2> 77721 INFO (recoveryExecutor-105-thread-1) [ ]
o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE,
tlog=null}
[junit4] 2> 77722 INFO (recoveryExecutor-105-thread-1) [ ]
o.a.s.c.RecoveryStrategy Publishing state of core
[collection1_shard1_replica_n47] as recovering, leader is
[http://127.0.0.1:43017/collection1_shard1_replica_n43/] and I am
[http://127.0.0.1:37095/collection1_shard1_replica_n47/]
[junit4] 2> 77730 INFO (recoveryExecutor-105-thread-1) [ ]
o.a.s.c.RecoveryStrategy Sending prep recovery command to
[http://127.0.0.1:43017]; [WaitForState:
action=PREPRECOVERY&core=collection1_shard1_replica_n43&nodeName=127.0.0.1:37095_&coreNodeName=core_node48&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 77731 INFO (qtp14852282-623) [ ] o.a.s.h.a.PrepRecoveryOp
Going to wait for coreNodeName: core_node48, state: recovering, checkLive:
true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 77735 INFO (qtp21558926-594) [ ] o.a.s.u.IndexFingerprint
IndexFingerprint millis:4.0 result:{maxVersionSpecified=9223372036854775807,
maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0,
maxDoc=0}
[junit4] 2> 77735 INFO (qtp21558926-594) [ ] o.a.s.c.S.Request
[collection1_shard2_replica_n41] webapp= path=/get
params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2}
status=0 QTime=5
[junit4] 2> 77736 INFO (qtp14852282-623) [ ] o.a.s.h.a.PrepRecoveryOp
In WaitForState(recovering): collection=collection1, shard=shard1,
thisCore=collection1_shard1_replica_n43, leaderDoesNotNeedRecovery=false,
isLeader? true, live=true, checkLive=true, currentState=down,
localState=active, nodeName=127.0.0.1:37095_, coreNodeName=core_node48,
onlyIfActiveCheckResult=false, nodeProps:
core_node48:{"core":"collection1_shard1_replica_n47","base_url":"http://127.0.0.1:37095","node_name":"127.0.0.1:37095_","state":"down","type":"NRT"}
[junit4] 2> 77749 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0
result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0,
maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.u.DirectUpdateHandler2 start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 77750 INFO (recoveryExecutor-98-thread-1) [ ]
o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 77852 INFO (zkCallback-107-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 77852 INFO (zkCallback-88-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 77852 INFO (zkCallback-94-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [collection1] has
occurred - updating... (live nodes size: [5])
[junit4] 2> 77852 INFO (zkCallback-100-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent
state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json] for collection [col
[...truncated too long message...]
it4] 2> 810449 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ChaosMonkey monkey: stop jetty! 39813
[junit4] 2> 810449 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.CoreContainer Shutting down CoreContainer instance=24109313
[junit4] 2> 810449 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node,
tag=null
[junit4] 2> 810449 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@13eb675: rootName = null,
domain = solr.node, service url = null, agent id = null] for registry solr.node
/ com.codahale.metrics.MetricRegistry@1d1b78d
[junit4] 2> 810454 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm,
tag=null
[junit4] 2> 810454 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@11d62cf: rootName = null,
domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm /
com.codahale.metrics.MetricRegistry@49cda0
[junit4] 2> 810457 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty,
tag=null
[junit4] 2> 810457 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@58e8e5: rootName = null,
domain = solr.jetty, service url = null, agent id = null] for registry
solr.jetty / com.codahale.metrics.MetricRegistry@ecb43f
[junit4] 2> 810458 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster,
tag=null
[junit4] 2> 810460 INFO (coreCloseExecutor-943-thread-1) [ ]
o.a.s.c.SolrCore [collection1_shard1_replica_t47] CLOSING SolrCore
org.apache.solr.core.SolrCore@1f67cdf
[junit4] 2> 810460 INFO (coreCloseExecutor-943-thread-1) [ ]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.collection1.shard1.replica_t47, tag=32931039
[junit4] 2> 810460 INFO (coreCloseExecutor-943-thread-1) [ ]
o.a.s.m.r.SolrJmxReporter Closing reporter
[org.apache.solr.metrics.reporters.SolrJmxReporter@158bfc3: rootName = null,
domain = solr.core.collection1.shard1.replica_t47, service url = null, agent id
= null] for registry solr.core.collection1.shard1.replica_t47 /
com.codahale.metrics.MetricRegistry@1ff8aea
[junit4] 2> 810470 INFO (coreCloseExecutor-943-thread-1) [ ]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.collection.collection1.shard1.leader, tag=32931039
[junit4] 2> 810473 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.Overseer Overseer (id=98702355281281042-127.0.0.1:39813_-n_0000000004)
closing
[junit4] 2> 810473 INFO
(OverseerStateUpdate-98702355281281042-127.0.0.1:39813_-n_0000000004) [ ]
o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39813_
[junit4] 2> 810478 INFO (zkCallback-280-thread-1) [ ]
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 810497 WARN (indexFetcher-896-thread-1) [ ]
o.a.s.h.IndexFetcher Could not log failed replication details
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: Already closed
[junit4] 2> at
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)
[junit4] 2> at
org.apache.solr.handler.ReplicationHandler.loadReplicationProperties(ReplicationHandler.java:1133)
[junit4] 2> at
org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:769)
[junit4] 2> at
org.apache.solr.handler.IndexFetcher.cleanup(IndexFetcher.java:677)
[junit4] 2> at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:659)
[junit4] 2> at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
[junit4] 2> at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421)
[junit4] 2> at
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1185)
[junit4] 2> at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
[junit4] 2> at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
[junit4] 2> at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)
[junit4] 2> at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
[junit4] 2> at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 810497 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.AbstractConnector Stopped
ServerConnector@274611{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 810498 ERROR (indexFetcher-896-thread-1) [ ]
o.a.s.h.ReplicationHandler Index fetch failed
:org.apache.solr.common.SolrException: No registered leader was found after
waiting for 4000ms , collection: collection1 slice: shard1
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:758)
[junit4] 2> at
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:744)
[junit4] 2> at
org.apache.solr.handler.IndexFetcher.getLeaderReplica(IndexFetcher.java:667)
[junit4] 2> at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
[junit4] 2> at
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
[junit4] 2> at
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:421)
[junit4] 2> at
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1185)
[junit4] 2> at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
[junit4] 2> at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
[junit4] 2> at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)
[junit4] 2> at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
[junit4] 2> at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2>
[junit4] 2> 810498 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.e.j.s.h.ContextHandler Stopped
o.e.j.s.ServletContextHandler@1426cce{/,null,UNAVAILABLE}
[junit4] 2> 810499 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ChaosMonkey monkey: stop jetty! 32931
[junit4] 2> 810499 ERROR
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper
server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 810500 INFO
(TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[D8504D6784902EB3]) [ ]
o.a.s.c.ZkTestServer connecting to 127.0.0.1:39905 39905
[junit4] 2> 815553 INFO (Thread-406) [ ] o.a.s.c.ZkTestServer
connecting to 127.0.0.1:39905 39905
[junit4] 2> 815554 WARN (Thread-406) [ ] o.a.s.c.ZkTestServer Watch
limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/collections/collection1/state.json
[junit4] 2> 7 /solr/aliases.json
[junit4] 2> 5 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/clusterstate.json
[junit4] 2> 7 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/live_nodes
[junit4] 2> 7 /solr/collections
[junit4] 2> 5 /solr/overseer/queue
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_D8504D6784902EB3-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
{rnd_b=BlockTreeOrds(blocksize=128), shard_s=FST50, a_t=FST50,
id=BlockTreeOrds(blocksize=128)},
docValues:{rnd_b=DocValuesFormat(name=Memory),
shard_s=DocValuesFormat(name=Asserting),
_version_=DocValuesFormat(name=Lucene70), n_ti=DocValuesFormat(name=Asserting),
a_t=DocValuesFormat(name=Asserting), a_i=DocValuesFormat(name=Memory),
id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1792,
maxMBSortInHeap=5.701114183996373, sim=RandomSimilarity(queryNorm=false): {},
locale=qu, timezone=Europe/Vaduz
[junit4] 2> NOTE: Linux 4.10.0-33-generic i386/Oracle Corporation 9
(32-bit)/cpus=8,threads=1,free=125178048,total=282066944
[junit4] 2> NOTE: All tests run in this JVM: [TestRebalanceLeaders,
TestRestManager, TestDistributedMissingSort, ShardSplitTest]
[junit4] Completed [145/733 (1!)] on J2 in 747.08s, 5 tests, 1 failure <<<
FAILURES!
[...truncated 41244 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
