Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/813/
3 tests failed.
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
2 threads leaked from SUITE scope at
org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 1)
Thread[id=4961, name=StoppableIndexingThread, state=TIMED_WAITING,
group=TGRP-HdfsChaosMonkeySafeLeaderTest] at
java.lang.Thread.sleep(Native Method) at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
2) Thread[id=4962, name=StoppableIndexingThread, state=TIMED_WAITING,
group=TGRP-HdfsChaosMonkeySafeLeaderTest] at
java.lang.Thread.sleep(Native Method) at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE
scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
1) Thread[id=4961, name=StoppableIndexingThread, state=TIMED_WAITING,
group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
2) Thread[id=4962, name=StoppableIndexingThread, state=TIMED_WAITING,
group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
at __randomizedtesting.SeedInfo.seed([AD2575BC4A303AF3]:0)
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
There are still zombie threads that couldn't be terminated: 1)
Thread[id=4961, name=StoppableIndexingThread, state=RUNNABLE,
group=TGRP-HdfsChaosMonkeySafeLeaderTest] at
java.lang.Thread.sleep(Native Method) at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
2) Thread[id=4962, name=StoppableIndexingThread, state=RUNNABLE,
group=TGRP-HdfsChaosMonkeySafeLeaderTest] at
java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304) at
java.lang.StringCoding.encode(StringCoding.java:344) at
java.lang.String.getBytes(String.java:906) at
org.apache.solr.common.util.ContentStreamBase$StringStream.getStream(ContentStreamBase.java:182)
at
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getStream(RequestWriter.java:125)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:439)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
at
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
1) Thread[id=4961, name=StoppableIndexingThread, state=RUNNABLE,
group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
2) Thread[id=4962, name=StoppableIndexingThread, state=RUNNABLE,
group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304)
at java.lang.StringCoding.encode(StringCoding.java:344)
at java.lang.String.getBytes(String.java:906)
at
org.apache.solr.common.util.ContentStreamBase$StringStream.getStream(ContentStreamBase.java:182)
at
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getStream(RequestWriter.java:125)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:439)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
at
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
at __randomizedtesting.SeedInfo.seed([AD2575BC4A303AF3]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 30 seconds and no jetties were stopped - this is worth
investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 30 seconds and no jetties
were stopped - this is worth investigating!
at
__randomizedtesting.SeedInfo.seed([AD2575BC4A303AF3:25714A66E4CC570B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:539)
at
org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:143)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10113 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/init-core-data-001
[junit4] 2> 1171020 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
[junit4] 2> 1171020 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property:
/px_wve/w
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1171106 WARN
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1171125 WARN
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1171127 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log jetty-6.1.26
[junit4] 2> 1171140 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log Extract
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
to ./temp/Jetty_localhost_53176_hdfs____rdn8d8/webapp
[junit4] 2> 1171290 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4] 2> 1171618 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log Started
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53176
[junit4] 2> 1171884 WARN
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1171888 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log jetty-6.1.26
[junit4] 2> 1171912 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log Extract
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
to ./temp/Jetty_localhost_55363_datanode____3dlhtj/webapp
[junit4] 2> 1172059 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4] 2> 1172315 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log Started
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55363
[junit4] 2> 1172443 WARN
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1172445 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log jetty-6.1.26
[junit4] 2> 1172476 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log Extract
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
to ./temp/Jetty_localhost_45382_datanode____.uvfqh9/webapp
[junit4] 2> 1172600 INFO (IPC Server handler 3 on 48386) [ ]
BlockStateChange BLOCK* processReport: from storage
DS-ada6e585-74ba-4660-9d87-0d874a8402fa node DatanodeRegistration(127.0.0.1,
datanodeUuid=7f5ff41c-ec8f-4f88-8a58-2c231134fdee, infoPort=55363,
ipcPort=39397, storageInfo=lv=-56;cid=testClusterID;nsid=847250699;c=0),
blocks: 0, hasStaleStorages: true, processing time: 1 msecs
[junit4] 2> 1172600 INFO (IPC Server handler 3 on 48386) [ ]
BlockStateChange BLOCK* processReport: from storage
DS-6e593737-9f25-451c-9266-0c669f0a7e49 node DatanodeRegistration(127.0.0.1,
datanodeUuid=7f5ff41c-ec8f-4f88-8a58-2c231134fdee, infoPort=55363,
ipcPort=39397, storageInfo=lv=-56;cid=testClusterID;nsid=847250699;c=0),
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
[junit4] 2> 1172676 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4] 2> 1172939 INFO
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[AD2575BC4A303AF3]-worker) [ ]
o.m.log Started
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45382
[junit4] 2> 1173131 INFO (IPC Server handler 6 on 48386) [ ]
BlockStateChange BLOCK* processReport: from storage
DS-d384dccd-bf16-442d-baf5-abb8a1ba08df node DatanodeRegistration(127.0.0.1,
datanodeUuid=778cb6a1-cafa-4ca7-b43b-51837814cb80, infoPort=45382,
ipcPort=60930, storageInfo=lv=-56;cid=testClusterID;nsid=847250699;c=0),
blocks: 0, hasStaleStorages: true, processing time: 4 msecs
[junit4] 2> 1173132 INFO (IPC Server handler 6 on 48386) [ ]
BlockStateChange BLOCK* processReport: from storage
DS-9847c19f-d760-49ac-b41e-dc2210b3dea6 node DatanodeRegistration(127.0.0.1,
datanodeUuid=778cb6a1-cafa-4ca7-b43b-51837814cb80, infoPort=45382,
ipcPort=60930, storageInfo=lv=-56;cid=testClusterID;nsid=847250699;c=0),
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
[junit4] 2> 1173319 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1173319 INFO (Thread-2305) [ ] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1173319 INFO (Thread-2305) [ ] o.a.s.c.ZkTestServer
Starting server
[junit4] 2> 1173419 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.ZkTestServer start zk server on port:53221
[junit4] 2> 1173419 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1173420 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1173423 INFO (zkCallback-423-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@33123c40
name:ZooKeeperConnection Watcher:127.0.0.1:53221 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1173423 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1173424 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1173424 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /solr
[junit4] 2> 1173427 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1173427 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1173429 INFO (zkCallback-424-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@168d43c name:ZooKeeperConnection
Watcher:127.0.0.1:53221/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1173429 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1173429 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1173429 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
[junit4] 2> 1173432 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
[junit4] 2> 1173434 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
[junit4] 2> 1173435 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
[junit4] 2> 1173436 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
to /configs/conf1/solrconfig.xml
[junit4] 2> 1173436 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1173438 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml
to /configs/conf1/schema.xml
[junit4] 2> 1173438 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
[junit4] 2> 1173439 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1173439 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath:
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1173441 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
to /configs/conf1/stopwords.txt
[junit4] 2> 1173441 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1173442 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
to /configs/conf1/protwords.txt
[junit4] 2> 1173442 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
[junit4] 2> 1173443 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
to /configs/conf1/currency.xml
[junit4] 2> 1173444 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
[junit4] 2> 1173445 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
to /configs/conf1/enumsConfig.xml
[junit4] 2> 1173445 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1173446 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1173447 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1173448 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1173448 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1173449 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
to /configs/conf1/old_synonyms.txt
[junit4] 2> 1173449 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1173450 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractZkTestCase put
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
to /configs/conf1/synonyms.txt
[junit4] 2> 1173451 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1173900 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.SolrTestCaseJ4 Writing core.properties file to
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores/collection1
[junit4] 2> 1173905 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.Server jetty-9.2.13.v20150730
[junit4] 2> 1173908 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@672a5766{/px_wve/w,null,AVAILABLE}
[junit4] 2> 1173908 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.ServerConnector Started
ServerConnector@913e961{HTTP/1.1}{127.0.0.1:43720}
[junit4] 2> 1173908 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.Server Started @1178352ms
[junit4] 2> 1173909 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties:
{solr.data.dir=hdfs://localhost:48386/hdfs__localhost_48386__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001_tempDir-002_control_data,
hostContext=/px_wve/w, hostPort=43720,
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores}
[junit4] 2> 1173909 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
sun.misc.Launcher$AppClassLoader@4e0e2f2a
[junit4] 2> 1173911 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/'
[junit4] 2> 1173926 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1173927 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1173928 INFO (zkCallback-425-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2c1a1aed
name:ZooKeeperConnection Watcher:127.0.0.1:53221/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1173929 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1173929 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1173930 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 1173930 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/solr.xml
[junit4] 2> 1173935 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoresLocator Config-defined core root directory:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores
[junit4] 2> 1173935 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoreContainer New CoreContainer 1225832478
[junit4] 2> 1173935 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoreContainer Loading cores into CoreContainer
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/]
[junit4] 2> 1173935 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoreContainer loading shared library:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/lib
[junit4] 2> 1173935 WARN
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to
classloader: lib (resolved as:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/lib).
[junit4] 2> 1173943 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections :
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime :
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 1173944 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1173945 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1173945 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.l.LogWatcher Registering Log Listener [Log4j
(org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1173945 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53221/solr
[junit4] 2> 1173945 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 1173945 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1173956 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1173958 INFO (zkCallback-427-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@15fd9401
name:ZooKeeperConnection Watcher:127.0.0.1:53221 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1173958 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1173958 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1173969 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ConnectionManager Waiting for
client to connect to ZooKeeper
[junit4] 2> 1173973 INFO
(zkCallback-428-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@2aafa8d8
name:ZooKeeperConnection Watcher:127.0.0.1:53221/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1173973 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ConnectionManager Client is
connected to ZooKeeper
[junit4] 2> 1173974 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer/queue
[junit4] 2> 1173977 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-queue-work
[junit4] 2> 1173981 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-running
[junit4] 2> 1173983 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-completed
[junit4] 2> 1173986 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer/collection-map-failure
[junit4] 2> 1173991 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 1173994 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/aliases.json
[junit4] 2> 1173996 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/clusterstate.json
[junit4] 2> 1173998 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/security.json
[junit4] 2> 1173999 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.ZkController Register node as live
in ZooKeeper:/live_nodes/127.0.0.1:43720_px_wve%2Fw
[junit4] 2> 1174000 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/live_nodes/127.0.0.1:43720_px_wve%2Fw
[junit4] 2> 1174003 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect
[junit4] 2> 1174004 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/election
[junit4] 2> 1174006 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1174007 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.LeaderElector Joined leadership
election with path:
/overseer_elect/election/94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000
[junit4] 2> 1174007 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.OverseerElectionContext I am going
to be the leader 127.0.0.1:43720_px_wve%2Fw
[junit4] 2> 1174007 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer_elect/leader
[junit4] 2> 1174009 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.Overseer Overseer
(id=94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000) starting
[junit4] 2> 1174014 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/overseer/queue-work
[junit4] 2> 1174023 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.OverseerAutoReplicaFailoverThread
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1174024 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.Overseer Starting to work on the
main queue
[junit4] 2> 1174025 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader Updating cluster
state from ZooKeeper...
[junit4] 2> 1174032 INFO
(OverseerCollectionConfigSetProcessor-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.OverseerTaskProcessor Process
current queue of overseer operations
[junit4] 2> 1174038 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader Updating cluster
state from ZooKeeper...
[junit4] 2> 1174076 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.CoreContainer Security conf doesn't
exist. Skipping setup for authorization module.
[junit4] 2> 1174076 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.CoreContainer No authentication
plugin used.
[junit4] 2> 1174077 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.CoresLocator Looking for core
definitions underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores
[junit4] 2> 1174078 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.SolrCore Created CoreDescriptor:
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml,
loadOnStartup=true, configSetProperties=configsetprops.json,
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores/collection1,
collection=control_collection,
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores/collection1/,
coreNodeName=, dataDir=data/, shard=}
[junit4] 2> 1174078 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.CoresLocator Found core collection1
in
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores/collection1/
[junit4] 2> 1174078 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.CoresLocator Found 1 core definitions
[junit4] 2> 1174079 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.ZkController publishing state=down
[junit4] 2> 1174079 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.ZkController numShards not found on descriptor - reading it from system
property
[junit4] 2> 1174086 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.s.SolrDispatchFilter
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 1174086 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.s.SolrDispatchFilter
SolrDispatchFilter.init() done
[junit4] 2> 1174086 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.Overseer processMessage: queueSize:
1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43720/px_wve/w";,
[junit4] 2> "node_name":"127.0.0.1:43720_px_wve%2Fw",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1174086 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.ZkController look for our core node name
[junit4] 2> 1174087 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.o.ReplicaMutator Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43720/px_wve/w";,
[junit4] 2> "node_name":"127.0.0.1:43720_px_wve%2Fw",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1174088 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.o.ClusterStateMutator building a new
cName: control_collection
[junit4] 2> 1174088 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.o.ReplicaMutator Assigning new node
to shard shard=shard1
[junit4] 2> 1174090 INFO
(zkCallback-428-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1174090 INFO
(zkCallback-428-thread-2-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1175087 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
[junit4] 2> 1175088 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.ZkController Check for collection zkNode:control_collection
[junit4] 2> 1175088 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.ZkController Collection zkNode exists
[junit4] 2> 1175088 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.c.ZkStateReader Load collection config
from:/collections/control_collection
[junit4] 2> 1175089 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1
specified config exists in ZooKeeper
[junit4] 2> 1175089 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores/collection1/'
[junit4] 2> 1175103 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.Config loaded config solrconfig.xml with version 0
[junit4] 2> 1175109 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.SolrConfig current version of requestparams : -1
[junit4] 2> 1175128 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
[junit4] 2> 1175138 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1175139 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1175147 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1175237 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.s.IndexSchema default search field in schema is text
[junit4] 2> 1175238 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.s.IndexSchema unique key field: id
[junit4] 2> 1175239 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
[junit4] 2> 1175241 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
[junit4] 2> 1175260 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default
properties: Can't find resource 'configsetprops.json' in classpath or
'/configs/conf1',
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
[junit4] 2> 1175260 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection x:collection1]
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from
collection control_collection
[junit4] 2> 1175260 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
[junit4] 2> 1175261 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.HdfsDirectoryFactory
solr.hdfs.home=hdfs://localhost:48386/solr_hdfs_home
[junit4] 2> 1175261 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication
disabled
[junit4] 2> 1175261 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/control-001/cores/collection1/],
dataDir=[null]
[junit4] 2> 1175261 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36753f55
[junit4] 2> 1175262 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path
hdfs://localhost:48386/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 1175279 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for
hdfs://localhost:48386/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 1175281 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore New index directory detected: old=null
new=hdfs://localhost:48386/solr_hdfs_home/control_collection/core_node1/data/index/
[junit4] 2> 1175292 WARN
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory
'hdfs:/localhost:48386/solr_hdfs_home/control_collection/core_node1/data/index'
doesn't exist. Creating new index...
[junit4] 2> 1175292 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path
hdfs://localhost:48386/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 1175300 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1]
with direct memory allocation set to [true]
[junit4] 2> 1175300 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage,
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1175300 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS
BlockCache
[junit4] 2> 1175310 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1175310 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for
hdfs://localhost:48386/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 1175311 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
org.apache.lucene.index.MockRandomMergePolicy:
org.apache.lucene.index.MockRandomMergePolicy@4719f21b
[junit4] 2> 1175362 INFO (IPC Server handler 3 on 48386) [ ]
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40143 is
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1,
replicas=[ReplicaUnderConstruction[[DISK]DS-9847c19f-d760-49ac-b41e-dc2210b3dea6:NORMAL:127.0.0.1:51921|RBW],
ReplicaUnderConstruction[[DISK]DS-6e593737-9f25-451c-9266-0c669f0a7e49:NORMAL:127.0.0.1:40143|FINALIZED]]}
size 0
[junit4] 2> 1175366 INFO (IPC Server handler 2 on 48386) [ ]
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51921 is
added to blk_1073741825_1001 size 71
[junit4] 2> 1175368 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7c848b0
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5fb617cb);
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
[junit4] 2> 1175369 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 1175370 INFO
(OldIndexDirectoryCleanupThreadForCore-collection1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup
for core collection1 in
hdfs://localhost:48386/solr_hdfs_home/control_collection/core_node1/data/
[junit4] 2> 1175374 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 1175374 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 1175374 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1175374 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 1175374 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1175374 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1175375 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1175375 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 1175375 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as
default, creating implicit default
[junit4] 2> 1175377 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 1175378 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 1175379 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 1175380 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
[junit4] 2> 1175384 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.RequestHandlers Registered paths:
/admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
[junit4] 2> 1175385 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore Using default statsCache cache:
org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 1175385 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation:
org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1175385 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir=
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
tlogDfsReplication=2
[junit4] 2> 1175396 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
[junit4] 2> 1175396 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
[junit4] 2> 1175398 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=15, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=16.20703125,
floorSegmentMB=1.81640625, forceMergeDeletesPctAllowed=10.339510392680666,
segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1175409 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7c848b0
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5fb617cb);
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
[junit4] 2> 1175409 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
[junit4] 2> 1175409 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@32548fca[collection1]
main
[junit4] 2> 1175410 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.c.ZkStateReader Load collection config
from:/collections/control_collection
[junit4] 2> 1175411 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection
configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1175411 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based
storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1175411 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO
with znodeBase: /configs/conf1
[junit4] 2> 1175411 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
[junit4] 2> 1175411 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1175412 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode
/configs/conf1/_rest_managed.json
[junit4] 2> 1175412 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1175412 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
[junit4] 2> 1175412 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1175413 INFO
(searcherExecutor-1594-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher
Searcher@32548fca[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1175414 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed
version buckets
[junit4] 2> 1175414 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for
256 version buckets from index
[junit4] 2> 1175414 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed
version bucket highest value from index
[junit4] 2> 1175414 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent
updates, using new clock 1514188907879596032
[junit4] 2> 1175414 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with
highest version 1514188907879596032
[junit4] 2> 1175414 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
[junit4] 2> 1175415 INFO
(coreLoadExecutor-1593-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.CoreContainer registering core: collection1
[junit4] 2> 1175415 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController Register replica - core:collection1
address:http://127.0.0.1:43720/px_wve/w collection:control_collection
shard:shard1
[junit4] 2> 1175416 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.c.SolrZkClient makePath:
/collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1175419 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.LeaderElector Joined leadership election with path:
/collections/control_collection/leader_elect/shard1/election/94636806605701124-core_node1-n_0000000000
[junit4] 2> 1175419 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process
for shard shard1
[junit4] 2> 1175421 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to
continue.
[junit4] 2> 1175421 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try
and sync
[junit4] 2> 1175421 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.Overseer processMessage: queueSize:
1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> 1175421 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SyncStrategy Sync replicas to
http://127.0.0.1:43720/px_wve/w/collection1/
[junit4] 2> 1175421 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1175422 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.SyncStrategy
http://127.0.0.1:43720/px_wve/w/collection1/ has no replicas
[junit4] 2> 1175422 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader:
http://127.0.0.1:43720/px_wve/w/collection1/ shard1
[junit4] 2> 1175423 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.c.SolrZkClient makePath:
/collections/control_collection/leaders/shard1
[junit4] 2> 1175426 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader
registration node
[junit4] 2> 1175428 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.Overseer processMessage: queueSize:
1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:43720/px_wve/w";,
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 1175532 INFO
(zkCallback-428-thread-2-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1175532 INFO
(zkCallback-428-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1175579 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController We are
http://127.0.0.1:43720/px_wve/w/collection1/ and leader is
http://127.0.0.1:43720/px_wve/w/collection1/
[junit4] 2> 1175579 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1
baseURL=http://127.0.0.1:43720/px_wve/w
[junit4] 2> 1175579 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1175579 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController publishing state=active
[junit4] 2> 1175579 INFO
(coreZkRegister-1587-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw
x:collection1 s:shard1 c:control_collection r:core_node1)
[n:127.0.0.1:43720_px_wve%2Fw c:control_collection s:shard1 r:core_node1
x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading
it from system property
[junit4] 2> 1175582 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.Overseer processMessage: queueSize:
1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43720/px_wve/w";,
[junit4] 2> "node_name":"127.0.0.1:43720_px_wve%2Fw",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 1175583 INFO
(OverseerStateUpdate-94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.o.ReplicaMutator Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43720/px_wve/w";,
[junit4] 2> "node_name":"127.0.0.1:43720_px_wve%2Fw",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1175588 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1175597 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1175611 INFO (zkCallback-430-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@73f16e56
name:ZooKeeperConnection Watcher:127.0.0.1:53221/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1175611 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1175612 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1175612 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 1175614 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection
loss:false
[junit4] 2> 1175686 INFO
(zkCallback-428-thread-2-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1175686 INFO
(zkCallback-428-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1175686 INFO (zkCallback-430-thread-1) [ ]
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred
- updating... (live nodes size: 1)
[junit4] 2> 1175953 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.SolrTestCaseJ4 Writing core.properties file to
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/cores/collection1
[junit4] 2> 1175954 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001
[junit4] 2> 1175955 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.Server jetty-9.2.13.v20150730
[junit4] 2> 1175963 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@36cb16a7{/px_wve/w,null,AVAILABLE}
[junit4] 2> 1175964 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.ServerConnector Started
ServerConnector@22bf2a42{HTTP/1.1}{127.0.0.1:49071}
[junit4] 2> 1175964 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.e.j.s.Server Started @1180407ms
[junit4] 2> 1175964 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.s.e.JettySolrRunner Jetty properties:
{solr.data.dir=hdfs://localhost:48386/hdfs__localhost_48386__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001_tempDir-002_jetty1,
solrconfig=solrconfig.xml, hostContext=/px_wve/w, hostPort=49071,
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/cores}
[junit4] 2> 1175964 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init():
sun.misc.Launcher$AppClassLoader@4e0e2f2a
[junit4] 2> 1175964 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory:
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/'
[junit4] 2> 1175983 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1175986 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1175987 INFO (zkCallback-431-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@5569ce86
name:ZooKeeperConnection Watcher:127.0.0.1:53221/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1175987 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1175990 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1175993 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in
ZooKeeper)
[junit4] 2> 1175993 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.SolrXmlConfig Loading container configuration from
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/solr.xml
[junit4] 2> 1175999 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoresLocator Config-defined core root directory:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/cores
[junit4] 2> 1175999 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoreContainer New CoreContainer 511604473
[junit4] 2> 1175999 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoreContainer Loading cores into CoreContainer
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/]
[junit4] 2> 1175999 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.CoreContainer loading shared library:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/lib
[junit4] 2> 1175999 WARN
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to
classloader: lib (resolved as:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/lib).
[junit4] 2> 1176014 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections :
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime :
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 1176015 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1176015 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1176015 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.l.LogWatcher Registering Log Listener [Log4j
(org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1176015 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53221/solr
[junit4] 2> 1176016 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 1176016 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 1176021 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1176023 INFO (zkCallback-433-thread-1) [ ]
o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@313ccca7
name:ZooKeeperConnection Watcher:127.0.0.1:53221 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1176023 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1176024 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3]) [ ]
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 1176031 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.c.ConnectionManager Waiting for
client to connect to ZooKeeper
[junit4] 2> 1176032 INFO
(zkCallback-434-thread-1-processing-n:127.0.0.1:49071_px_wve%2Fw)
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.c.ConnectionManager Watcher
org.apache.solr.common.cloud.ConnectionManager@639fed3a
name:ZooKeeperConnection Watcher:127.0.0.1:53221/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1176032 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.c.ConnectionManager Client is
connected to ZooKeeper
[junit4] 2> 1176037 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.c.ZkStateReader Updating cluster
state from ZooKeeper...
[junit4] 2> 1177041 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.ZkController Register node as live
in ZooKeeper:/live_nodes/127.0.0.1:49071_px_wve%2Fw
[junit4] 2> 1177042 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.c.SolrZkClient makePath:
/live_nodes/127.0.0.1:49071_px_wve%2Fw
[junit4] 2> 1177043 INFO
(zkCallback-428-thread-1-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A live node change:
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has
occurred - updating... (live nodes size: 1)
[junit4] 2> 1177043 INFO
(zkCallback-428-thread-2-processing-n:127.0.0.1:43720_px_wve%2Fw)
[n:127.0.0.1:43720_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A live node change:
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has
occurred - updating... (live nodes size: 1)
[junit4] 2> 1177043 INFO
(zkCallback-434-thread-1-processing-n:127.0.0.1:49071_px_wve%2Fw)
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.c.ZkStateReader A live node change:
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has
occurred - updating... (live nodes size: 1)
[junit4] 2> 1177043 INFO (zkCallback-430-thread-1) [ ]
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live
nodes size: 1)
[junit4] 2> 1177045 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1177045 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.LeaderElector Joined leadership
election with path:
/overseer_elect/election/94636806605701128-127.0.0.1:49071_px_wve%2Fw-n_0000000001
[junit4] 2> 1177046 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.LeaderElector Watching path
/overseer_elect/election/94636806605701124-127.0.0.1:43720_px_wve%2Fw-n_0000000000
to know if I could be the leader
[junit4] 2> 1177059 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.CoreContainer Security conf doesn't
exist. Skipping setup for authorization module.
[junit4] 2> 1177059 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.CoreContainer No authentication
plugin used.
[junit4] 2> 1177060 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.CoresLocator Looking for core
definitions underneath
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_AD2575BC4A303AF3-001/shard-1-001/cores
[junit4] 2> 1177061 INFO
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[AD2575BC4A303AF3])
[n:127.0.0.1:49071_px_wve%2Fw ] o.a.s.c.SolrCore Created CoreDescriptor:
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml,
loadOnStartup=true, configSetProperties=configsetprops.json,
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenk
[...truncated too long message...]
2> at
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
[junit4] 2> at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
[junit4] 2> REQUEST FAILED for id=1-3542
[junit4] 2> java.lang.IllegalStateException: Connection pool shut down
[junit4] 2> at org.apache.http.util.Asserts.check(Asserts.java:34)
[junit4] 2> at
org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
[junit4] 2> at
org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
[junit4] 2> at
org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
[junit4] 2> at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
[junit4] 2> at
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
[junit4] 2> at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
[junit4] 2> at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
[junit4] 2> at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
[junit4] 2> at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
[junit4] 2> at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
[junit4] 2> at
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
[junit4] 2> at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
[junit4] 2> REQUEST FAILED for id=0-3502
[junit4] 2> java.lang.IllegalStateException: Connection pool shut down
[junit4] 2> at org.apache.http.util.Asserts.check(Asserts.java:34)
[junit4] 2> at
org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
[junit4] 2> at
org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
[junit4] 2> at
org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
[junit4] 2> at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
[junit4] 2> at
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
[junit4] 2> at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
[junit4] 2> at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
[junit4] 2> at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
[junit4] 2> at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
[junit4] 2> at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
[junit4] 2> at
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
[junit4] 2> at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
[junit4] 2> NOTE: test params are: codec=CheapBastard,
sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=fr_CH,
timezone=Pacific/Efate
[junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation
1.8.0_45 (64-bit)/cpus=4,threads=4,free=237976328,total=504365056
[junit4] 2> NOTE: All tests run in this JVM: [TestConfigOverlay,
DistribJoinFromCollectionTest, MergeStrategyTest, TestInitQParser,
SmileWriterTest, DistribDocExpirationUpdateProcessorTest,
TestCopyFieldCollectionResource, TestJsonFacets, LoggingHandlerTest,
SortByFunctionTest, DirectSolrSpellCheckerTest, TestTrie,
CdcrRequestHandlerTest, TestFieldTypeResource, TestIntervalFaceting,
TestCustomSort, BaseCdcrDistributedZkTest, DeleteShardTest,
TestAnalyzeInfixSuggestions, ZkCLITest, BlockDirectoryTest, TestQueryTypes,
TestSolrDeletionPolicy2, TestDistributedMissingSort, TestManagedSchema,
SolrIndexConfigTest, TestScoreJoinQPScore, UUIDUpdateProcessorFallbackTest,
DistributedFacetPivotLongTailTest, TestPartialUpdateDeduplication,
HLLSerializationTest, FastVectorHighlighterTest, DistanceUnitsTest,
ChaosMonkeyNothingIsSafeTest, HdfsNNFailoverTest, ZkStateReaderTest,
TestStandardQParsers, WordBreakSolrSpellCheckerTest, TestObjectReleaseTracker,
DistributedTermsComponentTest, DistributedQueryComponentOptimizationTest,
OpenCloseCoreStressTest, TestReqParamsAPI, HdfsChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running
'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=HdfsChaosMonkeySafeLeaderTest -Dtests.seed=AD2575BC4A303AF3
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=fr_CH -Dtests.timezone=Pacific/Efate -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | HdfsChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE
scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
[junit4] > 1) Thread[id=4961, name=StoppableIndexingThread,
state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
[junit4] > 2) Thread[id=4962, name=StoppableIndexingThread,
state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
[junit4] > at
__randomizedtesting.SeedInfo.seed([AD2575BC4A303AF3]:0)Throwable #2:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
[junit4] > 1) Thread[id=4961, name=StoppableIndexingThread,
state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:136)
[junit4] > 2) Thread[id=4962, name=StoppableIndexingThread,
state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at
java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304)
[junit4] > at java.lang.StringCoding.encode(StringCoding.java:344)
[junit4] > at java.lang.String.getBytes(String.java:906)
[junit4] > at
org.apache.solr.common.util.ContentStreamBase$StringStream.getStream(ContentStreamBase.java:182)
[junit4] > at
org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getStream(RequestWriter.java:125)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:439)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:239)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
[junit4] > at
org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:177)
[junit4] > at
org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:116)
[junit4] > at
__randomizedtesting.SeedInfo.seed([AD2575BC4A303AF3]:0)
[junit4] Completed [152/547] on J1 in 107.62s, 1 test, 1 failure, 2 errors
<<< FAILURES!
[...truncated 1208 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:516:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432:
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987:
There were test failures: 547 suites (8 ignored), 2146 tests, 2 suite-level
errors, 1 failure, 1187 ignored (12 assumptions) [seed: AD2575BC4A303AF3]
Total time: 169 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
