Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/733/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test

Error Message:
The Monkey ran for over 30 seconds and no jetties were stopped - this is worth 
investigating!

Stack Trace:
java.lang.AssertionError: The Monkey ran for over 30 seconds and no jetties 
were stopped - this is worth investigating!
        at 
__randomizedtesting.SeedInfo.seed([9CD33C16F091D81F:148703CC5E6DB5E7]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:537)
        at 
org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:143)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:41779/db: Error CREATEing SolrCore 
'halfcollection_shard1_replica1': Unable to create core 
[halfcollection_shard1_replica1] Caused by: Could not get shard id for core: 
halfcollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:41779/db: Error CREATEing SolrCore 
'halfcollection_shard1_replica1': Unable to create core 
[halfcollection_shard1_replica1] Caused by: Could not get shard id for core: 
halfcollection_shard1_replica1
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:301)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:418)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:168)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=91932, name=collection2, 
state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=91932, name=collection2, state=RUNNABLE, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:43666: Could not find collection : 
awholynewstresscollection_collection2_1
        at __randomizedtesting.SeedInfo.seed([9CD33C16F091D81F]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:234)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:226)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:894)




Build Log:
[...truncated 10393 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/init-core-data-001
   [junit4]   2> 942906 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 942906 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /i/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 942946 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 942958 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 942968 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 942982 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_49849_hdfs____.ttkb3y/webapp
   [junit4]   2> 943113 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 943683 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49849
   [junit4]   2> 943861 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 943862 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 943874 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_45248_datanode____jjzvy4/webapp
   [junit4]   2> 944026 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 944947 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45248
   [junit4]   2> 945002 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 945003 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 945033 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_54138_datanode____xt4c72/webapp
   [junit4]   2> 945106 INFO  (IPC Server handler 2 on 49516) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-47bf18b7-d3f3-4d8b-9f96-e54b5e758769 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=75c00495-f557-4e5d-8fa0-41c4233d82f0, infoPort=45248, 
ipcPort=47254, storageInfo=lv=-56;cid=testClusterID;nsid=40507542;c=0), blocks: 
0, hasStaleStorages: true, processing time: 3 msecs
   [junit4]   2> 945107 INFO  (IPC Server handler 2 on 49516) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-f34ce943-4c02-41b7-a662-46f402f762e0 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=75c00495-f557-4e5d-8fa0-41c4233d82f0, infoPort=45248, 
ipcPort=47254, storageInfo=lv=-56;cid=testClusterID;nsid=40507542;c=0), blocks: 
0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 945181 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 945785 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[9CD33C16F091D81F]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54138
   [junit4]   2> 945863 INFO  (IPC Server handler 3 on 49516) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-903cd247-75e9-4131-b384-ba5082f93343 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=05aa3e1b-6fb5-4d75-92a1-e1471e1dd0d1, infoPort=54138, 
ipcPort=59881, storageInfo=lv=-56;cid=testClusterID;nsid=40507542;c=0), blocks: 
0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 945864 INFO  (IPC Server handler 3 on 49516) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-cdcf3d6a-fdee-444a-8a46-6621cdfdf7f6 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=05aa3e1b-6fb5-4d75-92a1-e1471e1dd0d1, infoPort=54138, 
ipcPort=59881, storageInfo=lv=-56;cid=testClusterID;nsid=40507542;c=0), blocks: 
0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 945953 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 945953 INFO  (Thread-3827) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 945954 INFO  (Thread-3827) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 946054 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:48362
   [junit4]   2> 946054 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 946055 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 946065 INFO  (zkCallback-365-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@322be1f1 
name:ZooKeeperConnection Watcher:127.0.0.1:48362 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 946065 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 946065 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 946065 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 946071 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 946072 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 946075 INFO  (zkCallback-366-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@285bab4 name:ZooKeeperConnection 
Watcher:127.0.0.1:48362/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 946076 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 946076 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 946076 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 946079 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 946080 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 946081 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 946083 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 946083 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 946085 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 946086 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 946089 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 946089 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 946091 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 946091 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 946092 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 946093 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 946094 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 946095 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 946096 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 946097 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 946099 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 946100 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 946103 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 946103 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 946105 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 946105 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 946107 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 946108 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 946448 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores/collection1
   [junit4]   2> 946450 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 946451 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@5dd1df20{/i,null,AVAILABLE}
   [junit4]   2> 946452 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@65cc78ba{HTTP/1.1}{127.0.0.1:46915}
   [junit4]   2> 946452 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.Server Started @949707ms
   [junit4]   2> 946452 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:49516/hdfs__localhost_49516__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001_tempDir-002_control_data,
 hostContext=/i, hostPort=46915, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores}
   [junit4]   2> 946453 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 946453 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/'
   [junit4]   2> 946467 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/solr.xml
   [junit4]   2> 946472 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores
   [junit4]   2> 946472 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 1040738979
   [junit4]   2> 946472 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/]
   [junit4]   2> 946472 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/lib
   [junit4]   2> 946472 WARN  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/lib).
   [junit4]   2> 946479 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 946480 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 946481 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 946481 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 946481 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48362/solr
   [junit4]   2> 946481 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 946481 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 946482 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 946484 INFO  (zkCallback-368-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@14813fe0 
name:ZooKeeperConnection Watcher:127.0.0.1:48362 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 946485 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 946485 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 946487 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x14e6c3716180002, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 946487 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 946490 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7d815694 
name:ZooKeeperConnection Watcher:127.0.0.1:48362/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 946491 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 946493 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 946495 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 946496 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 946498 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 946500 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 946502 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 946503 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 946504 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 946505 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:46915_i
   [junit4]   2> 946506 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:46915_i
   [junit4]   2> 946507 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 946508 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 946510 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 946510 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/94131626071031811-127.0.0.1:46915_i-n_0000000000
   [junit4]   2> 946511 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:46915_i
   [junit4]   2> 946511 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/leader
   [junit4]   2> 946512 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer Overseer 
(id=94131626071031811-127.0.0.1:46915_i-n_0000000000) starting
   [junit4]   2> 946514 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 946519 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 946519 INFO  
(OverseerCollectionProcessor-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.OverseerCollectionProcessor Process current 
queue of collection creations
   [junit4]   2> 946519 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 946520 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 946523 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 946523 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 946524 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.CoresLocator Looking for core definitions 
underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores
   [junit4]   2> 946525 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores/collection1,
 collection=control_collection, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 946526 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.CoresLocator Found core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores/collection1/
   [junit4]   2> 946526 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 946526 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 946526 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 946527 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 946527 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:46915_i    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() 
done
   [junit4]   2> 946527 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.ZkController 
look for our core node name
   [junit4]   2> 946527 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue 
state SyncConnected
   [junit4]   2> 946528 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46915/i";,
   [junit4]   2>   "node_name":"127.0.0.1:46915_i",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 946528 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46915/i";,
   [junit4]   2>   "node_name":"127.0.0.1:46915_i",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 946528 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
control_collection
   [junit4]   2> 946529 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 946530 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 946531 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader Updated cluster state version to 1
   [junit4]   2> 947527 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.ZkController 
waiting to find shard id in clusterstate for collection1
   [junit4]   2> 947527 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.ZkController 
Check for collection zkNode:control_collection
   [junit4]   2> 947528 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection zkNode exists
   [junit4]   2> 947528 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 947529 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 947529 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores/collection1/'
   [junit4]   2> 947544 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.Config 
loaded config solrconfig.xml with version 0 
   [junit4]   2> 947548 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.SolrConfig 
current version of requestparams : -1
   [junit4]   2> 947566 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 947573 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.c.Config 
Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 947575 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.s.IndexSchema 
Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 947579 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 947647 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.s.IndexSchema 
default search field in schema is text
   [junit4]   2> 947647 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] o.a.s.s.IndexSchema 
unique key field: id
   [junit4]   2> 947648 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 947651 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 947664 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 947664 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 947664 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:49516/solr_hdfs_home
   [junit4]   2> 947664 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 947664 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/control-001/cores/collection1/],
 dataDir=[null]
   [junit4]   2> 947665 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@74817fc2
   [junit4]   2> 947665 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 947700 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.h.HdfsLocalityReporter Registering direcotry 
hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data for 
locality metrics.
   [junit4]   2> 947700 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 947702 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 947710 WARN  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:49516/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 947711 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 947718 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 947718 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 947718 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 947731 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 947731 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.h.HdfsLocalityReporter Registering direcotry 
hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data/index 
for locality metrics.
   [junit4]   2> 947731 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 947732 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.00390625, 
noCFSRatio=0.25892960546340227]
   [junit4]   2> 947754 INFO  (IPC Server handler 8 on 49516) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42080 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-f34ce943-4c02-41b7-a662-46f402f762e0:NORMAL:127.0.0.1:37549|RBW],
 
ReplicaUnderConstruction[[DISK]DS-903cd247-75e9-4131-b384-ba5082f93343:NORMAL:127.0.0.1:42080|FINALIZED]]}
 size 0
   [junit4]   2> 947756 INFO  (IPC Server handler 9 on 49516) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:37549 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-f34ce943-4c02-41b7-a662-46f402f762e0:NORMAL:127.0.0.1:37549|RBW],
 
ReplicaUnderConstruction[[DISK]DS-903cd247-75e9-4131-b384-ba5082f93343:NORMAL:127.0.0.1:42080|FINALIZED]]}
 size 0
   [junit4]   2> 947761 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7e9e3b2 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@15521e66); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 947761 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 947762 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:49516/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 947765 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"nodistrib"
   [junit4]   2> 947765 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"dedupe"
   [junit4]   2> 947765 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 947765 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"stored_sig"
   [junit4]   2> 947766 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 947766 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-explicit"
   [junit4]   2> 947766 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 947766 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 947766 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating 
implicit default
   [junit4]   2> 947768 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 947769 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 947770 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 947771 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 947781 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 947781 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 947782 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 947782 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 947792 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 947792 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 947793 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=11, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=6.85546875, 
floorSegmentMB=1.822265625, forceMergeDeletesPctAllowed=29.77765088559873, 
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.3241432425070051
   [junit4]   2> 947801 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b7e9e3b2 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@15521e66); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 947801 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 947802 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening Searcher@520b585b[collection1] main
   [junit4]   2> 947803 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 947804 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 947804 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the 
RestManager with znodeBase: /configs/conf1
   [junit4]   2> 947805 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 947805 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 947805 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 947805 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 947805 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 947806 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 947806 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 947807 INFO  
(searcherExecutor-1894-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@520b585b[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 947811 WARN  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for 
hdfs://localhost:54281/solr3 due to exception: java.net.ConnectException: Call 
From lucene1-us-west/10.41.0.5 to localhost:54281 failed on connection 
exception: java.net.ConnectException: Connection refused; For more details see: 
 http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 947811 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 947812 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version 
buckets from index
   [junit4]   2> 947812 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket 
highest value from index
   [junit4]   2> 947812 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1506106019083714560
   [junit4]   2> 947812 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Took 1 ms to seed version buckets with highest version 
1506106019083714560
   [junit4]   2> 947812 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 947813 INFO  
(coreLoadExecutor-1893-thread-1-processing-n:127.0.0.1:46915_i) 
[n:127.0.0.1:46915_i c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 947813 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
Register replica - core:collection1 address:http://127.0.0.1:46915/i 
collection:control_collection shard:shard1
   [junit4]   2> 947814 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 947818 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector 
Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/94131626071031811-core_node1-n_0000000000
   [junit4]   2> 947825 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 947827 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue 
state SyncConnected
   [junit4]   2> 947828 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 947828 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 947828 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:46915/i/collection1/
   [junit4]   2> 947828 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 947828 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 947828 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:46915/i/collection1/ has no replicas
   [junit4]   2> 947829 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:46915/i/collection1/ shard1
   [junit4]   2> 947829 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 947832 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue 
state SyncConnected
   [junit4]   2> 947833 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:46915/i";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 947935 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 947937 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader Updated cluster state version to 2
   [junit4]   2> 947983 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
We are http://127.0.0.1:46915/i/collection1/ and leader is 
http://127.0.0.1:46915/i/collection1/
   [junit4]   2> 947984 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46915/i
   [junit4]   2> 947984 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 947984 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
publishing state=active
   [junit4]   2> 947984 INFO  
(coreZkRegister-1887-thread-1-processing-n:127.0.0.1:46915_i x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:46915_i 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 947985 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue 
state SyncConnected
   [junit4]   2> 947986 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46915/i";,
   [junit4]   2>   "node_name":"127.0.0.1:46915_i",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 947987 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46915/i";,
   [junit4]   2>   "node_name":"127.0.0.1:46915_i",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 948029 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 948030 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 948032 INFO  (zkCallback-371-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5ae72f98 
name:ZooKeeperConnection Watcher:127.0.0.1:48362/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 948032 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 948033 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 948033 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 948035 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 948035 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 948036 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 948036 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 948045 INFO  (zkCallback-372-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4a6b394a 
name:ZooKeeperConnection Watcher:127.0.0.1:48362/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 948045 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 948045 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 948047 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.DistributedQueue NodeChildrenChanged fired on path /overseer/queue 
state SyncConnected
   [junit4]   2> 948048 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"3",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 948048 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
collection1
   [junit4]   2> 948049 INFO  (zkCallback-371-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 948049 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 948051 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 948053 INFO  (zkCallback-371-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated cluster state version to 3
   [junit4]   2> 948153 INFO  
(OverseerStateUpdate-94131626071031811-127.0.0.1:46915_i-n_0000000000) 
[n:127.0.0.1:46915_i    ] o.a.s.c.o.ZkStateWriter going to create_collection 
/collections/collection1/state.json
   [junit4]   2> 948154 INFO  (zkCallback-371-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 948154 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 948156 INFO  (zkCallback-371-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 948156 INFO  
(zkCallback-369-thread-1-processing-n:127.0.0.1:46915_i) [n:127.0.0.1:46915_i   
 ] o.a.s.c.c.ZkStateReader Updated cluster state version to 4
   [junit4]   2> 948376 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/cores/collection1
   [junit4]   2> 948377 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001
   [junit4]   2> 948378 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.Server jetty-9.2.11.v20150529
   [junit4]   2> 948380 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@46667580{/i,null,AVAILABLE}
   [junit4]   2> 948381 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@3e8da3d4{HTTP/1.1}{127.0.0.1:50921}
   [junit4]   2> 948381 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.e.j.s.Server Started @951636ms
   [junit4]   2> 948381 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:49516/hdfs__localhost_49516__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001_tempDir-002_jetty1,
 solrconfig=solrconfig.xml, hostContext=/i, hostPort=50921, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/cores}
   [junit4]   2> 948381 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@c387f44
   [junit4]   2> 948382 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/'
   [junit4]   2> 948396 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/solr.xml
   [junit4]   2> 948401 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/cores
   [junit4]   2> 948401 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 119478646
   [junit4]   2> 948401 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/]
   [junit4]   2> 948402 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/lib
   [junit4]   2> 948402 WARN  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_9CD33C16F091D81F-001/shard-1-001/lib).
   [junit4]   2> 948409 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 948410 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 948410 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 948411 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 948411 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48362/solr
   [junit4]   2> 948411 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 948411 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 948412 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 948414 INFO  (zkCallback-374-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4364797f 
name:ZooKeeperConnection Watcher:127.0.0.1:48362 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 948414 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 948415 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 948417 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:50921_i    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 948419 INFO  
(zkCallback-375-thread-1-processing-n:127.0.0.1:50921_i) [n:127.0.0.1:50921_i   
 ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5e79a24a 
name:ZooKeeperConnection Watcher:127.0.0.1:48362/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 948419 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:50921_i    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 948426 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:50921_i    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 949430 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F091D81F]) 
[n:127.0.0.1:50921_i    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:50921_i
   [junit4]   2> 949430 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[9CD33C16F

[...truncated too long message...]

ava:212)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:50)
   [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2>        ... 53 more
   [junit4]   2> 5570189 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[9CD33C16F091D81F]-worker) 
[n:127.0.0.1:43666_ c:addReplicaColl s:shard1 r:core_node5 
x:addReplicaColl_shard1_replica3] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Jul 08, 2015 3:07:31 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene53, sim=DefaultSimilarity, 
locale=th, timezone=US/Alaska
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 
1.8.0_25 (64-bit)/cpus=4,threads=2,free=142078784,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [TestMiniSolrCloudClusterSSL, 
TestCopyFieldCollectionResource, TestSolrConfigHandlerConcurrent, 
TestCursorMarkWithoutUniqueKey, TestReplicationHandlerBackup, 
TestSolrQueryParser, TestRawResponseWriter, TestCollectionAPI, 
TestMaxScoreQueryParser, DistributedSuggestComponentTest, 
DistribJoinFromCollectionTest, TestSolrCloudWithKerberosAlt, DeleteReplicaTest, 
FullSolrCloudDistribCmdsTest, TestQuerySenderNoQuery, 
FileBasedSpellCheckerTest, StatsComponentTest, 
SimpleCollectionCreateDeleteTest, TestPerFieldSimilarity, TestLRUStatsCache, 
TestHashQParserPlugin, AddBlockUpdateTest, BinaryUpdateRequestHandlerTest, 
OverseerTest, TestReqParamsAPI, RegexBoostProcessorTest, 
TestDynamicFieldCollectionResource, UnloadDistributedZkTest, SOLR749Test, 
XsltUpdateRequestHandlerTest, SimpleFacetsTest, PluginInfoTest, 
HdfsSyncSliceTest, PolyFieldTest, TestQueryTypes, CollectionReloadTest, 
HdfsDirectoryTest, TestStressReorder, OpenExchangeRatesOrgProviderTest, 
StressHdfsTest, RecoveryZkTest, RuleEngineTest, DirectSolrSpellCheckerTest, 
HdfsLockFactoryTest, TestExtendedDismaxParser, TestZkChroot, TestQueryUtils, 
TestShortCircuitedRequests, LeaderInitiatedRecoveryOnCommitTest, 
LeaderElectionTest, HdfsNNFailoverTest, CircularListTest, 
TestChildDocTransformer, BasicDistributedZk2Test, TestCollationField, TestTrie, 
TestSmileRequest, DirectUpdateHandlerOptimizeTest, VersionInfoTest, 
SharedFSAutoReplicaFailoverTest, DistributedVersionInfoTest, TestInitParams, 
CloudExitableDirectoryReaderTest, DistributedQueryElevationComponentTest, 
TestBM25SimilarityFactory, TestFieldTypeResource, TestDocSet, 
TestDefaultSimilarityFactory, CustomCollectionTest, SynonymTokenizerTest, 
DirectUpdateHandlerTest, TestPartialUpdateDeduplication, TestXIncludeConfig, 
ReplicationFactorTest, TestSolrQueryParserDefaultOperatorResource, 
TestSolr4Spatial2, SearchHandlerTest, DOMUtilTest, TestAuthorizationFramework, 
TestBinaryResponseWriter, ClusterStateUpdateTest, TestRandomFaceting, 
ZkSolrClientTest, ZkCLITest, TestDistributedSearch, TestSolrDeletionPolicy2, 
SampleTest, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, 
OutputWriterTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, 
DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, 
EchoParamsTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, 
TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, 
ResourceLoaderTest, TestFastOutputStream, ScriptEngineTest, DateMathParserTest, 
PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, 
RAMDirectoryFactoryTest, ClusterStateTest, TestSolrJ, TestUtils, 
TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, 
DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, 
DistributedIntervalFacetingTest, AsyncMigrateRouteKeyTest, 
ConcurrentDeleteAndCreateCollectionTest, ConnectionManagerTest, 
DeleteShardTest, MigrateRouteKeyTest, 
OverriddenZkACLAndCredentialsProvidersTest, OverseerStatusTest, 
RecoveryAfterSoftCommitTest, RemoteQueryErrorTest, SSLMigrationTest, 
SaslZkACLProviderTest, ShardSplitTest, SharedFSAutoReplicaFailoverUtilsTest, 
SolrCloudExampleTest, TestAuthenticationFramework, TestCloudPivotFacet, 
TestCryptoKeys, TestDownShardTolerantSearch, TestLeaderElectionZkExpiry, 
TestRandomRequestDistribution, TestReplicaProperties, 
TestRequestStatusCollectionAPI, TriLevelCompositeIdRoutingTest, 
VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, 
HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [514/514] on J2 in 1945.34s, 1 test, 1 error <<< FAILURES!

[...truncated 13 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:533:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:474:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:230:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:513:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1426:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981:
 There were test failures: 514 suites, 2022 tests, 2 errors, 1 failure, 20 
ignored (10 assumptions)

Total time: 197 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to