Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/925/

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [HdfsTransactionLog]
        at __randomizedtesting.SeedInfo.seed([FF970A7698634A56]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
        at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test

Error Message:
shard2 is not consistent.  Got 2982 from http://127.0.0.1:60933/collection1 
(previous client) and got 2914 from http://127.0.0.1:50120/collection1

Stack Trace:
java.lang.AssertionError: shard2 is not consistent.  Got 2982 from 
http://127.0.0.1:60933/collection1 (previous client) and got 2914 from 
http://127.0.0.1:50120/collection1
        at 
__randomizedtesting.SeedInfo.seed([FF970A7698634A56:77C335AC369F27AE]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1265)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1244)
        at 
org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:162)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:939)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12393 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/init-core-data-001
   [junit4]   2> 4871172 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4871249 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4871263 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4871265 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 4871318 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_39235_hdfs____.7yqelk/webapp
   [junit4]   2> 4871514 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4871874 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39235
   [junit4]   2> 4872148 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4872150 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 4872163 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_46554_datanode____pxc7uh/webapp
   [junit4]   2> 4872313 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4872639 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46554
   [junit4]   2> 4872763 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4872765 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 4872799 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_53847_datanode____6bxl0/webapp
   [junit4]   2> 4872835 INFO  (IPC Server handler 3 on 44824) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-91c52479-3012-44c2-b159-5f1aebb1f904 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=82c2b256-7011-4c4d-b803-e6b1384aa407, infoPort=46554, 
ipcPort=43823, storageInfo=lv=-56;cid=testClusterID;nsid=1379210169;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4872835 INFO  (IPC Server handler 3 on 44824) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-fc068f29-d0fa-4150-8b28-8a100e8d5726 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=82c2b256-7011-4c4d-b803-e6b1384aa407, infoPort=46554, 
ipcPort=43823, storageInfo=lv=-56;cid=testClusterID;nsid=1379210169;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4873028 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 4873353 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[FF970A7698634A56]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53847
   [junit4]   2> 4873620 INFO  (IPC Server handler 5 on 44824) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-bac0d1be-a954-4195-956e-fd964549f186 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=1d7c42e4-6e64-47c8-9d3f-620a0d535345, infoPort=53847, 
ipcPort=35290, storageInfo=lv=-56;cid=testClusterID;nsid=1379210169;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 4873621 INFO  (IPC Server handler 5 on 44824) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-744bb83c-187a-4483-947d-c1f035dafaa4 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=1d7c42e4-6e64-47c8-9d3f-620a0d535345, infoPort=53847, 
ipcPort=35290, storageInfo=lv=-56;cid=testClusterID;nsid=1379210169;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 4873752 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4873761 INFO  (Thread-27162) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4873761 INFO  (Thread-27162) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 4873861 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:51229
   [junit4]   2> 4873861 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4873874 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4873876 INFO  (zkCallback-4054-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7ec3ff2c 
name:ZooKeeperConnection Watcher:127.0.0.1:51229 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4873876 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4873877 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4873877 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 4873886 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4873886 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4873887 INFO  (zkCallback-4055-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3ca0b061 
name:ZooKeeperConnection Watcher:127.0.0.1:51229/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4873887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4873887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4873887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 4873889 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 4873890 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 4873891 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 4873893 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 4873893 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4873895 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 4873895 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 4873897 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4873897 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4873898 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 4873899 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4873900 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 4873900 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4873901 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 4873901 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 4873903 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4873903 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4873904 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4873904 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4873905 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4873906 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4873907 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4873907 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4873908 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 4873909 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4873998 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/cores/collection1
   [junit4]   2> 4874000 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4874018 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@32c35bdc{/,null,AVAILABLE}
   [junit4]   2> 4874018 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@6419d197{HTTP/1.1}{127.0.0.1:40114}
   [junit4]   2> 4874018 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.Server Started @4877528ms
   [junit4]   2> 4874018 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:44824/hdfs__localhost_44824__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001_tempDir-002_control_data,
 hostContext=/, hostPort=40114, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/cores}
   [junit4]   2> 4874018 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 4874019 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001'
   [junit4]   2> 4874019 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4874019 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 4874019 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4874029 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4874038 INFO  (zkCallback-4056-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@a2fee81 name:ZooKeeperConnection 
Watcher:127.0.0.1:51229/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 4874038 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4874039 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4874049 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 4874050 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/solr.xml
   [junit4]   2> 4874054 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/cores
   [junit4]   2> 4874054 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 2147049090
   [junit4]   2> 4874054 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001]
   [junit4]   2> 4874055 WARN  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CoreContainer Couldn't add files from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/lib
 to classpath: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/lib
   [junit4]   2> 4874055 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4874056 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4874057 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4874057 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4874057 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51229/solr
   [junit4]   2> 4874057 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4874057 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4874073 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4874089 INFO  (zkCallback-4059-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@744ca242 
name:ZooKeeperConnection Watcher:127.0.0.1:51229 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4874090 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4874090 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4874115 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 4874123 INFO  
(zkCallback-4060-thread-1-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@1d5a6963 
name:ZooKeeperConnection Watcher:127.0.0.1:51229/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4874123 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 4874124 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 4874126 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 4874128 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 4874129 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 4874130 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 4874132 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 4874134 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 4874135 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 4874135 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 4874136 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:40114_
   [junit4]   2> 4874136 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:40114_
   [junit4]   2> 4874137 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 4874138 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 4874139 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4874139 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/95336584640200708-127.0.0.1:40114_-n_0000000000
   [junit4]   2> 4874140 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:40114_
   [junit4]   2> 4874140 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 4874141 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer Overseer 
(id=95336584640200708-127.0.0.1:40114_-n_0000000000) starting
   [junit4]   2> 4874142 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 4874145 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4874162 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 4874162 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 4874176 INFO  
(OverseerCollectionConfigSetProcessor-95336584640200708-127.0.0.1:40114_-n_0000000000)
 [n:127.0.0.1:40114_    ] o.a.s.c.OverseerTaskProcessor Process current queue 
of overseer operations
   [junit4]   2> 4874189 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 4874213 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 4874214 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4874214 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.CorePropertiesLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/cores
   [junit4]   2> 4874215 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 4874215 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.CorePropertiesLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/cores/collection1
   [junit4]   2> 4874215 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 4874230 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 4874231 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 4874238 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 4874238 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:40114_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() 
done
   [junit4]   2> 4874238 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40114";,
   [junit4]   2>   "node_name":"127.0.0.1:40114_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 4874239 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40114";,
   [junit4]   2>   "node_name":"127.0.0.1:40114_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4874239 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.ZkController 
look for our core node name
   [junit4]   2> 4874239 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
control_collection
   [junit4]   2> 4874240 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 4874241 INFO  
(zkCallback-4060-thread-1-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4874254 INFO  
(zkCallback-4060-thread-2-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4875239 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.ZkController 
waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4875240 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Check for collection zkNode:control_collection
   [junit4]   2> 4875240 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection zkNode exists
   [junit4]   2> 4875240 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/control_collection]
   [junit4]   2> 4875241 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=[/collections/control_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 4875241 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/cores/collection1'
   [junit4]   2> 4875241 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4875241 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 4875242 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.Config loaded 
config solrconfig.xml with version 0 
   [junit4]   2> 4875247 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
current version of requestparams : -1
   [junit4]   2> 4875250 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 4875260 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4875271 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 4875496 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
default search field in schema is text
   [junit4]   2> 4875497 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
unique key field: id
   [junit4]   2> 4875510 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4875513 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 4875533 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 4875534 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection 
control_collection
   [junit4]   2> 4875534 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 4875534 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:44824/solr_hdfs_home
   [junit4]   2> 4875534 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4875534 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/control-001/cores/collection1],
 
dataDir=[hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 4875534 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@136e0e7c
   [junit4]   2> 4875535 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4875566 WARN  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds 
and try again.
   [junit4]   2> 4880570 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 4880571 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 4880590 WARN  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:44824/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 4880590 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4880598 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 4880598 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 4880598 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 4880608 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 4880608 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 4880609 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=9, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=1.0]
   [junit4]   2> 4880666 INFO  (IPC Server handler 6 on 44824) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45850 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-744bb83c-187a-4483-947d-c1f035dafaa4:NORMAL:127.0.0.1:48754|RBW],
 
ReplicaUnderConstruction[[DISK]DS-91c52479-3012-44c2-b159-5f1aebb1f904:NORMAL:127.0.0.1:45850|RBW]]}
 size 0
   [junit4]   2> 4880668 INFO  (IPC Server handler 1 on 44824) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:48754 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-91c52479-3012-44c2-b159-5f1aebb1f904:NORMAL:127.0.0.1:45850|RBW],
 
ReplicaUnderConstruction[[DISK]DS-bac0d1be-a954-4195-956e-fd964549f186:NORMAL:127.0.0.1:48754|FINALIZED]]}
 size 0
   [junit4]   2> 4880670 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d8a813); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4880670 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 4880671 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 4880674 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"nodistrib"
   [junit4]   2> 4880675 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"dedupe"
   [junit4]   2> 4880675 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4880675 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"stored_sig"
   [junit4]   2> 4880675 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4880675 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-explicit"
   [junit4]   2> 4880675 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 4880676 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 4880676 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating 
implicit default
   [junit4]   2> 4880676 WARN  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 4880678 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4880679 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4880679 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4880680 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 4880684 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 4880684 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 4880685 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4880685 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
tlogDfsReplication=2
   [junit4]   2> 4880695 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 4880695 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4880696 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=21, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=55.7041015625, 
floorSegmentMB=1.185546875, forceMergeDeletesPctAllowed=28.84543042691591, 
segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.508673688239389
   [junit4]   2> 4880702 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:44824/solr_hdfs_home/control_collection/core_node1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d8a813); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 4880702 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 4880702 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@70eb1303[collection1] main]
   [junit4]   2> 4880702 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/control_collection]
   [junit4]   2> 4880703 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader path=[/collections/control_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 4880703 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the 
RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4880703 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 4880703 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 4880703 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4880704 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 4880704 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4880704 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 4880704 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4880705 INFO  
(searcherExecutor-9068-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@70eb1303[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4880705 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 4880705 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version 
buckets from index
   [junit4]   2> 4880706 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket 
highest value from index
   [junit4]   2> 4880706 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1525385361507745792
   [junit4]   2> 4880706 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 
1525385361507745792
   [junit4]   2> 4880706 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 4880706 INFO  
(coreLoadExecutor-9067-thread-1-processing-n:127.0.0.1:40114_) 
[n:127.0.0.1:40114_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 4880707 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
Register replica - core:collection1 address:http://127.0.0.1:40114 
collection:control_collection shard:shard1
   [junit4]   2> 4880707 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 4880710 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector 
Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/95336584640200708-core_node1-n_0000000000
   [junit4]   2> 4880710 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 
and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 4880712 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4880712 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4880712 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:40114/collection1/
   [junit4]   2> 4880712 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 4880712 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 4880712 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:40114/collection1/ has no replicas
   [junit4]   2> 4880713 INFO  
(zkCallback-4060-thread-3-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880713 INFO  
(zkCallback-4060-thread-2-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880713 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 4880716 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 4880718 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:40114/collection1/ shard1
   [junit4]   2> 4880718 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:40114";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 4880741 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4880741 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4880742 INFO  (zkCallback-4062-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@63721dff 
name:ZooKeeperConnection Watcher:127.0.0.1:51229/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4880742 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4880743 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4880743 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 4880745 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 4880745 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 4880745 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4880745 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4880746 INFO  (zkCallback-4063-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@11cb427b 
name:ZooKeeperConnection Watcher:127.0.0.1:51229/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4880746 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4880746 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4880748 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 2
   [junit4]   2> 4880748 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
collection1
   [junit4]   2> 4880749 INFO  
(zkCallback-4060-thread-3-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880749 INFO  
(zkCallback-4060-thread-2-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880749 INFO  (zkCallback-4062-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880769 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
We are http://127.0.0.1:40114/collection1/ and leader is 
http://127.0.0.1:40114/collection1/
   [junit4]   2> 4880769 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40114
   [junit4]   2> 4880769 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 4880769 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
publishing state=active
   [junit4]   2> 4880770 INFO  
(coreZkRegister-9061-thread-1-processing-n:127.0.0.1:40114_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40114_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 4880776 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40114";,
   [junit4]   2>   "node_name":"127.0.0.1:40114_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 4880777 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40114";,
   [junit4]   2>   "node_name":"127.0.0.1:40114_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 4880777 INFO  
(OverseerStateUpdate-95336584640200708-127.0.0.1:40114_-n_0000000000) 
[n:127.0.0.1:40114_    ] o.a.s.c.o.ZkStateWriter going to create_collection 
/collections/collection1/state.json
   [junit4]   2> 4880838 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/cores/collection1
   [junit4]   2> 4880839 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001
   [junit4]   2> 4880841 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 4880880 INFO  (zkCallback-4062-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880880 INFO  
(zkCallback-4060-thread-2-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880880 INFO  
(zkCallback-4060-thread-3-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 4880886 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@594fe2d9{/,null,AVAILABLE}
   [junit4]   2> 4880886 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@14524606{HTTP/1.1}{127.0.0.1:36220}
   [junit4]   2> 4880886 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.e.j.s.Server Started @4884396ms
   [junit4]   2> 4880886 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:44824/hdfs__localhost_44824__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001_tempDir-002_jetty1,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=36220, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/cores}
   [junit4]   2> 4880887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 4880887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001'
   [junit4]   2> 4880887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4880887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 4880887 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4880901 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4880909 INFO  (zkCallback-4064-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@63d40179 
name:ZooKeeperConnection Watcher:127.0.0.1:51229/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4880910 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4880910 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4880911 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 4880911 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/solr.xml
   [junit4]   2> 4880916 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/cores
   [junit4]   2> 4880916 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 565596405
   [junit4]   2> 4880916 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001]
   [junit4]   2> 4880916 WARN  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.CoreContainer Couldn't add files from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/lib
 to classpath: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/lib
   [junit4]   2> 4880916 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 4880918 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4880918 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4880918 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4880918 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51229/solr
   [junit4]   2> 4880918 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 4880919 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 4880922 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 4880925 INFO  (zkCallback-4067-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@37714d40 
name:ZooKeeperConnection Watcher:127.0.0.1:51229 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4880926 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 4880926 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 4880930 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 4880933 INFO  
(zkCallback-4068-thread-1-processing-n:127.0.0.1:36220_) [n:127.0.0.1:36220_    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6d91479c 
name:ZooKeeperConnection Watcher:127.0.0.1:51229/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4880933 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 4880938 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 4881987 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36220_
   [junit4]   2> 4881987 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:36220_
   [junit4]   2> 4881989 INFO  (zkCallback-4062-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 4881989 INFO  
(zkCallback-4060-thread-3-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 4881990 INFO  
(zkCallback-4060-thread-2-processing-n:127.0.0.1:40114_) [n:127.0.0.1:40114_    
] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 4881991 INFO  
(zkCallback-4068-thread-1-processing-n:127.0.0.1:36220_) [n:127.0.0.1:36220_    
] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 4881991 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4881992 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/95336584640200713-127.0.0.1:36220_-n_0000000001
   [junit4]   2> 4881992 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/95336584640200708-127.0.0.1:40114_-n_0000000000 to 
know if I could be the leader
   [junit4]   2> 4882049 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 4882049 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 4882049 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.CorePropertiesLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/cores
   [junit4]   2> 4882062 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 4882062 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.CorePropertiesLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_FF970A7698634A56-001/shard-1-001/cores/collection1
   [junit4]   2> 4882062 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 4882070 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 4882070 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[FF970A7698634A56]) 
[n:127.0.0.1:36220_    ] o.a.s.s.SolrDispatchFilter SolrDispa

[...truncated too long message...]

elog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-common-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-hdfs-2.6.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/hadoop-minikdc-2.6.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/analysis/icu/lucene-analyzers-icu-6.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/contrib/analysis-extras/lib/icu4j-54.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.2.jar
 com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J1-20160205_234557_613.events
 
@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/temp/junit4-J1-20160205_234557_613.suites
 -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from 
the forked process? This may indicate JVM crash or runner bugs.
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]     at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:747:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:684:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:524:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1457:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1014:
 At least one slave process threw an exception, first: Quit event not received 
from the forked process? This may indicate JVM crash or runner bugs.

Total time: 281 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to