Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/202/

4 tests failed.
FAILED:  
org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels

Error Message:
this writer hit an unrecoverable error; cannot complete forceMerge

Stack Trace:
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot 
complete forceMerge
        at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1900)
        at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1850)
        at 
org.apache.lucene.index.RandomIndexWriter.doRandomForceMerge(RandomIndexWriter.java:345)
        at 
org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:367)
        at 
org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
        at 
org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels(TestBlockJoin.java:1811)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
        at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
        at 
org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
        at 
org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
        at 
org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
        at 
org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
        at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
        at 
org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:186)
        at 
org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4901)
        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4377)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)


FAILED:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest.test

Error Message:
document count mismatch.  control=4065 sum(shards)=4066 cloudClient=4066

Stack Trace:
java.lang.AssertionError: document count mismatch.  control=4065 
sum(shards)=4066 cloudClient=4066
        at 
__randomizedtesting.SeedInfo.seed([CB0A51AB95826B36:435E6E713B7E06CE]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1323)
        at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:228)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets

Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2

Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /collections/withconfigset2
        at 
__randomizedtesting.SeedInfo.seed([CB0A51AB95826B36:E6741EF162A6D13A]:0)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={
   "replicationFactor":"3",   "shards":{"shard1":{       
"range":"80000000-7fffffff",       "state":"active",       "replicas":{}}},   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={
  "replicationFactor":"3",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
        at 
__randomizedtesting.SeedInfo.seed([CB0A51AB95826B36:837F251F93B144A3]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 7763 lines...]
   [junit4] Suite: org.apache.lucene.search.join.TestBlockJoin
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestBlockJoin 
-Dtests.method=testMultiChildQueriesOfDiffParentLevels 
-Dtests.seed=8114F6C2AAD21A68 -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=de-LU -Dtests.timezone=America/Panama -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR    114s J1 | 
TestBlockJoin.testMultiChildQueriesOfDiffParentLevels <<<
   [junit4]    > Throwable #1: java.lang.IllegalStateException: this writer hit 
an unrecoverable error; cannot complete forceMerge
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1900)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1850)
   [junit4]    >        at 
org.apache.lucene.index.RandomIndexWriter.doRandomForceMerge(RandomIndexWriter.java:345)
   [junit4]    >        at 
org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:367)
   [junit4]    >        at 
org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:313)
   [junit4]    >        at 
org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels(TestBlockJoin.java:1811)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]    >        at 
org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]    >        at 
org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    >        at 
org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    >        at 
org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]    >        at 
org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]    >        at 
org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]    >        at 
org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:186)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4901)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4377)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable
 #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an 
uncaught exception in thread: Thread[id=37, name=Lucene Merge Thread #24, 
state=RUNNABLE, group=TGRP-TestBlockJoin]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: 
java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([8114F6C2AAD21A68]:0)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]    >        at 
org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]    >        at 
org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    >        at 
org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    >        at 
org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]    >        at 
org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]    >        at 
org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]    >        at 
org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:186)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4901)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4377)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]    >        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> Nov 17, 2016 1:32:44 AM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge 
Thread #24,5,TGRP-TestBlockJoin]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: 
java.lang.OutOfMemoryError: Java heap space
   [junit4]   2>        at 
__randomizedtesting.SeedInfo.seed([8114F6C2AAD21A68]:0)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2>        at 
org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]   2>        at 
org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]   2>        at 
org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]   2>        at 
org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]   2>        at 
org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]   2>        at 
org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]   2>        at 
org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]   2>        at 
org.apache.lucene.codecs.simpletext.SimpleTextCompoundFormat.write(SimpleTextCompoundFormat.java:186)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4901)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4377)
   [junit4]   2>        at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3897)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]   2>        at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/join/test/J1/temp/lucene.search.join.TestBlockJoin_8114F6C2AAD21A68-001
   [junit4]   2> NOTE: test params are: codec=SimpleText, 
sim=RandomSimilarity(queryNorm=true,coord=no): {country=IB SPL-D1, 
qualification=DFI(Saturated), parent=LM Jelinek-Mercer(0.700000), 
docType=DFI(ChiSquared), skill=DFI(ChiSquared), childText=DFR I(F)2, 
parentText=LM Jelinek-Mercer(0.100000), child=IB LL-LZ(0.3)}, locale=de-LU, 
timezone=America/Panama
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=1,free=185888128,total=457179136
   [junit4]   2> NOTE: All tests run in this JVM: [TestBlockJoin]
   [junit4] Completed [6/6 (1!)] on J1 in 146.34s, 21 tests, 1 error <<< 
FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/join/test/temp/junit4-J1-20161117_063041_452.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/heapdumps/java_pid9184.hprof
 ...
   [junit4] Heap dump file created [435832165 bytes in 0.708 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 3921 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/init-core-data-001
   [junit4]   2> 1624591 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1624591 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: 
/lniqd/bf
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1624635 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1624644 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1624645 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1624662 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_37732_hdfs____.4a6p0/webapp
   [junit4]   2> 1625035 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37732
   [junit4]   2> 1625107 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1625108 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1625120 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_40358_datanode____7b8mbl/webapp
   [junit4]   2> 1625513 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40358
   [junit4]   2> 1625557 WARN  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1625558 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1625570 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_37886_datanode____z3yuyh/webapp
   [junit4]   2> 1625683 INFO  (IPC Server handler 3 on 41348) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-aef481a9-184d-4aa7-b6c7-dbbb3982b741 node 
DatanodeRegistration(127.0.0.1:55368, 
datanodeUuid=9ff2abbb-a864-4737-ba29-5bfb7b847797, infoPort=45909, 
infoSecurePort=0, ipcPort=33976, 
storageInfo=lv=-56;cid=testClusterID;nsid=606290088;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1625683 INFO  (IPC Server handler 3 on 41348) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-6d7cb9c6-821f-42e0-a230-e3fc8fff6eac node 
DatanodeRegistration(127.0.0.1:55368, 
datanodeUuid=9ff2abbb-a864-4737-ba29-5bfb7b847797, infoPort=45909, 
infoSecurePort=0, ipcPort=33976, 
storageInfo=lv=-56;cid=testClusterID;nsid=606290088;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1625996 INFO  
(SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[CB0A51AB95826B36]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37886
   [junit4]   2> 1626121 INFO  (IPC Server handler 2 on 41348) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-345ff4f9-57d6-4b45-837b-397634761d1f node 
DatanodeRegistration(127.0.0.1:46617, 
datanodeUuid=ad88d0e1-102a-46b3-83e8-39e2dbc6e66f, infoPort=37898, 
infoSecurePort=0, ipcPort=48336, 
storageInfo=lv=-56;cid=testClusterID;nsid=606290088;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1626121 INFO  (IPC Server handler 2 on 41348) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-45c84945-d8fb-4e28-91cd-a26655090373 node 
DatanodeRegistration(127.0.0.1:46617, 
datanodeUuid=ad88d0e1-102a-46b3-83e8-39e2dbc6e66f, infoPort=37898, 
infoSecurePort=0, ipcPort=48336, 
storageInfo=lv=-56;cid=testClusterID;nsid=606290088;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1626169 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1626169 INFO  (Thread-15606) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1626169 INFO  (Thread-15606) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1626269 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:52209
   [junit4]   2> 1626289 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1626291 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1626292 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1626294 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1626295 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1626296 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1626297 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1626298 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1626299 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1626300 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1626301 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1626404 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/control-001/cores/collection1
   [junit4]   2> 1626406 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1626407 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@6c3d6f18{/lniqd/bf,null,AVAILABLE}
   [junit4]   2> 1626407 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@3a300637{HTTP/1.1,[http/1.1]}{127.0.0.1:46145}
   [junit4]   2> 1626407 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.Server Started @1630752ms
   [junit4]   2> 1626407 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:41348/hdfs__localhost_41348__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001_tempDir-002_control_data,
 hostContext=/lniqd/bf, hostPort=46145, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/control-001/cores}
   [junit4]   2> 1626408 ERROR 
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1626408 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
6.4.0
   [junit4]   2> 1626408 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1626408 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1626408 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-17T08:27:23.934Z
   [junit4]   2> 1626413 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1626413 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/control-001/solr.xml
   [junit4]   2> 1626419 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1626420 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52209/solr
   [junit4]   2> 1626444 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.OverseerElectionContext I am going 
to be the leader 127.0.0.1:46145_lniqd%2Fbf
   [junit4]   2> 1626445 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.Overseer Overseer 
(id=96952073826861060-127.0.0.1:46145_lniqd%2Fbf-n_0000000000) starting
   [junit4]   2> 1626464 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.ZkController Register node as live 
in ZooKeeper:/live_nodes/127.0.0.1:46145_lniqd%2Fbf
   [junit4]   2> 1626471 INFO  
(zkCallback-2012-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 1626520 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/control-001/cores
   [junit4]   2> 1626521 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 1626523 INFO  
(OverseerStateUpdate-96952073826861060-127.0.0.1:46145_lniqd%2Fbf-n_0000000000) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.o.ReplicaMutator Assigning new node 
to shard shard=shard1
   [junit4]   2> 1627534 WARN  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection   x:collection1] 
o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use 
<mergePolicyFactory> instead.
   [junit4]   2> 1627535 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection   x:collection1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 1627549 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1627631 WARN  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] default search field in schema is text. 
WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1627632 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection   x:collection1] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1627648 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 1627648 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:41348/solr_hdfs_home
   [junit4]   2> 1627648 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication 
disabled
   [junit4]   2> 1627649 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/control-001/cores/collection1],
 
dataDir=[hdfs://localhost:41348/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 1627649 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b3e3678
   [junit4]   2> 1627649 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 1627657 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] 
with direct memory allocation set to [true]
   [junit4]   2> 1627657 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, 
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1627659 WARN  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will 
wait 5 seconds and try again.
   [junit4]   2> 1632660 WARN  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will 
wait 5 seconds and try again.
   [junit4]   2> 1637774 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1637775 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1637793 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1637799 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] 
with direct memory allocation set to [true]
   [junit4]   2> 1637799 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, 
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1637802 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1637803 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=27, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=29.0126953125, 
floorSegmentMB=2.0556640625, forceMergeDeletesPctAllowed=26.970422045950187, 
segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.8698983467050798
   [junit4]   2> 1637819 INFO  (IPC Server handler 6 on 41348) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:55368 is 
added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-45c84945-d8fb-4e28-91cd-a26655090373:NORMAL:127.0.0.1:46617|RBW],
 
ReplicaUC[[DISK]DS-aef481a9-184d-4aa7-b6c7-dbbb3982b741:NORMAL:127.0.0.1:55368|RBW]]}
 size 0
   [junit4]   2> 1637820 INFO  (IPC Server handler 7 on 41348) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46617 is 
added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-aef481a9-184d-4aa7-b6c7-dbbb3982b741:NORMAL:127.0.0.1:55368|RBW],
 
ReplicaUC[[DISK]DS-345ff4f9-57d6-4b45-837b-397634761d1f:NORMAL:127.0.0.1:46617|FINALIZED]]}
 size 0
   [junit4]   2> 1637826 WARN  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler 
{type = requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1637834 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1637834 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
numVersionBuckets=65536
   [junit4]   2> 1637835 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: 
tlogDfsReplication=2
   [junit4]   2> 1637843 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 
15000ms; 
   [junit4]   2> 1637843 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1637844 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@4c3e262a
   [junit4]   2> 1637851 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.SolrIndexSearcher Opening 
[Searcher@37432791[collection1] main]
   [junit4]   2> 1637852 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO 
with znodeBase: /configs/conf1
   [junit4]   2> 1637853 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1637853 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1637854 INFO  
(searcherExecutor-2865-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@37432791[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1637854 INFO  
(coreLoadExecutor-2864-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent 
updates, using new clock 1551233193481338880
   [junit4]   2> 1637861 INFO  
(coreZkRegister-2857-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to 
continue.
   [junit4]   2> 1637861 INFO  
(coreZkRegister-2857-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try 
and sync
   [junit4]   2> 1637861 INFO  
(coreZkRegister-2857-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:46145/lniqd/bf/collection1/
   [junit4]   2> 1637861 INFO  
(coreZkRegister-2857-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1637861 INFO  
(coreZkRegister-2857-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:46145/lniqd/bf/collection1/ has no replicas
   [junit4]   2> 1637866 INFO  
(coreZkRegister-2857-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:46145/lniqd/bf/collection1/ shard1
   [junit4]   2> 1638017 INFO  
(coreZkRegister-2857-thread-1-processing-n:127.0.0.1:46145_lniqd%2Fbf 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:46145_lniqd%2Fbf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1638028 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1638029 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 1638133 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-1-001/cores/collection1
   [junit4]   2> 1638134 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-1-001
   [junit4]   2> 1638134 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1638136 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@141d3ba5{/lniqd/bf,null,AVAILABLE}
   [junit4]   2> 1638136 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@67f7a700{HTTP/1.1,[http/1.1]}{127.0.0.1:56271}
   [junit4]   2> 1638136 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.Server Started @1642480ms
   [junit4]   2> 1638136 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:41348/hdfs__localhost_41348__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001_tempDir-002_jetty1,
 solrconfig=solrconfig.xml, hostContext=/lniqd/bf, hostPort=56271, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-1-001/cores}
   [junit4]   2> 1638137 ERROR 
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1638137 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
6.4.0
   [junit4]   2> 1638137 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1638137 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1638137 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-17T08:27:35.663Z
   [junit4]   2> 1638140 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1638140 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-1-001/solr.xml
   [junit4]   2> 1638146 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1638147 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52209/solr
   [junit4]   2> 1638156 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:56271_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (1)
   [junit4]   2> 1638161 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:56271_lniqd%2Fbf    ] o.a.s.c.ZkController Register node as live 
in ZooKeeper:/live_nodes/127.0.0.1:56271_lniqd%2Fbf
   [junit4]   2> 1638163 INFO  
(zkCallback-2012-thread-2-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (1) -> (2)
   [junit4]   2> 1638163 INFO  (zkCallback-2016-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1638164 INFO  
(zkCallback-2021-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (1) -> (2)
   [junit4]   2> 1638346 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:56271_lniqd%2Fbf    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-1-001/cores
   [junit4]   2> 1638346 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:56271_lniqd%2Fbf    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 1638349 INFO  
(OverseerStateUpdate-96952073826861060-127.0.0.1:46145_lniqd%2Fbf-n_0000000000) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.o.ReplicaMutator Assigning new node 
to shard shard=shard2
   [junit4]   2> 1639361 WARN  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1   x:collection1] o.a.s.c.Config 
Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> 
instead.
   [junit4]   2> 1639362 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 1639375 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1639438 WARN  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.s.IndexSchema [collection1] default search field in schema is text. 
WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1639456 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1639472 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection collection1
   [junit4]   2> 1639472 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:41348/solr_hdfs_home
   [junit4]   2> 1639472 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication 
disabled
   [junit4]   2> 1639472 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-1-001/cores/collection1],
 dataDir=[hdfs://localhost:41348/solr_hdfs_home/collection1/core_node1/data/]
   [junit4]   2> 1639472 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b3e3678
   [junit4]   2> 1639472 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
   [junit4]   2> 1639480 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] 
with direct memory allocation set to [true]
   [junit4]   2> 1639480 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, 
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1639483 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1639484 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/collection1/core_node1/data
   [junit4]   2> 1639503 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/collection1/core_node1/data/index
   [junit4]   2> 1639509 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] 
with direct memory allocation set to [true]
   [junit4]   2> 1639509 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, 
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1639512 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1639512 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=27, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=29.0126953125, 
floorSegmentMB=2.0556640625, forceMergeDeletesPctAllowed=26.970422045950187, 
segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.8698983467050798
   [junit4]   2> 1639524 INFO  (IPC Server handler 8 on 41348) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:55368 is 
added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-345ff4f9-57d6-4b45-837b-397634761d1f:NORMAL:127.0.0.1:46617|RBW],
 
ReplicaUC[[DISK]DS-6d7cb9c6-821f-42e0-a230-e3fc8fff6eac:NORMAL:127.0.0.1:55368|FINALIZED]]}
 size 0
   [junit4]   2> 1639525 INFO  (IPC Server handler 0 on 41348) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46617 is 
added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-6d7cb9c6-821f-42e0-a230-e3fc8fff6eac:NORMAL:127.0.0.1:55368|FINALIZED],
 
ReplicaUC[[DISK]DS-45c84945-d8fb-4e28-91cd-a26655090373:NORMAL:127.0.0.1:46617|FINALIZED]]}
 size 0
   [junit4]   2> 1639531 WARN  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler 
{type = requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1639551 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1639551 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
numVersionBuckets=65536
   [junit4]   2> 1639551 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: 
tlogDfsReplication=2
   [junit4]   2> 1639560 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 
15000ms; 
   [junit4]   2> 1639560 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1639562 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@1507e070
   [junit4]   2> 1639569 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.s.SolrIndexSearcher Opening 
[Searcher@29fa5c39[collection1] main]
   [junit4]   2> 1639570 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO 
with znodeBase: /configs/conf1
   [junit4]   2> 1639571 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1639571 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1639572 INFO  
(searcherExecutor-2876-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@29fa5c39[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1639572 INFO  
(coreLoadExecutor-2875-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent 
updates, using new clock 1551233195282792448
   [junit4]   2> 1639580 INFO  
(coreZkRegister-2870-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to 
continue.
   [junit4]   2> 1639580 INFO  
(coreZkRegister-2870-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try 
and sync
   [junit4]   2> 1639580 INFO  
(coreZkRegister-2870-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:56271/lniqd/bf/collection1/
   [junit4]   2> 1639580 INFO  
(coreZkRegister-2870-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1639580 INFO  
(coreZkRegister-2870-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:56271/lniqd/bf/collection1/ has no replicas
   [junit4]   2> 1639584 INFO  
(coreZkRegister-2870-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:56271/lniqd/bf/collection1/ shard2
   [junit4]   2> 1639737 INFO  
(coreZkRegister-2870-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf 
x:collection1 s:shard2 c:collection1 r:core_node1) 
[n:127.0.0.1:56271_lniqd%2Fbf c:collection1 s:shard2 r:core_node1 
x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1639946 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-2-001/cores/collection1
   [junit4]   2> 1639946 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-2-001
   [junit4]   2> 1639947 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1639949 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@18cf01f2{/lniqd/bf,null,AVAILABLE}
   [junit4]   2> 1639949 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@13caf387{HTTP/1.1,[http/1.1]}{127.0.0.1:52277}
   [junit4]   2> 1639949 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.e.j.s.Server Started @1644293ms
   [junit4]   2> 1639949 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:41348/hdfs__localhost_41348__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001_tempDir-002_jetty2,
 solrconfig=solrconfig.xml, hostContext=/lniqd/bf, hostPort=52277, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-2-001/cores}
   [junit4]   2> 1639949 ERROR 
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1639949 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
6.4.0
   [junit4]   2> 1639949 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1639950 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1639950 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-17T08:27:37.476Z
   [junit4]   2> 1639953 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1639953 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-2-001/solr.xml
   [junit4]   2> 1639959 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1639960 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52209/solr
   [junit4]   2> 1639969 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:52277_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (0) -> (2)
   [junit4]   2> 1639974 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:52277_lniqd%2Fbf    ] o.a.s.c.ZkController Register node as live 
in ZooKeeper:/live_nodes/127.0.0.1:52277_lniqd%2Fbf
   [junit4]   2> 1639976 INFO  
(zkCallback-2021-thread-1-processing-n:127.0.0.1:56271_lniqd%2Fbf) 
[n:127.0.0.1:56271_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (2) -> (3)
   [junit4]   2> 1639976 INFO  (zkCallback-2016-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1639976 INFO  
(zkCallback-2012-thread-2-processing-n:127.0.0.1:46145_lniqd%2Fbf) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (2) -> (3)
   [junit4]   2> 1639991 INFO  
(zkCallback-2027-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf    ] o.a.s.c.c.ZkStateReader Updated live nodes 
from ZooKeeper... (2) -> (3)
   [junit4]   2> 1640018 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:52277_lniqd%2Fbf    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-2-001/cores
   [junit4]   2> 1640018 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[CB0A51AB95826B36]) 
[n:127.0.0.1:52277_lniqd%2Fbf    ] o.a.s.c.CorePropertiesLocator Cores are: 
[collection1]
   [junit4]   2> 1640020 INFO  
(OverseerStateUpdate-96952073826861060-127.0.0.1:46145_lniqd%2Fbf-n_0000000000) 
[n:127.0.0.1:46145_lniqd%2Fbf    ] o.a.s.c.o.ReplicaMutator Assigning new node 
to shard shard=shard3
   [junit4]   2> 1641035 WARN  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1   x:collection1] o.a.s.c.Config 
Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> 
instead.
   [junit4]   2> 1641036 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 1641050 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1641125 WARN  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.s.IndexSchema [collection1] default search field in schema is text. 
WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1641126 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1641141 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection collection1
   [junit4]   2> 1641141 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:41348/solr_hdfs_home
   [junit4]   2> 1641141 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication 
disabled
   [junit4]   2> 1641141 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_CB0A51AB95826B36-001/shard-2-001/cores/collection1],
 dataDir=[hdfs://localhost:41348/solr_hdfs_home/collection1/core_node2/data/]
   [junit4]   2> 1641141 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b3e3678
   [junit4]   2> 1641142 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
   [junit4]   2> 1641149 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] 
with direct memory allocation set to [true]
   [junit4]   2> 1641149 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, 
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1641152 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1641153 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/collection1/core_node2/data
   [junit4]   2> 1641170 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:41348/solr_hdfs_home/collection1/core_node2/data/index
   [junit4]   2> 1641176 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] 
with direct memory allocation set to [true]
   [junit4]   2> 1641176 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, 
slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1641180 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1641181 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=27, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=29.0126953125, 
floorSegmentMB=2.0556640625, forceMergeDeletesPctAllowed=26.970422045950187, 
segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.8698983467050798
   [junit4]   2> 1641189 INFO  (IPC Server handler 3 on 41348) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:55368 is 
added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-345ff4f9-57d6-4b45-837b-397634761d1f:NORMAL:127.0.0.1:46617|RBW],
 
ReplicaUC[[DISK]DS-aef481a9-184d-4aa7-b6c7-dbbb3982b741:NORMAL:127.0.0.1:55368|FINALIZED]]}
 size 0
   [junit4]   2> 1641191 INFO  (IPC Server handler 6 on 41348) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46617 is 
added to blk_1073741827_1003 size 71
   [junit4]   2> 1641195 WARN  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler 
{type = requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1641219 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1641219 INFO  
(coreLoadExecutor-2886-thread-1-processing-n:127.0.0.1:52277_lniqd%2Fbf) 
[n:127.0.0.1:52277_lniqd%2Fbf c:collection1 s:shard3 r:core_node2 
x:collection1] o.a.s.u.Update

[...truncated too long message...]

cheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-6.4.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_102/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.4.0.jar
 com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20161117_080012_951.events
 
@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20161117_080012_951.suites
 -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from 
the forked process? This may indicate JVM crash or runner bugs.
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]     at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:773:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:709:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build.xml:252:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/common-build.xml:536:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:1443:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:984:
 At least one slave process threw an exception, first: Quit event not received 
from the forked process? This may indicate JVM crash or runner bugs.

Total time: 305 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to