Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/236/

3 tests failed.
FAILED:  
org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
        at 
__randomizedtesting.SeedInfo.seed([BE3B25C06784B95A:C5AFBF7FE54498ED]:0)
        at java.util.HashMap.newNode(HashMap.java:1742)
        at java.util.HashMap.putVal(HashMap.java:641)
        at java.util.HashMap.put(HashMap.java:611)
        at java.util.HashSet.add(HashSet.java:219)
        at 
org.apache.lucene.store.NRTCachingDirectory.listAll(NRTCachingDirectory.java:104)
        at 
org.apache.lucene.store.MockDirectoryWrapper.listAll(MockDirectoryWrapper.java:1037)
        at 
org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:57)
        at 
org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:428)
        at 
org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2239)
        at 
org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2181)
        at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1133)
        at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1175)
        at 
org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:301)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/6)={
   "replicationFactor":"4",   "shards":{     "shard1":{       
"range":"80000000-ffffffff",       "state":"active",       "replicas":{}},     
"shard2":{       "range":"0-7fffffff",       "state":"active",       
"replicas":{}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"3",   
"autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/6)={
  "replicationFactor":"4",
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"3",
  "autoAddReplicas":"false"}
        at 
__randomizedtesting.SeedInfo.seed([15CEDE5379D39D0D:5DBBAAE77FE0B298]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:265)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets

Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2

Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /collections/withconfigset2
        at 
__randomizedtesting.SeedInfo.seed([15CEDE5379D39D0D:38B091098EF72701]:0)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 993 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestIndexingSequenceNumbers 
-Dtests.method=testStressConcurrentCommit -Dtests.seed=BE3B25C06784B95A 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=hr -Dtests.timezone=America/Nipigon -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   1079s J0 | 
TestIndexingSequenceNumbers.testStressConcurrentCommit <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([BE3B25C06784B95A:C5AFBF7FE54498ED]:0)
   [junit4]    >        at java.util.HashMap.newNode(HashMap.java:1742)
   [junit4]    >        at java.util.HashMap.putVal(HashMap.java:641)
   [junit4]    >        at java.util.HashMap.put(HashMap.java:611)
   [junit4]    >        at java.util.HashSet.add(HashSet.java:219)
   [junit4]    >        at 
org.apache.lucene.store.NRTCachingDirectory.listAll(NRTCachingDirectory.java:104)
   [junit4]    >        at 
org.apache.lucene.store.MockDirectoryWrapper.listAll(MockDirectoryWrapper.java:1037)
   [junit4]    >        at 
org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:57)
   [junit4]    >        at 
org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:428)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2239)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2181)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1133)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1175)
   [junit4]    >        at 
org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:301)
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/core/test/J0/temp/lucene.index.TestIndexingSequenceNumbers_BE3B25C06784B95A-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {id=FST50}, 
docValues:{thread=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=2027, 
maxMBSortInHeap=6.545992085514463, 
sim=RandomSimilarity(queryNorm=false,coord=no): {id=DFR I(n)3(800.0)}, 
locale=hr, timezone=America/Nipigon
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=1,free=74458504,total=495976448
   [junit4]   2> NOTE: All tests run in this JVM: [TestSmallFloat, 
LimitedFiniteStringsIteratorTest, TestDirectMonotonic, TestWordlistLoader, 
TestCustomNorms, TestIndexWriterMergePolicy, TestFieldType, TestBooleanCoord, 
TestFixedBitDocIdSet, TestTwoPhaseCommitTool, TestTimSorter, 
TestIndexWriterOnVMError, TestDocValuesIndexing, TestNRTReaderWithThreads, 
TestOmitPositions, TestBytesRef, TestPhraseQuery, TestCharFilter, 
TestMultiPhraseQuery, TestIndexingSequenceNumbers]
   [junit4] Completed [220/442 (1!)] on J0 in 1304.15s, 8 tests, 1 error <<< 
FAILURES!

[...truncated 732 lines...]
   [junit4] JVM J0: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/core/test/temp/junit4-J0-20161224_112004_338.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/heapdumps/java_pid12132.hprof
 ...
   [junit4] Heap dump file created [555311893 bytes in 2.703 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 11184 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/init-core-data-001
   [junit4]   2> 4226026 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 4226027 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001
   [junit4]   2> 4226027 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4226031 INFO  (Thread-51930) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4226031 INFO  (Thread-51930) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 4226132 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.c.ZkTestServer start zk server on port:35605
   [junit4]   2> 4226141 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 4226143 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 4226144 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 4226145 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@b29b436{/solr,null,AVAILABLE}
   [junit4]   2> 4226147 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@f435327{SSL,[ssl, 
http/1.1]}{127.0.0.1:50303}
   [junit4]   2> 4226147 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.e.j.s.Server Started @4229925ms
   [junit4]   2> 4226147 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=50303}
   [junit4]   2> 4226147 ERROR (jetty-launcher-4817-thread-2) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 4226148 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
6.4.0
   [junit4]   2> 4226148 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4226148 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4226148 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-12-24T15:41:38.582Z
   [junit4]   2> 4226148 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@5b2fd475{/solr,null,AVAILABLE}
   [junit4]   2> 4226150 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@7ec131ee{SSL,[ssl, 
http/1.1]}{127.0.0.1:34894}
   [junit4]   2> 4226150 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.e.j.s.Server Started @4229928ms
   [junit4]   2> 4226150 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=34894}
   [junit4]   2> 4226151 ERROR (jetty-launcher-4817-thread-1) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 4226151 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
6.4.0
   [junit4]   2> 4226151 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4226151 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4226151 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-12-24T15:41:38.585Z
   [junit4]   2> 4226151 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@4e51bd22{/solr,null,AVAILABLE}
   [junit4]   2> 4226153 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 4226153 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@6243c3fe{SSL,[ssl, 
http/1.1]}{127.0.0.1:59127}
   [junit4]   2> 4226153 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.e.j.s.Server Started @4229931ms
   [junit4]   2> 4226153 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=59127}
   [junit4]   2> 4226153 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 4226158 ERROR (jetty-launcher-4817-thread-3) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 4226158 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
6.4.0
   [junit4]   2> 4226158 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 4226158 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4226158 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4226159 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-12-24T15:41:38.593Z
   [junit4]   2> 4226161 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 4226162 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4226165 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4226166 INFO  (jetty-launcher-4817-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35605/solr
   [junit4]   2> 4226167 INFO  (jetty-launcher-4817-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35605/solr
   [junit4]   2> 4226168 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@8bce84d{/solr,null,AVAILABLE}
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@77f282ad{SSL,[ssl, 
http/1.1]}{127.0.0.1:41982}
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.e.j.s.Server Started @4229949ms
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=41982}
   [junit4]   2> 4226171 ERROR (jetty-launcher-4817-thread-4) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35605/solr
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
6.4.0
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4226171 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4226172 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-12-24T15:41:38.606Z
   [junit4]   2> 4226172 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x159317e9d220004, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4226174 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x159317e9d220006, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4226175 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x159317e9d220005, likely client has closed socket
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4226177 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 4226182 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 4226183 INFO  (jetty-launcher-4817-thread-4) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35605/solr
   [junit4]   2> 4226212 INFO  (jetty-launcher-4817-thread-1) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:34894_solr
   [junit4]   2> 4226216 INFO  (jetty-launcher-4817-thread-3) 
[n:127.0.0.1:59127_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:59127_solr
   [junit4]   2> 4226216 INFO  (jetty-launcher-4817-thread-1) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.Overseer Overseer 
(id=97163286837067785-127.0.0.1:34894_solr-n_0000000000) starting
   [junit4]   2> 4226216 INFO  (jetty-launcher-4817-thread-4) 
[n:127.0.0.1:41982_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:41982_solr
   [junit4]   2> 4226217 INFO  (jetty-launcher-4817-thread-2) 
[n:127.0.0.1:50303_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:50303_solr
   [junit4]   2> 4226218 INFO  
(zkCallback-4833-thread-1-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 4226226 INFO  
(zkCallback-4831-thread-1-processing-n:127.0.0.1:50303_solr) 
[n:127.0.0.1:50303_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (2)
   [junit4]   2> 4226226 INFO  
(zkCallback-4834-thread-1-processing-n:127.0.0.1:59127_solr) 
[n:127.0.0.1:59127_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 4226226 INFO  
(zkCallback-4838-thread-1-processing-n:127.0.0.1:41982_solr) 
[n:127.0.0.1:41982_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 4226229 INFO  
(zkCallback-4831-thread-2-processing-n:127.0.0.1:50303_solr) 
[n:127.0.0.1:50303_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 4226229 INFO  
(zkCallback-4833-thread-1-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (2) -> (3)
   [junit4]   2> 4226253 INFO  (jetty-launcher-4817-thread-1) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:34894_solr
   [junit4]   2> 4226254 INFO  
(zkCallback-4834-thread-1-processing-n:127.0.0.1:59127_solr) 
[n:127.0.0.1:59127_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 4226254 INFO  
(zkCallback-4838-thread-1-processing-n:127.0.0.1:41982_solr) 
[n:127.0.0.1:41982_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 4226255 INFO  
(zkCallback-4831-thread-2-processing-n:127.0.0.1:50303_solr) 
[n:127.0.0.1:50303_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 4226258 INFO  (jetty-launcher-4817-thread-3) 
[n:127.0.0.1:59127_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001/node3/.
   [junit4]   2> 4226259 INFO  
(zkCallback-4833-thread-1-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 4226275 INFO  (jetty-launcher-4817-thread-4) 
[n:127.0.0.1:41982_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001/node4/.
   [junit4]   2> 4226308 INFO  (jetty-launcher-4817-thread-1) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001/node1/.
   [junit4]   2> 4226375 INFO  (jetty-launcher-4817-thread-2) 
[n:127.0.0.1:50303_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001/node2/.
   [junit4]   2> 4226380 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 4226381 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35605/solr ready
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 4226424 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 4226439 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4226441 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 4226455 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_49824_hdfs____.fx8dj5/webapp
   [junit4]   2> 4226871 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49824
   [junit4]   2> 4226959 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4226960 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 4226974 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_56716_datanode____.k5ff3y/webapp
   [junit4]   2> 4227379 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56716
   [junit4]   2> 4227432 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 4227436 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 4227461 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_36786_datanode____imdb89/webapp
   [junit4]   2> 4227518 INFO  (IPC Server handler 5 on 44124) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-9e1107aa-ad8e-4ffb-a42c-9cfc809dfd9a node 
DatanodeRegistration(127.0.0.1:49908, 
datanodeUuid=bb82dd9d-9a6a-4dc5-84d9-94b29565624d, infoPort=41892, 
infoSecurePort=0, ipcPort=36893, 
storageInfo=lv=-56;cid=testClusterID;nsid=1907367615;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 4227518 INFO  (IPC Server handler 5 on 44124) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-2f78cb15-eb71-4065-b8c9-cededc399b6b node 
DatanodeRegistration(127.0.0.1:49908, 
datanodeUuid=bb82dd9d-9a6a-4dc5-84d9-94b29565624d, infoPort=41892, 
infoSecurePort=0, ipcPort=36893, 
storageInfo=lv=-56;cid=testClusterID;nsid=1907367615;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 4227893 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36786
   [junit4]   2> 4227990 INFO  (IPC Server handler 4 on 44124) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-f0f5d9f0-51e8-41bc-a047-e62712ff4357 node 
DatanodeRegistration(127.0.0.1:39529, 
datanodeUuid=4e4ea2bc-a7dd-4b28-b295-ee86a86647cf, infoPort=41487, 
infoSecurePort=0, ipcPort=33128, 
storageInfo=lv=-56;cid=testClusterID;nsid=1907367615;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 4227990 INFO  (IPC Server handler 4 on 44124) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-6bf68e04-1e61-4cdb-a9f7-374233efbb14 node 
DatanodeRegistration(127.0.0.1:39529, 
datanodeUuid=4e4ea2bc-a7dd-4b28-b295-ee86a86647cf, infoPort=41487, 
infoSecurePort=0, ipcPort=33128, 
storageInfo=lv=-56;cid=testClusterID;nsid=1907367615;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 4228096 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreationAndDeletion
   [junit4]   2> 4228097 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 4228157 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=1&collection.configName=conf&name=created_and_deleted&action=CREATE&numShards=1&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 4228160 INFO  
(OverseerThreadFactory-10819-thread-1-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
created_and_deleted
   [junit4]   2> 4228325 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2
   [junit4]   2> 4228431 INFO  
(zkCallback-4833-thread-2-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/created_and_deleted/state.json] for collection 
[created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4229336 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 4229358 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.IndexSchema [created_and_deleted_shard1_replica1] Schema name=minimal
   [junit4]   2> 4229361 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 4229362 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.CoreContainer Creating SolrCore 'created_and_deleted_shard1_replica1' 
using configuration from collection created_and_deleted
   [junit4]   2> 4229369 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44124/data
   [junit4]   2> 4229369 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 4229369 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.SolrCore [[created_and_deleted_shard1_replica1] ] Opening new SolrCore 
at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001/node1/created_and_deleted_shard1_replica1],
 dataDir=[hdfs://localhost:44124/data/created_and_deleted/core_node1/data/]
   [junit4]   2> 4229369 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:44124/data/created_and_deleted/core_node1/data/snapshot_metadata
   [junit4]   2> 4229378 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 4229378 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 4229378 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 4229388 WARN  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds 
and try again.
   [junit4]   2> 4234560 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 4234561 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:44124/data/created_and_deleted/core_node1/data
   [junit4]   2> 4234577 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:44124/data/created_and_deleted/core_node1/data/index
   [junit4]   2> 4234584 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 4234584 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 4234584 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 4234600 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 4234627 INFO  (IPC Server handler 0 on 44124) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:49908 is 
added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-6bf68e04-1e61-4cdb-a9f7-374233efbb14:NORMAL:127.0.0.1:39529|RBW],
 
ReplicaUC[[DISK]DS-2f78cb15-eb71-4065-b8c9-cededc399b6b:NORMAL:127.0.0.1:49908|RBW]]}
 size 0
   [junit4]   2> 4234628 INFO  (IPC Server handler 2 on 44124) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39529 is 
added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-6bf68e04-1e61-4cdb-a9f7-374233efbb14:NORMAL:127.0.0.1:39529|RBW],
 
ReplicaUC[[DISK]DS-2f78cb15-eb71-4065-b8c9-cededc399b6b:NORMAL:127.0.0.1:49908|RBW]]}
 size 0
   [junit4]   2> 4234643 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4234643 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4234643 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 4234663 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4234663 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4234673 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1349bec5[created_and_deleted_shard1_replica1] main]
   [junit4]   2> 4234674 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf
   [junit4]   2> 4234674 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 4234674 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4234675 INFO  
(searcherExecutor-10828-thread-1-processing-n:127.0.0.1:34894_solr 
x:created_and_deleted_shard1_replica1 s:shard1 c:created_and_deleted) 
[n:127.0.0.1:34894_solr c:created_and_deleted s:shard1  
x:created_and_deleted_shard1_replica1] o.a.s.c.SolrCore 
[created_and_deleted_shard1_replica1] Registered new searcher 
Searcher@1349bec5[created_and_deleted_shard1_replica1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4234675 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1554612598455926784
   [junit4]   2> 4234681 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4234681 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4234681 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.SyncStrategy Sync replicas to 
https://127.0.0.1:34894/solr/created_and_deleted_shard1_replica1/
   [junit4]   2> 4234681 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4234681 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.SyncStrategy 
https://127.0.0.1:34894/solr/created_and_deleted_shard1_replica1/ has no 
replicas
   [junit4]   2> 4234683 INFO  
(zkCallback-4833-thread-3-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/created_and_deleted/state.json] for collection 
[created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4234686 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:34894/solr/created_and_deleted_shard1_replica1/ shard1
   [junit4]   2> 4234790 INFO  
(zkCallback-4833-thread-3-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/created_and_deleted/state.json] for collection 
[created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4234838 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4234840 INFO  (qtp170159908-70883) [n:127.0.0.1:34894_solr 
c:created_and_deleted s:shard1  x:created_and_deleted_shard1_replica1] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2}
 status=0 QTime=6514
   [junit4]   2> 4234842 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 4234942 INFO  
(zkCallback-4833-thread-2-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/created_and_deleted/state.json] for collection 
[created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4235843 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=1&collection.configName=conf&name=created_and_deleted&action=CREATE&numShards=1&wt=javabin&version=2}
 status=0 QTime=7685
   [junit4]   2> 4235845 INFO  (qtp1444828252-70897) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params 
action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4235846 INFO  (qtp1444828252-70897) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 4235847 INFO  (qtp1444828252-70889) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params 
name=created_and_deleted&action=DELETE&wt=javabin&version=2 and 
sendToOCPQueue=true
   [junit4]   2> 4235849 INFO  
(OverseerThreadFactory-10819-thread-2-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing 
Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 4235850 INFO  
(OverseerCollectionConfigSetProcessor-97163286837067785-127.0.0.1:34894_solr-n_0000000000)
 [n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 4235852 INFO  (qtp170159908-70884) [n:127.0.0.1:34894_solr    
] o.a.s.c.SolrCore [created_and_deleted_shard1_replica1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@a841a33
   [junit4]   2> 4235856 INFO  (qtp170159908-70884) [n:127.0.0.1:34894_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/created_and_deleted/core_node1/data/snapshot_metadata
   [junit4]   2> 4235857 INFO  (qtp170159908-70884) [n:127.0.0.1:34894_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/created_and_deleted/core_node1/data/index
   [junit4]   2> 4235857 INFO  (qtp170159908-70884) [n:127.0.0.1:34894_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/created_and_deleted/core_node1/data
   [junit4]   2> 4235866 INFO  (IPC Server handler 5 on 44124) [    ] 
BlockStateChange BLOCK* addToInvalidates: blk_1073741825_1001 127.0.0.1:39529 
127.0.0.1:49908 
   [junit4]   2> 4235868 INFO  (qtp170159908-70884) [n:127.0.0.1:34894_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={deleteInstanceDir=true&core=created_and_deleted_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2}
 status=0 QTime=17
   [junit4]   2> 4235944 INFO  
(org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@5546dd0)
 [    ] BlockStateChange BLOCK* BlockManager: ask 127.0.0.1:39529 to delete 
[blk_1073741825_1001]
   [junit4]   2> 4236574 INFO  (qtp1444828252-70889) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={name=created_and_deleted&action=DELETE&wt=javabin&version=2} status=0 
QTime=726
   [junit4]   2> 4236576 INFO  (qtp1444828252-70893) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params 
action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4236576 INFO  (qtp1444828252-70893) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 4236576 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testCreationAndDeletion
   [junit4]   2> 4236656 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
   [junit4]   2> 4236657 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 4236661 INFO  (qtp1444828252-70893) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 4236663 INFO  
(OverseerThreadFactory-10819-thread-3-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
collection
   [junit4]   2> 4236663 WARN  
(OverseerThreadFactory-10819-thread-3-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.CreateCollectionCmd Specified 
replicationFactor of 10 on collection collection is higher than or equal to the 
number of Solr instances currently live or live and part of your 
createNodeSet(4). It's unusual to run two replica of the same slice on the same 
Solr-instance.
   [junit4]   2> 4236663 INFO  
(OverseerCollectionConfigSetProcessor-97163286837067785-127.0.0.1:34894_solr-n_0000000000)
 [n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 4236664 ERROR 
(OverseerThreadFactory-10819-thread-3-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerCollectionMessageHandler 
Collection: collection operation: create 
failed:org.apache.solr.common.SolrException: Cannot create collection 
collection. Value of maxShardsPerNode is 1, and the number of nodes currently 
live or live and part of your createNodeSet is 4. This allows a maximum of 4 to 
be created. Value of numShards is 2 and value of replicationFactor is 10. This 
requires 20 shards to be created (higher than the allowed number)
   [junit4]   2>        at 
org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:155)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 4236666 INFO  (qtp1444828252-70893) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2}
 status=400 QTime=5
   [junit4]   2> 4236667 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
   [junit4]   2> 4236743 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting deleteCollectionOnlyInZk
   [junit4]   2> 4236745 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 4236750 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params 
name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4236753 INFO  
(OverseerCollectionConfigSetProcessor-97163286837067785-127.0.0.1:34894_solr-n_0000000000)
 [n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 4236766 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=15
   [junit4]   2> 4236768 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params 
action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4236768 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 4236769 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=1&collection.configName=conf&name=onlyinzk&action=CREATE&numShards=2&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 4236775 INFO  
(OverseerThreadFactory-10819-thread-5-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
onlyinzk
   [junit4]   2> 4236775 INFO  
(OverseerCollectionConfigSetProcessor-97163286837067785-127.0.0.1:34894_solr-n_0000000000)
 [n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 4236986 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2
   [junit4]   2> 4236986 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2
   [junit4]   2> 4237093 INFO  
(zkCallback-4833-thread-1-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - 
updating... (live nodes size: [4])
   [junit4]   2> 4237093 INFO  
(zkCallback-4834-thread-2-processing-n:127.0.0.1:59127_solr) 
[n:127.0.0.1:59127_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - 
updating... (live nodes size: [4])
   [junit4]   2> 4237996 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 6.4.0
   [junit4]   2> 4237996 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 6.4.0
   [junit4]   2> 4238005 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.IndexSchema 
[onlyinzk_shard2_replica1] Schema name=minimal
   [junit4]   2> 4238007 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.IndexSchema 
[onlyinzk_shard1_replica1] Schema name=minimal
   [junit4]   2> 4238008 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.IndexSchema Loaded 
schema minimal/1.1 with uniqueid field id
   [junit4]   2> 4238009 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.IndexSchema Loaded 
schema minimal/1.1 with uniqueid field id
   [junit4]   2> 4238009 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.CoreContainer Creating 
SolrCore 'onlyinzk_shard2_replica1' using configuration from collection onlyinzk
   [junit4]   2> 4238010 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.CoreContainer Creating 
SolrCore 'onlyinzk_shard1_replica1' using configuration from collection onlyinzk
   [junit4]   2> 4238010 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:44124/data
   [junit4]   2> 4238010 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
Solr Kerberos Authentication disabled
   [junit4]   2> 4238010 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SolrCore 
[[onlyinzk_shard2_replica1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001/node1/onlyinzk_shard2_replica1],
 dataDir=[hdfs://localhost:44124/data/onlyinzk/core_node1/data/]
   [junit4]   2> 4238010 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:44124/data
   [junit4]   2> 4238010 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
creating directory factory for path 
hdfs://localhost:44124/data/onlyinzk/core_node1/data/snapshot_metadata
   [junit4]   2> 4238010 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
Solr Kerberos Authentication disabled
   [junit4]   2> 4238010 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SolrCore 
[[onlyinzk_shard1_replica1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001/tempDir-001/node3/onlyinzk_shard1_replica1],
 dataDir=[hdfs://localhost:44124/data/onlyinzk/core_node2/data/]
   [junit4]   2> 4238010 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
creating directory factory for path 
hdfs://localhost:44124/data/onlyinzk/core_node2/data/snapshot_metadata
   [junit4]   2> 4238019 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 4238019 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 4238019 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
Block cache target memory usage, slab size of [16777216] will allocate [1] 
slabs and use ~[16777216] bytes
   [junit4]   2> 4238019 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
Block cache target memory usage, slab size of [16777216] will allocate [1] 
slabs and use ~[16777216] bytes
   [junit4]   2> 4238019 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
Creating new single instance HDFS BlockCache
   [junit4]   2> 4238019 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
Creating new single instance HDFS BlockCache
   [junit4]   2> 4238030 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.b.BlockDirectory Block 
cache on write is disabled
   [junit4]   2> 4238030 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.b.BlockDirectory Block 
cache on write is disabled
   [junit4]   2> 4238031 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
creating directory factory for path 
hdfs://localhost:44124/data/onlyinzk/core_node1/data
   [junit4]   2> 4238031 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
creating directory factory for path 
hdfs://localhost:44124/data/onlyinzk/core_node2/data
   [junit4]   2> 4238050 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
creating directory factory for path 
hdfs://localhost:44124/data/onlyinzk/core_node1/data/index
   [junit4]   2> 4238051 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
creating directory factory for path 
hdfs://localhost:44124/data/onlyinzk/core_node2/data/index
   [junit4]   2> 4238057 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 4238057 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
Block cache target memory usage, slab size of [16777216] will allocate [1] 
slabs and use ~[16777216] bytes
   [junit4]   2> 4238058 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.HdfsDirectoryFactory 
Creating new single instance HDFS BlockCache
   [junit4]   2> 4238058 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 4238058 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
Block cache target memory usage, slab size of [16777216] will allocate [1] 
slabs and use ~[16777216] bytes
   [junit4]   2> 4238058 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.HdfsDirectoryFactory 
Creating new single instance HDFS BlockCache
   [junit4]   2> 4238072 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.b.BlockDirectory Block 
cache on write is disabled
   [junit4]   2> 4238072 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.b.BlockDirectory Block 
cache on write is disabled
   [junit4]   2> 4238094 INFO  (IPC Server handler 4 on 44124) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:49908 is 
added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-6bf68e04-1e61-4cdb-a9f7-374233efbb14:NORMAL:127.0.0.1:39529|RBW],
 
ReplicaUC[[DISK]DS-9e1107aa-ad8e-4ffb-a42c-9cfc809dfd9a:NORMAL:127.0.0.1:49908|RBW]]}
 size 0
   [junit4]   2> 4238095 INFO  (IPC Server handler 2 on 44124) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39529 is 
added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-9e1107aa-ad8e-4ffb-a42c-9cfc809dfd9a:NORMAL:127.0.0.1:49908|RBW],
 
ReplicaUC[[DISK]DS-f0f5d9f0-51e8-41bc-a047-e62712ff4357:NORMAL:127.0.0.1:39529|FINALIZED]]}
 size 0
   [junit4]   2> 4238096 INFO  (IPC Server handler 8 on 44124) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:49908 is 
added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-6bf68e04-1e61-4cdb-a9f7-374233efbb14:NORMAL:127.0.0.1:39529|RBW],
 
ReplicaUC[[DISK]DS-2f78cb15-eb71-4065-b8c9-cededc399b6b:NORMAL:127.0.0.1:49908|RBW]]}
 size 0
   [junit4]   2> 4238097 INFO  (IPC Server handler 7 on 44124) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39529 is 
added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-6bf68e04-1e61-4cdb-a9f7-374233efbb14:NORMAL:127.0.0.1:39529|RBW],
 
ReplicaUC[[DISK]DS-2f78cb15-eb71-4065-b8c9-cededc399b6b:NORMAL:127.0.0.1:49908|RBW]]}
 size 0
   [junit4]   2> 4238121 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.UpdateHandler Using 
UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4238121 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.UpdateLog Initializing 
UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4238121 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.HdfsUpdateLog 
Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 4238127 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.UpdateHandler Using 
UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 4238127 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.UpdateLog Initializing 
UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 
maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4238127 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.HdfsUpdateLog 
Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 4238133 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.CommitTracker Hard 
AutoCommit: disabled
   [junit4]   2> 4238133 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.CommitTracker Soft 
AutoCommit: disabled
   [junit4]   2> 4238137 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.CommitTracker Hard 
AutoCommit: disabled
   [junit4]   2> 4238137 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.CommitTracker Soft 
AutoCommit: disabled
   [junit4]   2> 4238147 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.SolrIndexSearcher 
Opening [Searcher@6fd75977[onlyinzk_shard2_replica1] main]
   [junit4]   2> 4238148 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.r.ManagedResourceStorage 
Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 4238149 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.r.ManagedResourceStorage 
Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 4238149 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.h.ReplicationHandler 
Commits will be reserved for  10000
   [junit4]   2> 4238149 INFO  
(searcherExecutor-10833-thread-1-processing-n:127.0.0.1:34894_solr 
x:onlyinzk_shard2_replica1 s:shard2 c:onlyinzk) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SolrCore 
[onlyinzk_shard2_replica1] Registered new searcher 
Searcher@6fd75977[onlyinzk_shard2_replica1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4238149 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.u.UpdateLog Could not 
find max version in index or recent updates, using new clock 1554612602098679808
   [junit4]   2> 4238151 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.SolrIndexSearcher 
Opening [Searcher@3e391290[onlyinzk_shard1_replica1] main]
   [junit4]   2> 4238152 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.r.ManagedResourceStorage 
Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 4238153 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.r.ManagedResourceStorage 
Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 4238153 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.h.ReplicationHandler 
Commits will be reserved for  10000
   [junit4]   2> 4238153 INFO  
(searcherExecutor-10834-thread-1-processing-n:127.0.0.1:59127_solr 
x:onlyinzk_shard1_replica1 s:shard1 c:onlyinzk) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SolrCore 
[onlyinzk_shard1_replica1] Registered new searcher 
Searcher@3e391290[onlyinzk_shard1_replica1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4238153 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.u.UpdateLog Could not 
find max version in index or recent updates, using new clock 1554612602102874112
   [junit4]   2> 4238159 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4238159 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4238159 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy Sync 
replicas to https://127.0.0.1:34894/solr/onlyinzk_shard2_replica1/
   [junit4]   2> 4238159 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy Sync 
Success - now sync replicas to me
   [junit4]   2> 4238160 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy 
https://127.0.0.1:34894/solr/onlyinzk_shard2_replica1/ has no replicas
   [junit4]   2> 4238162 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4238162 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4238162 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy Sync 
replicas to https://127.0.0.1:59127/solr/onlyinzk_shard1_replica1/
   [junit4]   2> 4238162 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy Sync 
Success - now sync replicas to me
   [junit4]   2> 4238162 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy 
https://127.0.0.1:59127/solr/onlyinzk_shard1_replica1/ has no replicas
   [junit4]   2> 4238164 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:34894/solr/onlyinzk_shard2_replica1/ shard2
   [junit4]   2> 4238166 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:59127/solr/onlyinzk_shard1_replica1/ shard1
   [junit4]   2> 4238268 INFO  
(zkCallback-4833-thread-1-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - 
updating... (live nodes size: [4])
   [junit4]   2> 4238268 INFO  
(zkCallback-4834-thread-2-processing-n:127.0.0.1:59127_solr) 
[n:127.0.0.1:59127_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - 
updating... (live nodes size: [4])
   [junit4]   2> 4238315 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.c.ZkController I am the 
leader, no recovery necessary
   [junit4]   2> 4238317 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.c.ZkController I am the 
leader, no recovery necessary
   [junit4]   2> 4238317 INFO  (qtp170159908-70877) [n:127.0.0.1:34894_solr 
c:onlyinzk s:shard2  x:onlyinzk_shard2_replica1] o.a.s.s.HttpSolrCall [admin] 
webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2}
 status=0 QTime=1331
   [junit4]   2> 4238318 INFO  (qtp1444828252-70895) [n:127.0.0.1:59127_solr 
c:onlyinzk s:shard1  x:onlyinzk_shard1_replica1] o.a.s.s.HttpSolrCall [admin] 
webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2}
 status=0 QTime=1332
   [junit4]   2> 4238321 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 4238423 INFO  
(zkCallback-4833-thread-3-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - 
updating... (live nodes size: [4])
   [junit4]   2> 4238423 INFO  
(zkCallback-4834-thread-2-processing-n:127.0.0.1:59127_solr) 
[n:127.0.0.1:59127_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - 
updating... (live nodes size: [4])
   [junit4]   2> 4238778 INFO  
(OverseerCollectionConfigSetProcessor-97163286837067785-127.0.0.1:34894_solr-n_0000000000)
 [n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 4238944 INFO  
(org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@5546dd0)
 [    ] BlockStateChange BLOCK* BlockManager: ask 127.0.0.1:49908 to delete 
[blk_1073741825_1001]
   [junit4]   2> 4239321 INFO  (qtp1444828252-70896) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=1&collection.configName=conf&name=onlyinzk&action=CREATE&numShards=2&wt=javabin&version=2}
 status=0 QTime=2551
   [junit4]   2> 4239323 INFO  (qtp1444828252-70889) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params 
action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4239323 INFO  (qtp1444828252-70889) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 4239323 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending deleteCollectionOnlyInZk
   [junit4]   2> 4239401 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testMissingRequiredParameters
   [junit4]   2> 4239402 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[15CEDE5379D39D0D])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 4239405 INFO  (qtp1444828252-70889) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params 
name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4239407 INFO  
(OverseerThreadFactory-10819-thread-5-processing-n:127.0.0.1:34894_solr) 
[n:127.0.0.1:34894_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing 
Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 4239409 INFO  (qtp170159908-70880) [n:127.0.0.1:34894_solr    
] o.a.s.c.SolrCore [onlyinzk_shard2_replica1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@56b59170
   [junit4]   2> 4239409 INFO  (qtp1444828252-70897) [n:127.0.0.1:59127_solr    
] o.a.s.c.SolrCore [onlyinzk_shard1_replica1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@1533e029
   [junit4]   2> 4239415 INFO  (qtp1444828252-70897) [n:127.0.0.1:59127_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/onlyinzk/core_node2/data/snapshot_metadata
   [junit4]   2> 4239415 INFO  (qtp1444828252-70897) [n:127.0.0.1:59127_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/onlyinzk/core_node2/data
   [junit4]   2> 4239418 INFO  (qtp1444828252-70897) [n:127.0.0.1:59127_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/onlyinzk/core_node2/data/index
   [junit4]   2> 4239418 INFO  (qtp170159908-70880) [n:127.0.0.1:34894_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/onlyinzk/core_node1/data/snapshot_metadata
   [junit4]   2> 4239423 INFO  (qtp170159908-70880) [n:127.0.0.1:34894_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/onlyinzk/core_node1/data/index
   [junit4]   2> 4239423 INFO  (qtp170159908-70880) [n:127.0.0.1:34894_solr    
] o.a.s.s.h.HdfsDirectory Closing hdfs directory 
hdfs://localhost:44124/data/onlyinzk/core_node1/data
   [junit4]   2> 4239446 INFO  (IPC Server handler 3 on 44124) [    ] 
BlockStateChange BLOCK* addToInvalidates: blk_1073741826_1002 127.0.0.1:39529 
127.0.0.1:49908 
   [junit4]   2> 4239450 INFO  (qtp1444828252-70897) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={deleteInstanceDir=true&core=onlyinzk_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2}
 status=0 QTime=42
   [junit4]   2> 4239452 INFO  (IPC Server handler 0 on 44124) [    ] 
BlockStateChange BLOCK* addToInvalidates: blk_1073741827_1003 127.0.0.1:39529 
127.0.0.1:49908 
   [junit4]   2> 4239454 INFO  (qtp170159908-70880) [n:127.0.0.1:34894_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={deleteInstanceDir=true&core=onlyinzk_shard2_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2}
 status=0 QTime=45
   [junit4]   2> 4240158 INFO  (qtp1444828252-70889) [n:127.0.0.1:59127_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=753
   [junit4]   2> 4240160 INFO  (qtp1444828252-70894) [n:127.0.0.1:59127_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4240161 ERROR (qtp1444828252-70894) [n:127.0.0.1:59127_solr    
] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Missing 
required parameter: name
   [junit4]   2>        at 
org.apache.solr.common.params.RequiredSolrParams.getParams(RequiredSolrParams.java:90)
   [junit4]   2>        at 
org.apache.solr.common.params.SolrParams.getAll(SolrParams.java:461)
   [junit4]   2>        at 
org.apache.solr.common.params.SolrParams.getAll(SolrParams.java:478)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.lambda$static$0(CollectionsHandler.java:378)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.execute(CollectionsHandler.java:872)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:209)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:152)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:269)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2>        at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2>        at 
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4

[...truncated too long message...]

s.server.namenode.NameNode.stop(NameNode.java:872)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
   [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
   [junit4]   2>        ... 54 more
   [junit4]   2> 4387901 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35605 35605
   [junit4]   2> 4387901 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[15CEDE5379D39D0D]-worker) [    
] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_15CEDE5379D39D0D-001
   [junit4]   2> Dec 24, 2016 3:44:20 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 35 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62), 
sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=nl-NL, 
timezone=America/Montreal
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=2,free=315741392,total=514326528
   [junit4]   2> NOTE: All tests run in this JVM: 
[TestReversedWildcardFilterFactory, TestCloudDeleteByQuery, 
TestShardHandlerFactory, SolrPluginUtilsTest, TestCollectionAPI, 
BlobRepositoryCloudTest, PrimUtilsTest, TestSolrJ, DistributedQueueTest, 
AtomicUpdatesTest, TestSha256AuthenticationProvider, HardAutoCommitTest, 
TestSubQueryTransformerDistrib, TestSortByMinMaxFunction, 
TestLeaderInitiatedRecoveryThread, ShowFileRequestHandlerTest, 
TestGroupingSearch, TestExtendedDismaxParser, TestDFISimilarityFactory, 
TestLFUCache, TestReloadDeadlock, ResponseLogComponentTest, 
SaslZkACLProviderTest, DirectUpdateHandlerTest, TestAuthorizationFramework, 
CircularListTest, TestDistribIDF, SolrCoreCheckLockOnStartupTest, 
ConnectionReuseTest, HdfsBasicDistributedZk2Test, RecoveryZkTest, 
ManagedSchemaRoundRobinCloudTest, OpenExchangeRatesOrgProviderTest, 
OutOfBoxZkACLAndCredentialsProvidersTest, DirectUpdateHandlerOptimizeTest, 
QueryParsingTest, TestBulkSchemaConcurrent, DocValuesMultiTest, TestJsonFacets, 
ForceLeaderTest, HdfsChaosMonkeySafeLeaderTest, LukeRequestHandlerTest, 
DOMUtilTest, SuggesterTSTTest, MinimalSchemaTest, CoreAdminHandlerTest, 
HLLSerializationTest, TestFieldCacheSort, UtilsToolTest, HdfsDirectoryTest, 
TestSolrConfigHandlerCloud, TestSolrXml, TestIntervalFaceting, 
TestSolrQueryParser, DeleteInactiveReplicaTest, TestGraphTermsQParserPlugin, 
TestReload, TestShortCircuitedRequests, TestRestoreCore, NoCacheHeaderTest, 
DebugComponentTest, TestMaxScoreQueryParser, XmlUpdateRequestHandlerTest, 
OverseerTest, TestDFRSimilarityFactory, ConjunctionSolrSpellCheckerTest, 
TestUseDocValuesAsStored, TestRebalanceLeaders, 
TestOverriddenPrefixQueryForCustomFieldType, TestIndexingPerformance, 
DistributedQueryComponentOptimizationTest, CurrencyFieldXmlFileTest, 
SolrTestCaseJ4Test, TestNumericTerms64, DistributedFacetPivotLargeTest, 
TestZkChroot, SharedFSAutoReplicaFailoverUtilsTest, TestRestManager, 
TestRTimerTree, TestSolrDeletionPolicy1, DateRangeFieldTest, 
TestCloudSchemaless, TestFieldCache, CoreSorterTest, 
FullSolrCloudDistribCmdsTest, CdcrBootstrapTest, CollectionsAPISolrJTest, 
IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestPostingsSolrHighlighter, 
TestDynamicLoading, TermVectorComponentDistributedTest, TestLRUStatsCache, 
AnalysisErrorHandlingTest, TestCustomSort, HdfsUnloadDistributedZkTest, 
TestFieldTypeCollectionResource, SolrCloudExampleTest, 
TestFieldCacheWithThreads, PeerSyncWithIndexFingerprintCachingTest, 
DistribCursorPagingTest, TestConfigSets, TestGraphMLResponseWriter, 
PrimitiveFieldTypeTest, LeaderFailoverAfterPartitionTest, BadCopyFieldTest, 
TestOnReconnectListenerSupport, BasicDistributedZkTest, 
CollectionsAPIDistributedZkTest, TestReplicationHandler, SimpleFacetsTest, 
TestSolr4Spatial, SolrCoreTest, SpellCheckComponentTest, 
QueryElevationComponentTest, PeerSyncTest, BadIndexSchemaTest, 
ConvertedLegacyTest, TestFiltering, TestBadConfig, 
DistributedQueryElevationComponentTest, SpellCheckCollatorTest, SuggesterTest, 
TestTrie, SpatialFilterTest, SuggesterWFSTTest, PolyFieldTest, 
WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, 
TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, 
DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, 
SortByFunctionTest, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, 
DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, TestQueryTypes, 
RequestHandlersTest, PathHierarchyTokenizerFactoryTest, 
FieldAnalysisRequestHandlerTest, LoggingHandlerTest, TestJmxIntegration, 
UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, 
TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, 
PingRequestHandlerTest, TestBinaryResponseWriter, SOLR749Test, 
TestQuerySenderListener, AlternateDirectoryTest, TestQuerySenderNoQuery, 
SolrIndexConfigTest, SampleTest, TestBinaryField, TestFuzzyAnalyzedSuggestions, 
ExternalFileFieldSortTest, TestLuceneMatchVersion, TestPhraseSuggestions, 
TestXIncludeConfig, TestSweetSpotSimilarityFactory, 
TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, 
TestJmxMonitoredMap, PluginInfoTest, TestSystemIdResolver, 
TestSuggestSpellingConverter, DistributedMLTComponentTest, 
TestHighlightDedupGrouping, TestTolerantSearch, ActionThrottleTest, AssignTest, 
CdcrReplicationHandlerTest, ConfigSetsAPITest, ConnectionManagerTest, 
CreateCollectionCleanupTest, DeleteNodeTest, DistribJoinFromCollectionTest, 
DistributedVersionInfoTest, LeaderInitiatedRecoveryOnCommitTest, 
LeaderInitiatedRecoveryOnShardRestartTest, OverseerModifyCollectionTest, 
OverseerStatusTest, OverseerTaskQueueTest, RemoteQueryErrorTest, 
ReplicationFactorTest, RestartWhileUpdatingTest, ShardSplitTest, 
TestTolerantUpdateProcessorRandomCloud, TriLevelCompositeIdRoutingTest, 
HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [533/665 (1!)] on J2 in 171.02s, 20 tests, 1 failure, 1 
error <<< FAILURES!

[...truncated 12046 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:783:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:727:
 Some of the tests produced a heap dump, but did not fail. Maybe a suppressed 
OutOfMemoryError? Dumps created:
* java_pid12132.hprof

Total time: 279 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to