Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/7/
4 tests failed. FAILED: org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr Error Message: Error from server at http://127.0.0.1:41477/jv: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 401 </title> </head> <body> <h2>HTTP ERROR: 401</h2> <p>Problem accessing /jv/admin/collections. Reason: <pre> Authentication required</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41477/jv: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 401 </title> </head> <body> <h2>HTTP ERROR: 401</h2> <p>Problem accessing /jv/admin/collections. Reason: <pre> Authentication required</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> at __randomizedtesting.SeedInfo.seed([CEC58863C9722182:65DC5EA48313E1B5]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152) at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:157) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithKerberos Error Message: Some resources were not closed, shutdown, or released. Stack Trace: java.lang.AssertionError: Some resources were not closed, shutdown, or released. at __randomizedtesting.SeedInfo.seed([CEC58863C9722182]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234) at sun.reflect.GeneratedMethodAccessor52.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithKerberos Error Message: 5 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithKerberos: 1) Thread[id=11044, name=zkCallback-1760-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=11080, name=zkCallback-1760-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=11043, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 4) Thread[id=11042, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-SendThread(127.0.0.1:57610), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 5) Thread[id=11045, name=zkCallback-1760-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithKerberos: 1) Thread[id=11044, name=zkCallback-1760-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=11080, name=zkCallback-1760-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=11043, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 4) Thread[id=11042, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-SendThread(127.0.0.1:57610), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 5) Thread[id=11045, name=zkCallback-1760-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at __randomizedtesting.SeedInfo.seed([CEC58863C9722182]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithKerberos Error Message: There are still zombie threads that couldn't be terminated: 1) Thread[id=11044, name=zkCallback-1760-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=11080, name=zkCallback-1760-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=11042, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-SendThread(127.0.0.1:57610), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 4) Thread[id=11045, name=zkCallback-1760-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: 1) Thread[id=11044, name=zkCallback-1760-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=11080, name=zkCallback-1760-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=11042, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-SendThread(127.0.0.1:57610), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 4) Thread[id=11045, name=zkCallback-1760-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at __randomizedtesting.SeedInfo.seed([CEC58863C9722182]:0) Build Log: [...truncated 20811 lines...] [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithKerberos [junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/init-core-data-001 [junit4] 2> 1635349 T10881 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /jv/ [junit4] 2> 1640266 T10881 oadsc.DefaultDirectoryService.showSecurityWarnings WARN You didn't change the admin password of directory service instance 'DefaultKrbServer'. Please update the admin password as soon as possible to prevent a possible security breach. [junit4] 2> 1640958 T10881 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 2> 1640967 T10890 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 1640968 T10890 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 1641068 T10881 oasc.ZkTestServer.run start zk server on port:57610 [junit4] 2> 1641077 T10897 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1641088 T10901 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1641093 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 1641095 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 1641097 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1641099 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 1641101 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 1641104 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 1641106 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 1641107 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 1641109 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1641111 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 1641113 T10881 oasc.AbstractZkTestCase.putConfig put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 1641124 T10905 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1641432 T10881 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1 [junit4] 2> 1641434 T10881 oejs.Server.doStart jetty-9.2.10.v20150310 [junit4] 2> 1641435 T10881 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@36c2a8b{/jv,null,AVAILABLE} [junit4] 2> 1641436 T10881 oejs.AbstractConnector.doStart Started ServerConnector@edb95cf{HTTP/1.1}{127.0.0.1:41477} [junit4] 2> 1641436 T10881 oejs.Server.doStart Started @1643600ms [junit4] 2> 1641436 T10881 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-002/control/data, hostContext=/jv, hostPort=41477, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores} [junit4] 2> 1641437 T10881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1641437 T10881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/' [junit4] 2> 1641451 T10881 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/solr.xml [junit4] 2> 1641455 T10881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores [junit4] 2> 1641456 T10881 oasc.CoreContainer.<init> New CoreContainer 1591338152 [junit4] 2> 1641456 T10881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/] [junit4] 2> 1641456 T10881 oasc.CoreContainer.load loading shared library: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/lib [junit4] 2> 1641457 T10881 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/lib). [junit4] 2> 1641464 T10881 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 1641465 T10881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1641466 T10881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1641466 T10881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1641466 T10881 oasc.CoreContainer.load Node Name: 127.0.0.1 [junit4] 2> 1641466 T10881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57610/solr [junit4] 2> 1641467 T10881 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1641468 T10920 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1641470 T10924 n:127.0.0.1:41477_jv oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1641483 T10881 n:127.0.0.1:41477_jv oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41477_jv [junit4] 2> 1641486 T10881 n:127.0.0.1:41477_jv oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1641488 T10881 n:127.0.0.1:41477_jv oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:41477_jv [junit4] 2> 1641489 T10881 n:127.0.0.1:41477_jv oasc.Overseer.start Overseer (id=93872981088665604-127.0.0.1:41477_jv-n_0000000000) starting [junit4] 2> 1641495 T10881 n:127.0.0.1:41477_jv oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 1641496 T10927 n:127.0.0.1:41477_jv oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 1641496 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 1641500 T10881 n:127.0.0.1:41477_jv oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin [junit4] 2> 1641511 T10881 n:127.0.0.1:41477_jv oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT} [junit4] 2> 1641533 T10881 n:127.0.0.1:41477_jv oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler. [junit4] 2> 1641534 T10881 n:127.0.0.1:41477_jv oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@35cded9b [junit4] 2> 1641534 T10881 n:127.0.0.1:41477_jv oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1641564 T10881 n:127.0.0.1:41477_jv oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@35cded9b [junit4] 2> 1641565 T10881 n:127.0.0.1:41477_jv oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1641566 T10881 n:127.0.0.1:41477_jv oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores [junit4] 2> 1641567 T10881 n:127.0.0.1:41477_jv oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 1641567 T10881 n:127.0.0.1:41477_jv oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/ [junit4] 2> 1641568 T10881 n:127.0.0.1:41477_jv oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1641572 T10930 n:127.0.0.1:41477_jv c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection [junit4] 2> 1641572 T10930 n:127.0.0.1:41477_jv c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1641573 T10924 n:127.0.0.1:41477_jv oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1641573 T10930 n:127.0.0.1:41477_jv oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1641573 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41477/jv";, [junit4] 2> "node_name":"127.0.0.1:41477_jv", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 0 [junit4] 2> 1641574 T10926 n:127.0.0.1:41477_jv oasco.ReplicaMutator.updateState Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41477/jv";, [junit4] 2> "node_name":"127.0.0.1:41477_jv", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 1641574 T10926 n:127.0.0.1:41477_jv oasco.ClusterStateMutator.createCollection building a new cName: control_collection [junit4] 2> 1641574 T10926 n:127.0.0.1:41477_jv oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1 [junit4] 2> 1642573 T10930 n:127.0.0.1:41477_jv oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1642574 T10930 n:127.0.0.1:41477_jv oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection [junit4] 2> 1642574 T10930 n:127.0.0.1:41477_jv oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1642575 T10930 n:127.0.0.1:41477_jv oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/' [junit4] 2> 1642591 T10930 n:127.0.0.1:41477_jv oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1642595 T10930 n:127.0.0.1:41477_jv oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1642599 T10930 n:127.0.0.1:41477_jv oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1642607 T10930 n:127.0.0.1:41477_jv oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1642608 T10930 n:127.0.0.1:41477_jv oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1642612 T10930 n:127.0.0.1:41477_jv oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1642712 T10930 n:127.0.0.1:41477_jv oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1642718 T10930 n:127.0.0.1:41477_jv oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1642719 T10930 n:127.0.0.1:41477_jv oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1642727 T10930 n:127.0.0.1:41477_jv oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1642729 T10930 n:127.0.0.1:41477_jv oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1642731 T10930 n:127.0.0.1:41477_jv oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1642731 T10930 n:127.0.0.1:41477_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1642731 T10930 n:127.0.0.1:41477_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1642732 T10930 n:127.0.0.1:41477_jv oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1642732 T10930 n:127.0.0.1:41477_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1642732 T10930 n:127.0.0.1:41477_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1642733 T10930 n:127.0.0.1:41477_jv oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 1642733 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1642733 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/], dataDir=[null] [junit4] 2> 1642733 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4fcd2d1d [junit4] 2> 1642734 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/data [junit4] 2> 1642734 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/data/index/ [junit4] 2> 1642734 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1642735 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/control-001/cores/collection1/data/index [junit4] 2> 1642736 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=41.1806640625, floorSegmentMB=1.806640625, forceMergeDeletesPctAllowed=12.035944439818008, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 1642744 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6273c1af),segFN=segments_1,generation=1} [junit4] 2> 1642744 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1642751 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1642752 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1642752 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1642752 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1642752 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1642753 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1642753 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1642753 T10930 n:127.0.0.1:41477_jv x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1642753 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1642756 T10930 n:127.0.0.1:41477_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1642757 T10930 n:127.0.0.1:41477_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1642758 T10930 n:127.0.0.1:41477_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1642759 T10930 n:127.0.0.1:41477_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1642761 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file [junit4] 2> 1642762 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1642763 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog [junit4] 2> 1642763 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 1642764 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1642764 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1642765 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.1103515625, noCFSRatio=0.2491958114430644] [junit4] 2> 1642766 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6273c1af),segFN=segments_1,generation=1} [junit4] 2> 1642766 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1642767 T10930 n:127.0.0.1:41477_jv x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@5d35578e[collection1] main [junit4] 2> 1642767 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field [junit4] 2> 1642767 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index [junit4] 2> 1642767 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index [junit4] 2> 1642767 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501967699298746368 [junit4] 2> 1642770 T10930 n:127.0.0.1:41477_jv x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 3 ms to seed version buckets with highest version 1501967699298746368 [junit4] 2> 1642771 T10930 n:127.0.0.1:41477_jv x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1642771 T10930 n:127.0.0.1:41477_jv x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1642771 T10930 n:127.0.0.1:41477_jv x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1642771 T10930 n:127.0.0.1:41477_jv x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1642772 T10930 n:127.0.0.1:41477_jv x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1642772 T10930 n:127.0.0.1:41477_jv x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1642772 T10930 n:127.0.0.1:41477_jv x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1642772 T10930 n:127.0.0.1:41477_jv x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1642773 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1 [junit4] 2> 1642773 T10931 n:127.0.0.1:41477_jv x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d35578e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1642773 T10930 n:127.0.0.1:41477_jv x:collection1 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1642774 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41477/jv collection:control_collection shard:shard1 [junit4] 2> 1642774 T10881 n:127.0.0.1:41477_jv oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1642775 T10881 n:127.0.0.1:41477_jv oass.SolrDispatchFilter.init user.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1 [junit4] 2> 1642775 T10881 n:127.0.0.1:41477_jv oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1642776 T10881 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1642777 T10881 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1642782 T10937 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1642783 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 1642787 T10924 n:127.0.0.1:41477_jv oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1642788 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 1642788 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C1463 name=collection1 org.apache.solr.core.SolrCore@2b1c6732 url=http://127.0.0.1:41477/jv/collection1 node=127.0.0.1:41477_jv C1463_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:41477/jv, node_name=127.0.0.1:41477_jv, state=down} [junit4] 2> 1642788 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 C1463 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41477/jv/collection1/ [junit4] 2> 1642788 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 C1463 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 1642788 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 C1463 oasc.SyncStrategy.syncToMe http://127.0.0.1:41477/jv/collection1/ has no replicas [junit4] 2> 1642788 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41477/jv/collection1/ shard1 [junit4] 2> 1642789 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection"} current state version: 1 [junit4] 2> 1642791 T10881 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 1642795 T10924 n:127.0.0.1:41477_jv oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1642796 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "base_url":"http://127.0.0.1:41477/jv";, [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 1 [junit4] 2> 1642943 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:41477/jv/collection1/ and leader is http://127.0.0.1:41477/jv/collection1/ [junit4] 2> 1642943 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41477/jv [junit4] 2> 1642943 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 1642944 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection [junit4] 2> 1642944 T10934 n:127.0.0.1:41477_jv c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1642945 T10924 n:127.0.0.1:41477_jv oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1642946 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41477/jv";, [junit4] 2> "node_name":"127.0.0.1:41477_jv", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 1642946 T10926 n:127.0.0.1:41477_jv oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41477/jv";, [junit4] 2> "node_name":"127.0.0.1:41477_jv", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 1643085 T10881 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1 [junit4] 2> 1643086 T10881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001 [junit4] 2> 1643087 T10881 oejs.Server.doStart jetty-9.2.10.v20150310 [junit4] 2> 1643089 T10881 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@4e1e8ebc{/jv,null,AVAILABLE} [junit4] 2> 1643090 T10881 oejs.AbstractConnector.doStart Started ServerConnector@3d324caf{HTTP/1.1}{127.0.0.1:34557} [junit4] 2> 1643090 T10881 oejs.Server.doStart Started @1645254ms [junit4] 2> 1643090 T10881 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-002/jetty1, solrconfig=solrconfig.xml, hostContext=/jv, hostPort=34557, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores} [junit4] 2> 1643091 T10881 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1643091 T10881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1643091 T10881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/' [junit4] 2> 1643105 T10881 oasc.SolrXmlConfig.fromFile Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/solr.xml [junit4] 2> 1643110 T10881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores [junit4] 2> 1643110 T10881 oasc.CoreContainer.<init> New CoreContainer 1091447886 [junit4] 2> 1643111 T10881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/] [junit4] 2> 1643111 T10881 oasc.CoreContainer.load loading shared library: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/lib [junit4] 2> 1643111 T10881 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/lib). [junit4] 2> 1643118 T10881 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 1643119 T10881 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1643120 T10881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1643120 T10881 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1643121 T10881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1643121 T10881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1643121 T10881 oasc.CoreContainer.load Node Name: 127.0.0.1 [junit4] 2> 1643121 T10881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57610/solr [junit4] 2> 1643122 T10881 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1643123 T10952 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1643126 T10955 n:127.0.0.1:34557_jv oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1644134 T10881 n:127.0.0.1:34557_jv oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34557_jv [junit4] 2> 1644138 T10881 n:127.0.0.1:34557_jv oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1644140 T10881 n:127.0.0.1:34557_jv oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin [junit4] 2> 1644140 T10881 n:127.0.0.1:34557_jv oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT} [junit4] 2> 1644142 T10881 n:127.0.0.1:34557_jv oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler. [junit4] 2> 1644143 T10881 n:127.0.0.1:34557_jv oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5966fd90 [junit4] 2> 1644143 T10881 n:127.0.0.1:34557_jv oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1644143 T10881 n:127.0.0.1:34557_jv oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5966fd90 [junit4] 2> 1644144 T10881 n:127.0.0.1:34557_jv oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1644145 T10881 n:127.0.0.1:34557_jv oasc.CorePropertiesLocator.discover Looking for core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores [junit4] 2> 1644146 T10881 n:127.0.0.1:34557_jv oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 1644146 T10881 n:127.0.0.1:34557_jv oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/ [junit4] 2> 1644147 T10881 n:127.0.0.1:34557_jv oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1644147 T10957 n:127.0.0.1:34557_jv c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 1644148 T10957 n:127.0.0.1:34557_jv c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1644148 T10924 n:127.0.0.1:41477_jv oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1644148 T10957 n:127.0.0.1:34557_jv oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1644149 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:34557/jv";, [junit4] 2> "node_name":"127.0.0.1:34557_jv", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 3 [junit4] 2> 1644150 T10926 n:127.0.0.1:41477_jv oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:34557/jv";, [junit4] 2> "node_name":"127.0.0.1:34557_jv", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1644150 T10926 n:127.0.0.1:41477_jv oasco.ClusterStateMutator.createCollection building a new cName: collection1 [junit4] 2> 1644150 T10926 n:127.0.0.1:41477_jv oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2 [junit4] 2> 1645149 T10957 n:127.0.0.1:34557_jv oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1645149 T10957 n:127.0.0.1:34557_jv oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 1645150 T10957 n:127.0.0.1:34557_jv oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1645151 T10957 n:127.0.0.1:34557_jv oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/' [junit4] 2> 1645166 T10957 n:127.0.0.1:34557_jv oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1645171 T10957 n:127.0.0.1:34557_jv oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1645175 T10957 n:127.0.0.1:34557_jv oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1645184 T10957 n:127.0.0.1:34557_jv oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1645185 T10957 n:127.0.0.1:34557_jv oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1645190 T10957 n:127.0.0.1:34557_jv oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1645305 T10957 n:127.0.0.1:34557_jv oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1645313 T10957 n:127.0.0.1:34557_jv oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1645314 T10957 n:127.0.0.1:34557_jv oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1645335 T10957 n:127.0.0.1:34557_jv oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1645338 T10957 n:127.0.0.1:34557_jv oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1645339 T10957 n:127.0.0.1:34557_jv oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1645343 T10957 n:127.0.0.1:34557_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1645344 T10957 n:127.0.0.1:34557_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1645344 T10957 n:127.0.0.1:34557_jv oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1645346 T10957 n:127.0.0.1:34557_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1645346 T10957 n:127.0.0.1:34557_jv oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1645346 T10957 n:127.0.0.1:34557_jv oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 1645346 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1645347 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/], dataDir=[null] [junit4] 2> 1645347 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4fcd2d1d [junit4] 2> 1645348 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/data [junit4] 2> 1645348 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/data/index/ [junit4] 2> 1645348 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1645349 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.CachingDirectoryFactory.get return new directory for /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/shard-1-001/cores/collection1/data/index [junit4] 2> 1645349 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=41.1806640625, floorSegmentMB=1.806640625, forceMergeDeletesPctAllowed=12.035944439818008, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 1645369 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6273c1af),segFN=segments_1,generation=1} [junit4] 2> 1645370 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1645374 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1645375 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1645375 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1645375 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1645375 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1645375 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1645376 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1645376 T10957 n:127.0.0.1:34557_jv x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1645376 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1645378 T10957 n:127.0.0.1:34557_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1645380 T10957 n:127.0.0.1:34557_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1645381 T10957 n:127.0.0.1:34557_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1645382 T10957 n:127.0.0.1:34557_jv x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1645389 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file [junit4] 2> 1645390 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1645392 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog [junit4] 2> 1645392 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 1645393 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1645393 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1645400 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.1103515625, noCFSRatio=0.2491958114430644] [junit4] 2> 1645402 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6273c1af),segFN=segments_1,generation=1} [junit4] 2> 1645402 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1645402 T10957 n:127.0.0.1:34557_jv x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@3928bacb[collection1] main [junit4] 2> 1645403 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field [junit4] 2> 1645403 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index [junit4] 2> 1645403 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.VersionInfo.getMaxVersionFromIndex WARN No terms found for _version_, cannot seed version bucket highest value from index [junit4] 2> 1645403 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion WARN Could not find max version in index or recent updates, using new clock 1501967702062792704 [junit4] 2> 1645406 T10957 n:127.0.0.1:34557_jv x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 3 ms to seed version buckets with highest version 1501967702062792704 [junit4] 2> 1645407 T10957 n:127.0.0.1:34557_jv x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1645408 T10957 n:127.0.0.1:34557_jv x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1645408 T10957 n:127.0.0.1:34557_jv x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1645409 T10957 n:127.0.0.1:34557_jv x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1645409 T10957 n:127.0.0.1:34557_jv x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1645410 T10957 n:127.0.0.1:34557_jv x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1645410 T10957 n:127.0.0.1:34557_jv x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1645410 T10957 n:127.0.0.1:34557_jv x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1645411 T10958 n:127.0.0.1:34557_jv x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3928bacb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1645411 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1 [junit4] 2> 1645412 T10957 n:127.0.0.1:34557_jv x:collection1 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1645413 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34557/jv collection:collection1 shard:shard2 [junit4] 2> 1645413 T10881 n:127.0.0.1:34557_jv oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1645414 T10881 n:127.0.0.1:34557_jv oass.SolrDispatchFilter.init user.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1 [junit4] 2> 1645414 T10881 n:127.0.0.1:34557_jv oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1645415 T10881 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithKerberos CEC58863C9722182-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1645420 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2 [junit4] 2> 1645421 T10924 n:127.0.0.1:41477_jv oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1645422 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 1645422 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> 1645422 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1"} current state version: 4 [junit4] 2> ASYNC NEW_CORE C1464 name=collection1 org.apache.solr.core.SolrCore@75f86748 url=http://127.0.0.1:34557/jv/collection1 node=127.0.0.1:34557_jv C1464_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:34557/jv, node_name=127.0.0.1:34557_jv, state=down} [junit4] 2> 1645422 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 C1464 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34557/jv/collection1/ [junit4] 2> 1645423 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 C1464 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 1645423 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 C1464 oasc.SyncStrategy.syncToMe http://127.0.0.1:34557/jv/collection1/ has no replicas [junit4] 2> 1645423 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34557/jv/collection1/ shard2 [junit4] 2> 1645427 T10924 n:127.0.0.1:41477_jv oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1645427 T10926 n:127.0.0.1:41477_jv oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "base_url":"http://127.0.0.1:34557/jv";, [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 4 [junit4] 2> 1645578 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collection1 oasc.ZkController.register We are http://127.0.0.1:34557/jv/collection1/ and leader is http://127.0.0.1:34557/jv/collection1/ [junit4] 2> 1645579 T10961 n:127.0.0.1:34557_jv c:collection1 s:shard2 x:collect [...truncated too long message...] unit4] > at __randomizedtesting.SeedInfo.seed([CEC58863C9722182]:0) [junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234) [junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithKerberos: [junit4] > 1) Thread[id=11044, name=zkCallback-1760-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > 2) Thread[id=11080, name=zkCallback-1760-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > 3) Thread[id=11043, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) [junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) [junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) [junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) [junit4] > 4) Thread[id=11042, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-SendThread(127.0.0.1:57610), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) [junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) [junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) [junit4] > 5) Thread[id=11045, name=zkCallback-1760-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > at __randomizedtesting.SeedInfo.seed([CEC58863C9722182]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: [junit4] > 1) Thread[id=11044, name=zkCallback-1760-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > 2) Thread[id=11080, name=zkCallback-1760-thread-3, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > 3) Thread[id=11042, name=TEST-TestSolrCloudWithKerberos.testKerberizedSolr-seed#[CEC58863C9722182]-SendThread(127.0.0.1:57610), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) [junit4] > 4) Thread[id=11045, name=zkCallback-1760-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithKerberos] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > at __randomizedtesting.SeedInfo.seed([CEC58863C9722182]:0) [junit4] Completed [424/502] on J1 in 52.86s, 1 test, 1 failure, 3 errors <<< FAILURES! [...truncated 240 lines...] BUILD FAILED /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:526: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:474: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:61: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/extra-targets.xml:39: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build.xml:229: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/common-build.xml:512: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:1415: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:973: There were test failures: 502 suites, 1986 tests, 3 suite-level errors, 1 error, 358 ignored (25 assumptions) Total time: 51 minutes 53 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts No prior successful build to compare, so performing full copy of artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
