Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/10/
1 tests failed. FAILED: org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr Error Message: No live SolrServers available to handle this request:[http://127.0.0.1:39003, http://127.0.0.1:51276, http://127.0.0.1:33817, http://127.0.0.1:40748, http://127.0.0.1:39669] Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:39003, http://127.0.0.1:51276, http://127.0.0.1:33817, http://127.0.0.1:40748, http://127.0.0.1:39669] at __randomizedtesting.SeedInfo.seed([7F80B71B49A3C81:ACE1DDB6FEFBFCB6]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152) at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39669: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /admin/collections. Reason: <pre> org.apache.solr.common.SolrException: Error during request authentication, </pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328) ... 47 more Build Log: [...truncated 10521 lines...] [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithKerberos [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/init-core-data-001 [junit4] 2> 1478245 T8713 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2> 1483607 T8713 oadsc.DefaultDirectoryService.showSecurityWarnings WARN You didn't change the admin password of directory service instance 'DefaultKrbServer'. Please update the admin password as soon as possible to prevent a possible security breach. [junit4] 2> 1484334 T8713 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 2> 1484334 T8722 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 1484335 T8722 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 1484435 T8713 oasc.ZkTestServer.run start zk server on port:40569 [junit4] 2> 1484437 T8729 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1484447 T8732 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1484452 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 1484454 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 1484456 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1484458 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 1484459 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 1484461 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 1484462 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 1484464 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 1484465 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1484467 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 1484468 T8713 oasc.AbstractZkTestCase.putConfig put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 1484472 T8736 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1484812 T8713 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1 [junit4] 2> 1484814 T8713 oejs.Server.doStart jetty-9.2.10.v20150310 [junit4] 2> 1484816 T8713 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@4436ea90{/,null,AVAILABLE} [junit4] 2> 1484817 T8713 oejs.AbstractConnector.doStart Started ServerConnector@3a49cb8c{HTTP/1.1}{127.0.0.1:40748} [junit4] 2> 1484817 T8713 oejs.Server.doStart Started @1486683ms [junit4] 2> 1484817 T8713 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-002/control/data, hostContext=/, hostPort=40748, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores} [junit4] 2> 1484818 T8713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1484818 T8713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/' [junit4] 2> 1485974 T8713 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/solr.xml [junit4] 2> 1485980 T8713 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores [junit4] 2> 1485981 T8713 oasc.CoreContainer.<init> New CoreContainer 222157298 [junit4] 2> 1485981 T8713 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/] [junit4] 2> 1485981 T8713 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/lib [junit4] 2> 1485982 T8713 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/lib). [junit4] 2> 1485998 T8713 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 1486001 T8713 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1486002 T8713 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1486002 T8713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1486003 T8713 oasc.CoreContainer.load Node Name: 127.0.0.1 [junit4] 2> 1486003 T8713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40569/solr [junit4] 2> 1486003 T8713 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1486004 T8751 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1486015 T8755 n:127.0.0.1:40748_ oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1486025 T8713 n:127.0.0.1:40748_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40748_ [junit4] 2> 1486030 T8713 n:127.0.0.1:40748_ oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1486031 T8713 n:127.0.0.1:40748_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40748_ [junit4] 2> 1486033 T8713 n:127.0.0.1:40748_ oasc.Overseer.start Overseer (id=93883821892239364-127.0.0.1:40748_-n_0000000000) starting [junit4] 2> 1486040 T8713 n:127.0.0.1:40748_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 1486041 T8758 n:127.0.0.1:40748_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 1486042 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 1486045 T8713 n:127.0.0.1:40748_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin [junit4] 2> 1486054 T8713 n:127.0.0.1:40748_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT} [junit4] 2> 1486075 T8713 n:127.0.0.1:40748_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler. [junit4] 2> 1486076 T8713 n:127.0.0.1:40748_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5c575b56 [junit4] 2> 1486077 T8713 n:127.0.0.1:40748_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1486080 T8713 n:127.0.0.1:40748_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@5c575b56 [junit4] 2> 1486080 T8713 n:127.0.0.1:40748_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1486083 T8713 n:127.0.0.1:40748_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores [junit4] 2> 1486084 T8713 n:127.0.0.1:40748_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 1486084 T8713 n:127.0.0.1:40748_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/ [junit4] 2> 1486085 T8713 n:127.0.0.1:40748_ oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1486086 T8761 n:127.0.0.1:40748_ c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection [junit4] 2> 1486086 T8761 n:127.0.0.1:40748_ c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1486087 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1486087 T8761 n:127.0.0.1:40748_ oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1486088 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:40748";, [junit4] 2> "node_name":"127.0.0.1:40748_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 0 [junit4] 2> 1486088 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:40748";, [junit4] 2> "node_name":"127.0.0.1:40748_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 1486088 T8757 n:127.0.0.1:40748_ oasco.ClusterStateMutator.createCollection building a new cName: control_collection [junit4] 2> 1486089 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1 [junit4] 2> 1487087 T8761 n:127.0.0.1:40748_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1487088 T8761 n:127.0.0.1:40748_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection [junit4] 2> 1487091 T8761 n:127.0.0.1:40748_ oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1487092 T8761 n:127.0.0.1:40748_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/' [junit4] 2> 1487113 T8761 n:127.0.0.1:40748_ oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1487120 T8761 n:127.0.0.1:40748_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1487139 T8761 n:127.0.0.1:40748_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1487148 T8761 n:127.0.0.1:40748_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1487149 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1487155 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1487275 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1487281 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1487283 T8761 n:127.0.0.1:40748_ oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1487292 T8761 n:127.0.0.1:40748_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1487294 T8761 n:127.0.0.1:40748_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1487296 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1487296 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1487297 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1487297 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 1487298 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1487299 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/], dataDir=[null] [junit4] 2> 1487299 T8761 n:127.0.0.1:40748_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17cfcf30 [junit4] 2> 1487299 T8761 n:127.0.0.1:40748_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data [junit4] 2> 1487300 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data/index/ [junit4] 2> 1487301 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1487301 T8761 n:127.0.0.1:40748_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/control-001/cores/collection1/data/index [junit4] 2> 1487301 T8761 n:127.0.0.1:40748_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3006284492664655] [junit4] 2> 1487302 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3229d82c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@477df76e),segFN=segments_1,generation=1} [junit4] 2> 1487302 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1487306 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1487306 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1487306 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1487307 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1487308 T8761 n:127.0.0.1:40748_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1487308 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1487310 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1487311 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1487312 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1487313 T8761 n:127.0.0.1:40748_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1487316 T8761 n:127.0.0.1:40748_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file [junit4] 2> 1487317 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1487322 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog [junit4] 2> 1487323 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 1487325 T8761 n:127.0.0.1:40748_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1487325 T8761 n:127.0.0.1:40748_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1487326 T8761 n:127.0.0.1:40748_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0] [junit4] 2> 1487327 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3229d82c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@477df76e),segFN=segments_1,generation=1} [junit4] 2> 1487327 T8761 n:127.0.0.1:40748_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1487327 T8761 n:127.0.0.1:40748_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@6edfb126[collection1] main [junit4] 2> 1487328 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1487329 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1487329 T8761 n:127.0.0.1:40748_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1487329 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1487330 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1487330 T8761 n:127.0.0.1:40748_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1487330 T8761 n:127.0.0.1:40748_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1487331 T8761 n:127.0.0.1:40748_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1487331 T8763 n:127.0.0.1:40748_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6edfb126[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1487332 T8761 n:127.0.0.1:40748_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1 [junit4] 2> 1487332 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field [junit4] 2> 1487332 T8761 n:127.0.0.1:40748_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index [junit4] 2> 1487333 T8761 n:127.0.0.1:40748_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex No terms found for _version_, cannot seed version bucket highest value from index [junit4] 2> 1487333 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Could not find max version in index or recent updates, using new clock 1502141153418412032 [junit4] 2> 1487335 T8761 n:127.0.0.1:40748_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 3 ms to seed version buckets with highest version 1502141153418412032 [junit4] 2> 1487336 T8761 n:127.0.0.1:40748_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1487341 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40748 collection:control_collection shard:shard1 [junit4] 2> 1487341 T8713 n:127.0.0.1:40748_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1487342 T8713 n:127.0.0.1:40748_ oass.SolrDispatchFilter.init user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2 [junit4] 2> 1487342 T8713 n:127.0.0.1:40748_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1487343 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1487344 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1487347 T8769 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1487352 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 1487354 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1487354 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 1487355 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection"} current state version: 1 [junit4] 2> 1487355 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C1209 name=collection1 org.apache.solr.core.SolrCore@c68063c url=http://127.0.0.1:40748/collection1 node=127.0.0.1:40748_ C1209_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:40748, node_name=127.0.0.1:40748_, state=down} [junit4] 2> 1487356 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 C1209 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40748/collection1/ [junit4] 2> 1487356 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 C1209 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 1487357 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 C1209 oasc.SyncStrategy.syncToMe http://127.0.0.1:40748/collection1/ has no replicas [junit4] 2> 1487357 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40748/collection1/ shard1 [junit4] 2> 1487360 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1487361 T8713 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 1487361 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "base_url":"http://127.0.0.1:40748";, [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 1 [junit4] 2> 1487512 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:40748/collection1/ and leader is http://127.0.0.1:40748/collection1/ [junit4] 2> 1487512 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40748 [junit4] 2> 1487513 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 1487513 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection [junit4] 2> 1487513 T8766 n:127.0.0.1:40748_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1487514 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1487515 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:40748";, [junit4] 2> "node_name":"127.0.0.1:40748_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 1487516 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:40748";, [junit4] 2> "node_name":"127.0.0.1:40748_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 1487719 T8713 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1 [junit4] 2> 1487721 T8713 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001 [junit4] 2> 1487722 T8713 oejs.Server.doStart jetty-9.2.10.v20150310 [junit4] 2> 1487727 T8713 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@2eab3622{/,null,AVAILABLE} [junit4] 2> 1487728 T8713 oejs.AbstractConnector.doStart Started ServerConnector@31002725{HTTP/1.1}{127.0.0.1:39003} [junit4] 2> 1487729 T8713 oejs.Server.doStart Started @1489595ms [junit4] 2> 1487729 T8713 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-002/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=39003, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores} [junit4] 2> 1487729 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1487730 T8713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1487730 T8713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/' [junit4] 2> 1487745 T8713 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/solr.xml [junit4] 2> 1487750 T8713 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores [junit4] 2> 1487751 T8713 oasc.CoreContainer.<init> New CoreContainer 454062642 [junit4] 2> 1487751 T8713 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/] [junit4] 2> 1487751 T8713 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/lib [junit4] 2> 1487751 T8713 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/lib). [junit4] 2> 1487758 T8713 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 1487759 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1487760 T8713 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1487761 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1487761 T8713 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1487761 T8713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1487762 T8713 oasc.CoreContainer.load Node Name: 127.0.0.1 [junit4] 2> 1487762 T8713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40569/solr [junit4] 2> 1487762 T8713 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1487764 T8783 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1487767 T8786 n:127.0.0.1:39003_ oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1488783 T8713 n:127.0.0.1:39003_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39003_ [junit4] 2> 1488786 T8713 n:127.0.0.1:39003_ oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1488788 T8713 n:127.0.0.1:39003_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin [junit4] 2> 1488788 T8713 n:127.0.0.1:39003_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT} [junit4] 2> 1488790 T8713 n:127.0.0.1:39003_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler. [junit4] 2> 1488790 T8713 n:127.0.0.1:39003_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@4334d2 [junit4] 2> 1488791 T8713 n:127.0.0.1:39003_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1488791 T8713 n:127.0.0.1:39003_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@4334d2 [junit4] 2> 1488791 T8713 n:127.0.0.1:39003_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1488793 T8713 n:127.0.0.1:39003_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores [junit4] 2> 1488794 T8713 n:127.0.0.1:39003_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 1488794 T8713 n:127.0.0.1:39003_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/ [junit4] 2> 1488795 T8713 n:127.0.0.1:39003_ oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1488799 T8788 n:127.0.0.1:39003_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 1488799 T8788 n:127.0.0.1:39003_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1488800 T8788 n:127.0.0.1:39003_ oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1488800 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1488801 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:39003";, [junit4] 2> "node_name":"127.0.0.1:39003_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 3 [junit4] 2> 1488801 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:39003";, [junit4] 2> "node_name":"127.0.0.1:39003_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1488802 T8757 n:127.0.0.1:40748_ oasco.ClusterStateMutator.createCollection building a new cName: collection1 [junit4] 2> 1488802 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2 [junit4] 2> 1489800 T8788 n:127.0.0.1:39003_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1489801 T8788 n:127.0.0.1:39003_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 1489801 T8788 n:127.0.0.1:39003_ oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1489802 T8788 n:127.0.0.1:39003_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/' [junit4] 2> 1489814 T8788 n:127.0.0.1:39003_ oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1489820 T8788 n:127.0.0.1:39003_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1489826 T8788 n:127.0.0.1:39003_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1489841 T8788 n:127.0.0.1:39003_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1489842 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1489854 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1490019 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1490025 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1490026 T8788 n:127.0.0.1:39003_ oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1490034 T8788 n:127.0.0.1:39003_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1490036 T8788 n:127.0.0.1:39003_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1490038 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1490039 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1490039 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1490039 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 1490040 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1490041 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/], dataDir=[null] [junit4] 2> 1490041 T8788 n:127.0.0.1:39003_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17cfcf30 [junit4] 2> 1490041 T8788 n:127.0.0.1:39003_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data [junit4] 2> 1490042 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data/index/ [junit4] 2> 1490042 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1490042 T8788 n:127.0.0.1:39003_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-1-001/cores/collection1/data/index [junit4] 2> 1490043 T8788 n:127.0.0.1:39003_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3006284492664655] [junit4] 2> 1490043 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7211656a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@293e526a),segFN=segments_1,generation=1} [junit4] 2> 1490043 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1490047 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1490048 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1490049 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1490049 T8788 n:127.0.0.1:39003_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1490049 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1490051 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1490052 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1490053 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1490054 T8788 n:127.0.0.1:39003_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1490058 T8788 n:127.0.0.1:39003_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file [junit4] 2> 1490059 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1490060 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog [junit4] 2> 1490060 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 1490061 T8788 n:127.0.0.1:39003_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1490061 T8788 n:127.0.0.1:39003_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1490062 T8788 n:127.0.0.1:39003_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0] [junit4] 2> 1490062 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7211656a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@293e526a),segFN=segments_1,generation=1} [junit4] 2> 1490062 T8788 n:127.0.0.1:39003_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1490063 T8788 n:127.0.0.1:39003_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@365f21f0[collection1] main [junit4] 2> 1490070 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1490071 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1490071 T8788 n:127.0.0.1:39003_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1490071 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1490072 T8788 n:127.0.0.1:39003_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1490073 T8788 n:127.0.0.1:39003_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1 [junit4] 2> 1490074 T8789 n:127.0.0.1:39003_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@365f21f0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.onFirstSearcher On first searcher opened, looking up max value of version field [junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex Refreshing highest value of _version_ for 65536 version buckets from index [junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.VersionInfo.getMaxVersionFromIndex No terms found for _version_, cannot seed version bucket highest value from index [junit4] 2> 1490074 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Could not find max version in index or recent updates, using new clock 1502141156292558848 [junit4] 2> 1490077 T8788 n:127.0.0.1:39003_ x:collection1 oasu.UpdateLog.seedBucketsWithHighestVersion Took 3 ms to seed version buckets with highest version 1502141156292558848 [junit4] 2> 1490077 T8788 n:127.0.0.1:39003_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1490078 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39003 collection:collection1 shard:shard2 [junit4] 2> 1490078 T8713 n:127.0.0.1:39003_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1490079 T8713 n:127.0.0.1:39003_ oass.SolrDispatchFilter.init user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2 [junit4] 2> 1490079 T8713 n:127.0.0.1:39003_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1490080 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1490082 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2 [junit4] 2> 1490084 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1490084 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 1490084 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C1210 name=collection1 org.apache.solr.core.SolrCore@680e69fb url=http://127.0.0.1:39003/collection1 node=127.0.0.1:39003_ C1210_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:39003, node_name=127.0.0.1:39003_, state=down} [junit4] 2> 1490084 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 C1210 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39003/collection1/ [junit4] 2> 1490084 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1"} current state version: 4 [junit4] 2> 1490085 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 C1210 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 1490085 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 C1210 oasc.SyncStrategy.syncToMe http://127.0.0.1:39003/collection1/ has no replicas [junit4] 2> 1490085 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39003/collection1/ shard2 [junit4] 2> 1490088 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1490089 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "base_url":"http://127.0.0.1:39003";, [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 4 [junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register We are http://127.0.0.1:39003/collection1/ and leader is http://127.0.0.1:39003/collection1/ [junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39003 [junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 1490240 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1 [junit4] 2> 1490241 T8792 n:127.0.0.1:39003_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1490241 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1490242 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:39003";, [junit4] 2> "node_name":"127.0.0.1:39003_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 5 [junit4] 2> 1490243 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:39003";, [junit4] 2> "node_name":"127.0.0.1:39003_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1490432 T8713 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1 [junit4] 2> 1490433 T8713 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001 [junit4] 2> 1490434 T8713 oejs.Server.doStart jetty-9.2.10.v20150310 [junit4] 2> 1490436 T8713 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@5f1593e7{/,null,AVAILABLE} [junit4] 2> 1490437 T8713 oejs.AbstractConnector.doStart Started ServerConnector@4a36cc46{HTTP/1.1}{127.0.0.1:33817} [junit4] 2> 1490437 T8713 oejs.Server.doStart Started @1492303ms [junit4] 2> 1490438 T8713 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-002/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=33817, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores} [junit4] 2> 1490438 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1490438 T8713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1490439 T8713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/' [junit4] 2> 1490454 T8713 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/solr.xml [junit4] 2> 1490475 T8713 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores [junit4] 2> 1490476 T8713 oasc.CoreContainer.<init> New CoreContainer 6697740 [junit4] 2> 1490477 T8713 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/] [junit4] 2> 1490477 T8713 oasc.CoreContainer.load loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/lib [junit4] 2> 1490479 T8713 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/lib). [junit4] 2> 1490486 T8713 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 1490487 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1490488 T8713 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1490488 T8713 oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1490489 T8713 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1490489 T8713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1490489 T8713 oasc.CoreContainer.load Node Name: 127.0.0.1 [junit4] 2> 1490489 T8713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40569/solr [junit4] 2> 1490490 T8713 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1490495 T8806 oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1490508 T8810 n:127.0.0.1:33817_ oascc.ConnectionManager.process WARN zkClient received AuthFailed [junit4] 2> 1491530 T8713 n:127.0.0.1:33817_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33817_ [junit4] 2> 1491534 T8713 n:127.0.0.1:33817_ oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1491535 T8713 n:127.0.0.1:33817_ oasc.CoreContainer.initializeAuthenticationPlugin Authentication plugin class obtained from ZK: org.apache.solr.security.KerberosPlugin [junit4] 2> 1491536 T8713 n:127.0.0.1:33817_ oass.KerberosPlugin.init Params: {token.valid=30, kerberos.principal=HTTP/127.0.0.1, cookie.domain=127.0.0.1, kerberos.keytab=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/keytabs, cookie.path=/, type=kerberos, kerberos.name.rules=DEFAULT} [junit4] 2> 1491538 T8713 n:127.0.0.1:33817_ oasc.CoreContainer.initializeAuthenticationPlugin Reconfiguring the shard handler factory and update shard handler. [junit4] 2> 1491538 T8713 n:127.0.0.1:33817_ oashc.HttpShardHandlerFactory.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@3c3729d7 [junit4] 2> 1491539 T8713 n:127.0.0.1:33817_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1491539 T8713 n:127.0.0.1:33817_ oasu.UpdateShardHandler.reconfigureHttpClient Reconfiguring the default client with: org.apache.solr.client.solrj.impl.Krb5HttpClientConfigurer@3c3729d7 [junit4] 2> 1491539 T8713 n:127.0.0.1:33817_ oascsi.Krb5HttpClientConfigurer.configure Setting up SPNego auth with config: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/tempDir-001/minikdc/jaas-client.conf [junit4] 2> 1491540 T8713 n:127.0.0.1:33817_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores [junit4] 2> 1491541 T8713 n:127.0.0.1:33817_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1, collection=collection1, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 1491542 T8713 n:127.0.0.1:33817_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/ [junit4] 2> 1491542 T8713 n:127.0.0.1:33817_ oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1491543 T8813 n:127.0.0.1:33817_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 1491543 T8813 n:127.0.0.1:33817_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1491544 T8755 n:127.0.0.1:40748_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1491544 T8813 n:127.0.0.1:33817_ oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1491545 T8757 n:127.0.0.1:40748_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:33817";, [junit4] 2> "node_name":"127.0.0.1:33817_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 6 [junit4] 2> 1491545 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:33817";, [junit4] 2> "node_name":"127.0.0.1:33817_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1491545 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Collection already exists with numShards=2 [junit4] 2> 1491546 T8757 n:127.0.0.1:40748_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1 [junit4] 2> 1492544 T8813 n:127.0.0.1:33817_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1492545 T8813 n:127.0.0.1:33817_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 1492545 T8813 n:127.0.0.1:33817_ oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1492546 T8813 n:127.0.0.1:33817_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/' [junit4] 2> 1492558 T8813 n:127.0.0.1:33817_ oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1492562 T8813 n:127.0.0.1:33817_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1492566 T8813 n:127.0.0.1:33817_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1492575 T8813 n:127.0.0.1:33817_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1492576 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1492581 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1492685 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1492690 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1492692 T8813 n:127.0.0.1:33817_ oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1492700 T8813 n:127.0.0.1:33817_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1492702 T8813 n:127.0.0.1:33817_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1492703 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1492704 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1492704 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1492705 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1492705 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1492706 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/], dataDir=[null] [junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17cfcf30 [junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data [junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data/index/ [junit4] 2> 1492707 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1492708 T8813 n:127.0.0.1:33817_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001/shard-2-001/cores/collection1/data/index [junit4] 2> 1492708 T8813 n:127.0.0.1:33817_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3006284492664655] [junit4] 2> 1492708 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@d394c40 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@abd78ba),segFN=segments_1,generation=1} [junit4] 2> 1492709 T8813 n:127.0.0.1:33817_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1492712 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1492713 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1492714 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1492714 T8813 n:127.0.0.1:33817_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit [...truncated too long message...] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 1509722 T8713 oejs.AbstractConnector.doStop Stopped ServerConnector@4dab548e{HTTP/1.1}{127.0.0.1:0} [junit4] 2> 1509722 T8713 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@61c3db9c{/,null,UNAVAILABLE} [junit4] 2> 1509725 T8713 c:control_collection s:shard1 x:collection1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40569 40569 [junit4] 2> 1509998 T8722 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40569 40569 [junit4] 2> 1509999 T8722 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 7 /solr/aliases.json [junit4] 2> 7 /solr/clusterstate.json [junit4] 2> 5 /solr/configs/conf1 [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 7 /solr/live_nodes [junit4] 2> 5 /solr/overseer/collection-queue-work [junit4] 2> 4 /solr/overseer/queue [junit4] 2> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithKerberos -Dtests.method=testKerberizedSolr -Dtests.seed=7F80B71B49A3C81 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es_BO -Dtests.timezone=Canada/Newfoundland -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 31.8s J2 | TestSolrCloudWithKerberos.testKerberizedSolr <<< [junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:39003, http://127.0.0.1:51276, http://127.0.0.1:33817, http://127.0.0.1:40748, http://127.0.0.1:39669] [junit4] > at __randomizedtesting.SeedInfo.seed([7F80B71B49A3C81:ACE1DDB6FEFBFCB6]:0) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799) [junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135) [junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:152) [junit4] > at org.apache.solr.cloud.TestSolrCloudWithKerberos.testKerberizedSolr(TestSolrCloudWithKerberos.java:159) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39669: Expected mime type application/octet-stream but got text/html. <html> [junit4] > <head> [junit4] > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> [junit4] > <title>Error 500 </title> [junit4] > </head> [junit4] > <body> [junit4] > <h2>HTTP ERROR: 500</h2> [junit4] > <p>Problem accessing /admin/collections. Reason: [junit4] > <pre> org.apache.solr.common.SolrException: Error during request authentication, </pre></p> [junit4] > <hr /><i><small>Powered by Jetty://</small></i> [junit4] > </body> [junit4] > </html> [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:529) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328) [junit4] > ... 47 more [junit4] 2> 1510012 T8713 c:control_collection s:shard1 x:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithKerberos 7F80B71B49A3C81-001 [junit4] 2> 31774 T8712 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene50), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_BO, timezone=Canada/Newfoundland [junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=4,threads=1,free=171080232,total=513802240 [junit4] 2> NOTE: All tests run in this JVM: [QueryParsingTest, SuggestComponentTest, OutputWriterTest, TestBinaryResponseWriter, HdfsUnloadDistributedZkTest, OutOfBoxZkACLAndCredentialsProvidersTest, DeleteShardTest, MultiThreadedOCPTest, TestStandardQParsers, SoftAutoCommitTest, XmlUpdateRequestHandlerTest, BasicDistributedZk2Test, DirectUpdateHandlerTest, SolrInfoMBeanTest, LoggingHandlerTest, TestManagedSchemaFieldResource, TestFieldCollectionResource, TestFastOutputStream, TestFieldTypeResource, RegexBoostProcessorTest, HdfsSyncSliceTest, TestPseudoReturnFields, DistributedQueryElevationComponentTest, TestSolrJ, TestSort, CircularListTest, SolrIndexSplitterTest, StatsComponentTest, TestSolrXml, TestSolrQueryParserDefaultOperatorResource, TestStressRecovery, TestQuerySenderNoQuery, DocValuesTest, TestSolr4Spatial2, SolrTestCaseJ4Test, FacetPivotSmallTest, SearchHandlerTest, TestManagedStopFilterFactory, TestShortCircuitedRequests, FileBasedSpellCheckerTest, TestMissingGroups, AsyncMigrateRouteKeyTest, DistribJoinFromCollectionTest, AutoCommitTest, CachingDirectoryFactoryTest, StressHdfsTest, QueryEqualityTest, TestSolrConfigHandler, RecoveryZkTest, MergeStrategyTest, TestRecovery, SolrCoreCheckLockOnStartupTest, TestDynamicFieldCollectionResource, TestExactSharedStatsCache, XsltUpdateRequestHandlerTest, TestMiniSolrCloudClusterSSL, TestCloudInspectUtil, AliasIntegrationTest, TestPartialUpdateDeduplication, TestCopyFieldCollectionResource, ResponseLogComponentTest, PluginInfoTest, DistributedIntervalFacetingTest, CustomCollectionTest, TestSearcherReuse, OverriddenZkACLAndCredentialsProvidersTest, DistanceUnitsTest, TestSolrQueryParser, VMParamsZkACLAndCredentialsProvidersTest, TestGroupingSearch, TestPerFieldSimilarity, HdfsRecoverLeaseTest, TestLeaderElectionZkExpiry, TestSystemIdResolver, TestMiniSolrCloudCluster, PolyFieldTest, SOLR749Test, TestZkChroot, CollectionsAPIDistributedZkTest, OverseerTest, ClusterStateUpdateTest, LeaderElectionTest, ShardRoutingCustomTest, TestFaceting, TestHashPartitioner, TestStressReorder, TestReload, TestStressVersions, SpellCheckComponentTest, BadIndexSchemaTest, TestBadConfig, CurrencyFieldXmlFileTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestTrie, StatelessScriptUpdateProcessorFactoryTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestQueryTypes, DocumentBuilderTest, TestCollationField, ReturnFieldsTest, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, TestQuerySenderListener, TestSolrIndexConfig, TestMergePolicyConfig, SampleTest, TestDocSet, NumericFieldsTest, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestXIncludeConfig, TestSweetSpotSimilarityFactory, TestFastWriter, ScriptEngineTest, PrimUtilsTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestUtils, ZkNodePropsTest, TestRTGBase, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, BaseCdcrDistributedZkTest, CdcrVersionReplicationTest, DistribCursorPagingTest, DistributedVersionInfoTest, OverseerStatusTest, ReplicationFactorTest, RollingRestartTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, TestRebalanceLeaders, TestSolrCloudWithKerberos] [junit4] Completed [433/502] on J2 in 32.55s, 1 test, 1 error <<< FAILURES! [...truncated 222 lines...] BUILD FAILED /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:526: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:474: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/build.xml:61: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/extra-targets.xml:39: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/build.xml:229: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/solr/common-build.xml:512: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:1415: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java8/lucene/common-build.xml:973: There were test failures: 502 suites, 1984 tests, 1 error, 35 ignored (25 assumptions) Total time: 50 minutes 23 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts No prior successful build to compare, so performing full copy of artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
