Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/663/
1 tests failed. REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch Error Message: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1 Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1 at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:569) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 11845 lines...] [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest [junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/init-core-data-001 [junit4] 2> 61841 T58 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false) [junit4] 2> 61842 T58 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_fmo/vv [junit4] 2> 61851 T58 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 61889 T58 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 61909 T59 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 62310 T58 oasc.ZkTestServer.run start zk server on port:51405 [junit4] 2> 62321 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 62566 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 62761 T65 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58990034 name:ZooKeeperConnection Watcher:127.0.0.1:51405 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 62762 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 62763 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 62777 T58 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 62928 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 62929 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 62931 T67 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c52d244 name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 62931 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 62932 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 62942 T58 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 62946 T58 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 62950 T58 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 62953 T58 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 62957 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 62982 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 62988 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 62989 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 62993 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 62994 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 62998 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 62998 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 63003 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 63004 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 63008 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 63009 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 63012 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 63013 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 63016 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 63017 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 63022 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 63022 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 63026 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 63027 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 63031 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 63031 T58 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> Using new style solr.xml [junit4] 2> 63037 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 63037 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 63039 T69 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@129c5a28 name:ZooKeeperConnection Watcher:127.0.0.1:51405 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 63039 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 63040 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 63040 T58 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 63056 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 63057 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 63059 T71 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59cb9e9 name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 63059 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 63059 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 63061 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml [junit4] 2> 63062 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml [junit4] 2> 63067 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml [junit4] 2> 63068 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml [junit4] 2> 63072 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 63073 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 63076 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt [junit4] 2> 63077 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt [junit4] 2> 63080 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt [junit4] 2> 63081 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt [junit4] 2> 63085 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml [junit4] 2> 63085 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml [junit4] 2> 63089 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json [junit4] 2> 63089 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json [junit4] 2> 63092 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt [junit4] 2> 63093 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt [junit4] 2> 63096 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt [junit4] 2> 63097 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt [junit4] 2> 63101 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt [junit4] 2> 63101 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt [junit4] 2> 63105 T58 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml [junit4] 2> 63105 T58 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml [junit4] 2> 64835 T58 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 65107 T58 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:40604 [junit4] 2> 65270 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 65270 T58 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 65271 T58 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-002 [junit4] 2> 65271 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-002/' [junit4] 2> 65309 T58 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-002/solr.xml [junit4] 2> 65412 T58 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-002 [junit4] 2> 65413 T58 oasc.CoreContainer.<init> New CoreContainer 998079790 [junit4] 2> 65413 T58 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-002/] [junit4] 2> 65478 T58 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 65478 T58 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 65479 T58 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 65479 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 65480 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 65480 T58 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 65480 T58 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 65481 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 65481 T58 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 65481 T58 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 65482 T58 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 65482 T58 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 65483 T58 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 65483 T58 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 65483 T58 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51405/solr [junit4] 2> 65516 T58 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 65517 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 65518 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 65521 T83 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fa15f92 name:ZooKeeperConnection Watcher:127.0.0.1:51405 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 65521 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 65522 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 65536 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 65538 T85 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5dde3094 name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 65539 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 65554 T58 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 65567 T58 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 65575 T58 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 65578 T58 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 65582 T58 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 65602 T58 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 65604 T58 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40604__fmo%2Fvv [junit4] 2> 65611 T58 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40604__fmo%2Fvv [junit4] 2> 65615 T58 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 65617 T58 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 65620 T58 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 65635 T58 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40604__fmo%2Fvv [junit4] 2> 65636 T58 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 65638 T58 oasc.Overseer.start Overseer (id=92662336192708613-127.0.0.1:40604__fmo%2Fvv-n_0000000000) starting [junit4] 2> 65661 T58 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 65692 T58 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 65711 T87 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 65712 T58 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 65715 T58 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 65717 T58 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 65758 T86 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 65764 T58 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-002 [junit4] 2> 65869 T58 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-002/conf/ [junit4] 2> 66669 T58 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 66671 T58 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3 [junit4] 2> 66671 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 66772 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 66773 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 66775 T91 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f6eb750 name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 66776 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 66776 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 66778 T58 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 66786 T58 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 66787 T58 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2 [junit4] 2> 66787 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 66788 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 66790 T93 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4bc988ac name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 66790 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 66790 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 66818 T94 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 66844 T86 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1 [junit4] 2> 66845 T86 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2] [junit4] 2> 66887 T86 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2 [junit4] 2> 66928 T86 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json [junit4] 2> 66931 T94 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 66931 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 68104 T58 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 68105 T58 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 68107 T58 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50093 [junit4] 2> 68108 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 68108 T58 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 68109 T58 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003 [junit4] 2> 68109 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/' [junit4] 2> 68145 T58 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solr.xml [junit4] 2> 68199 T58 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003 [junit4] 2> 68200 T58 oasc.CoreContainer.<init> New CoreContainer 1775649763 [junit4] 2> 68200 T58 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/] [junit4] 2> 68201 T58 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 68202 T58 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 68202 T58 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 68202 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 68203 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 68203 T58 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 68203 T58 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 68204 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 68204 T58 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 68205 T58 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 68205 T58 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 68206 T58 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 68206 T58 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 68206 T58 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 68207 T58 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51405/solr [junit4] 2> 68207 T58 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 68208 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 68209 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 68210 T106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7203225d name:ZooKeeperConnection Watcher:127.0.0.1:51405 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 68211 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 68211 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 68214 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 68223 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@715c7b5f name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 68223 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 68232 T58 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 69240 T58 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50093__fmo%2Fvv [junit4] 2> 69242 T58 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50093__fmo%2Fvv [junit4] 2> 69247 T58 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 69256 T58 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003 [junit4] 2> 69258 T58 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/conf/ [junit4] 2> 70179 T58 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 70181 T58 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3 [junit4] 2> 70181 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 71840 T58 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 71846 T58 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 71849 T58 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36769 [junit4] 2> 71849 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 71850 T58 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 71850 T58 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004 [junit4] 2> 71850 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/' [junit4] 2> 71920 T58 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solr.xml [junit4] 2> 72046 T58 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004 [junit4] 2> 72047 T58 oasc.CoreContainer.<init> New CoreContainer 208880146 [junit4] 2> 72048 T58 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/] [junit4] 2> 72049 T58 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 72050 T58 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 72050 T58 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 72051 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 72051 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 72052 T58 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 72052 T58 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 72053 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 72053 T58 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 72054 T58 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 72055 T58 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 72055 T58 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 72056 T58 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 72056 T58 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 72057 T58 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51405/solr [junit4] 2> 72057 T58 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 72058 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 72059 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 72062 T121 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62f6b220 name:ZooKeeperConnection Watcher:127.0.0.1:51405 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 72062 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 72063 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 72066 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 72068 T123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11167ff8 name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 72069 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 72080 T58 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 73092 T58 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36769__fmo%2Fvv [junit4] 2> 73094 T58 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36769__fmo%2Fvv [junit4] 2> 73100 T58 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 73107 T58 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004 [junit4] 2> 74384 T58 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/conf/ [junit4] 2> 74664 T58 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 74667 T58 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3 [junit4] 2> 74667 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 76025 T58 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3 [junit4] 2> 76026 T58 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 76030 T58 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:47311 [junit4] 2> 76031 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 76031 T58 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 76032 T58 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005 [junit4] 2> 76032 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/' [junit4] 2> 76096 T58 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solr.xml [junit4] 2> 76174 T58 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005 [junit4] 2> 76175 T58 oasc.CoreContainer.<init> New CoreContainer 1853253742 [junit4] 2> 76176 T58 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/] [junit4] 2> 76178 T58 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 76178 T58 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 76179 T58 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 76179 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 76180 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 76180 T58 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 76181 T58 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 76181 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 76182 T58 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 76182 T58 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 76183 T58 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 76184 T58 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 76184 T58 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 76185 T58 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 76185 T58 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51405/solr [junit4] 2> 76186 T58 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 76186 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 76188 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 76189 T136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fc2466a name:ZooKeeperConnection Watcher:127.0.0.1:51405 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 76190 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 76190 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 76192 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 76194 T138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13d938ad name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 76194 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 76201 T58 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 77208 T58 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47311__fmo%2Fvv [junit4] 2> 77210 T58 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47311__fmo%2Fvv [junit4] 2> 77218 T58 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 77225 T58 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005 [junit4] 2> 77330 T58 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/conf/ [junit4] 2> 79116 T58 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 79118 T58 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3 [junit4] 2> 79119 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 80482 T58 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4 [junit4] 2> 80483 T58 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 80486 T58 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:33090 [junit4] 2> 80487 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 80487 T58 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 80487 T58 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006 [junit4] 2> 80488 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/' [junit4] 2> 80521 T58 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solr.xml [junit4] 2> 80684 T58 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006 [junit4] 2> 80685 T58 oasc.CoreContainer.<init> New CoreContainer 1339791304 [junit4] 2> 80685 T58 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/] [junit4] 2> 80686 T58 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 80687 T58 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 80687 T58 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 80687 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 80688 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 80688 T58 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 80688 T58 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 80689 T58 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 80689 T58 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 80689 T58 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 80690 T58 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 80690 T58 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 80691 T58 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 80691 T58 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 80691 T58 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51405/solr [junit4] 2> 80692 T58 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 80692 T58 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 80693 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 80695 T151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59c9f46d name:ZooKeeperConnection Watcher:127.0.0.1:51405 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 80696 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 80696 T58 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 80699 T58 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 80701 T153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52c6fc30 name:ZooKeeperConnection Watcher:127.0.0.1:51405/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 80701 T58 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 80709 T58 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 81716 T58 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33090__fmo%2Fvv [junit4] 2> 81718 T58 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33090__fmo%2Fvv [junit4] 2> 81724 T58 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 81730 T58 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006 [junit4] 2> 81756 T58 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/conf/ [junit4] 2> 83017 T58 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 83019 T58 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3 [junit4] 2> 83019 T58 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 85054 T99 oasha.CollectionsHandler.handleCreateAction Creating Collection : name=solrj_collection&version=2&wt=javabin&router.field=myOwnField&collection.configName=conf1&numShards=2&action=CREATE&replicationFactor=2 [junit4] 2> 85059 T94 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged [junit4] 2> 85062 T87 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{ [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"solrj_collection", [junit4] 2> "replicationFactor":"2", [junit4] 2> "collection.configName":"conf1", [junit4] 2> "numShards":"2", [junit4] 2> "router.field":"myOwnField"} [junit4] 2> 85072 T156 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , { [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"solrj_collection", [junit4] 2> "replicationFactor":"2", [junit4] 2> "collection.configName":"conf1", [junit4] 2> "numShards":"2", [junit4] 2> "router.field":"myOwnField"} [junit4] 2> 85081 T156 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection [junit4] 2> 85083 T156 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection [junit4] 2> 85090 T94 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 85097 T86 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection [junit4] 2> 85098 T86 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2] [junit4] 2> 85098 T86 oasc.Overseer$ClusterStateUpdater.createCollection state version solrj_collection 1 [junit4] 2> 85103 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85104 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85104 T139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85104 T94 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85104 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85103 T154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85191 T156 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2 [junit4] 2> 85193 T156 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:47311__fmo%2Fvv [junit4] 2> 85231 T156 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:50093__fmo%2Fvv [junit4] 2> 85233 T156 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:36769__fmo%2Fvv [junit4] 2> 85235 T156 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:33090__fmo%2Fvv [junit4] 2> 85347 T115 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard2_replica1&qt=/admin/cores&version=2&wt=javabin&collection.configName=conf1&numShards=2&collection=solrj_collection&action=CREATE&shard=shard2 [junit4] 2> 85347 T143 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard2_replica2&qt=/admin/cores&version=2&wt=javabin&collection.configName=conf1&numShards=2&collection=solrj_collection&action=CREATE&shard=shard2 [junit4] 2> 85347 T129 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard1_replica1&qt=/admin/cores&version=2&wt=javabin&collection.configName=conf1&numShards=2&collection=solrj_collection&action=CREATE&shard=shard1 [junit4] 2> 85347 T100 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard1_replica2&qt=/admin/cores&version=2&wt=javabin&collection.configName=conf1&numShards=2&collection=solrj_collection&action=CREATE&shard=shard1 [junit4] 2> 85364 T100 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection [junit4] 2> 85365 T115 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection [junit4] 2> 85364 T129 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection [junit4] 2> 85364 T143 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection [junit4] 2> 85376 T94 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 85377 T143 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 85377 T100 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 85376 T115 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 85376 T129 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 85391 T86 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:47311__fmo%2Fvv", [junit4] 2> "base_url":"http://127.0.0.1:47311/_fmo/vv", [junit4] 2> "operation":"state", [junit4] 2> "core":"solrj_collection_shard1_replica1", [junit4] 2> "numShards":"2", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "shard":"shard1"} [junit4] 2> 85406 T86 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:36769__fmo%2Fvv", [junit4] 2> "base_url":"http://127.0.0.1:36769/_fmo/vv", [junit4] 2> "operation":"state", [junit4] 2> "core":"solrj_collection_shard2_replica1", [junit4] 2> "numShards":"2", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "shard":"shard2"} [junit4] 2> 85412 T94 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 85414 T86 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:50093__fmo%2Fvv", [junit4] 2> "base_url":"http://127.0.0.1:50093/_fmo/vv", [junit4] 2> "operation":"state", [junit4] 2> "core":"solrj_collection_shard1_replica2", [junit4] 2> "numShards":"2", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "shard":"shard1"} [junit4] 2> 85421 T86 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:33090__fmo%2Fvv", [junit4] 2> "base_url":"http://127.0.0.1:33090/_fmo/vv", [junit4] 2> "operation":"state", [junit4] 2> "core":"solrj_collection_shard2_replica2", [junit4] 2> "numShards":"2", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "shard":"shard2"} [junit4] 2> 85425 T94 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 85428 T94 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85429 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85428 T154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85428 T95 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85428 T139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 85428 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 86380 T143 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2 [junit4] 2> 86380 T143 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 86380 T100 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2 [junit4] 2> 86381 T143 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 86381 T100 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 86381 T115 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1 [junit4] 2> 86381 T129 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1 [junit4] 2> 86381 T143 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 86382 T129 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 86382 T115 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 86382 T100 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 86384 T115 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 86383 T129 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 86383 T143 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 86385 T129 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 86385 T143 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solrj_collection_shard2_replica2/' [junit4] 2> 86384 T115 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 86384 T100 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 86386 T129 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 86387 T115 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 86387 T100 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 86387 T129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solrj_collection_shard1_replica1/' [junit4] 2> 86388 T100 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solrj_collection_shard1_replica2/' [junit4] 2> 86387 T115 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solrj_collection_shard2_replica1/' [junit4] 2> 86514 T115 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 86544 T100 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 86546 T129 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 86547 T143 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 86601 T100 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 86603 T100 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 86603 T115 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 86605 T115 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 86605 T129 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 86607 T129 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 86620 T143 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 86622 T143 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 86659 T100 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test [junit4] 2> 86669 T129 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test [junit4] 2> 86669 T115 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test [junit4] 2> 86690 T143 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test [junit4] 2> 87250 T129 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 87264 T129 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 87267 T129 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 87306 T115 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 87306 T129 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87308 T100 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 87313 T129 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87315 T115 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 87315 T143 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 87317 T115 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 87319 T100 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 87320 T129 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87322 T129 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87323 T129 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87324 T100 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 87324 T129 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87326 T129 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87327 T129 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87327 T129 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using configuration from collection solrj_collection [junit4] 2> 87328 T143 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 87328 T129 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 87329 T129 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solrj_collection_shard1_replica1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solrj_collection_shard1_replica1/data/ [junit4] 2> 87332 T143 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 87338 T129 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ebccde3 [junit4] 2> 87350 T115 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87357 T115 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87358 T100 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87361 T143 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87364 T115 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87365 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87365 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87365 T100 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87366 T115 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87367 T143 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 87368 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87368 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87369 T115 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection [junit4] 2> 87369 T115 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 87370 T115 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solrj_collection_shard2_replica1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solrj_collection_shard2_replica1/data/ [junit4] 2> 87371 T115 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ebccde3 [junit4] 2> 87373 T100 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87373 T143 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87374 T100 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87374 T143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87374 T100 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87375 T143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87375 T100 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87376 T143 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 87377 T100 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87377 T100 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87377 T143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 87378 T100 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection [junit4] 2> 87378 T143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 87379 T100 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 87379 T143 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection [junit4] 2> 87380 T100 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solrj_collection_shard1_replica2/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solrj_collection_shard1_replica2/data/ [junit4] 2> 87380 T143 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 87381 T100 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ebccde3 [junit4] 2> 87381 T143 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solrj_collection_shard2_replica2/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solrj_collection_shard2_replica2/data/ [junit4] 2> 87382 T143 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ebccde3 [junit4] 2> 87454 T115 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solrj_collection_shard2_replica1/data [junit4] 2> 87454 T115 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solrj_collection_shard2_replica1/data/index/ [junit4] 2> 87455 T129 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solrj_collection_shard1_replica1/data [junit4] 2> 87455 T100 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solrj_collection_shard1_replica2/data [junit4] 2> 87455 T115 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index... [junit4] 2> 87454 T143 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solrj_collection_shard2_replica2/data [junit4] 2> 87456 T115 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-004/solrj_collection_shard2_replica1/data/index [junit4] 2> 87456 T100 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solrj_collection_shard1_replica2/data/index/ [junit4] 2> 87455 T129 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solrj_collection_shard1_replica1/data/index/ [junit4] 2> 87458 T100 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index... [junit4] 2> 87457 T115 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=85.458984375, floorSegmentMB=0.779296875, forceMergeDeletesPctAllowed=29.731447856587874, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 87457 T143 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solrj_collection_shard2_replica2/data/index/ [junit4] 2> 87459 T100 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-003/solrj_collection_shard1_replica2/data/index [junit4] 2> 87458 T129 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index... [junit4] 2> 87460 T100 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=85.458984375, floorSegmentMB=0.779296875, forceMergeDeletesPctAllowed=29.731447856587874, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 87459 T115 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory(id=lucene-4a9b9fe6)),segFN=segments_1,generation=1} [junit4] 2> 87459 T143 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index... [junit4] 2> 87461 T115 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 87460 T129 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-005/solrj_collection_shard1_replica1/data/index [junit4] 2> 87462 T100 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory(id=lucene-1322d921)),segFN=segments_1,generation=1} [junit4] 2> 87462 T143 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001/tempDir-006/solrj_collection_shard2_replica2/data/index [junit4] 2> 87463 T100 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 87463 T129 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=85.458984375, floorSegmentMB=0.779296875, forceMergeDeletesPctAllowed=29.731447856587874, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 87464 T143 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=85.458984375, floorSegmentMB=0.779296875, forceMergeDeletesPctAllowed=29.731447856587874, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 87466 T129 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory(id=lucene-4778d17b)),segFN=segments_1,generation=1} [junit4] 2> 87466 T143 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory(id=lucene-13b83f77)),segFN=segments_1,generation=1} [junit4] 2> 87466 T129 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 87467 T143 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 87481 T115 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 87485 T100 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 87490 T129 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 87490 T143 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 87547 T115 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 87547 T143 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 87547 T129 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 87547 T100 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 87549 T129 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 8 [...truncated too long message...] collections/awholynewstresscollection_collection3_0/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_1/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection3_2/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection2_0/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection0_1/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection1_0/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_2/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection2_1/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection0_2/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_1/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_0/state.json [junit4] 2> 2 /solr/collections/awholynewcollection_5/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_2/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection0_0/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection2_2/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection1_1/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_0/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection3_1/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection1_2/state.json [junit4] 2> [junit4] 2> Maximum concurrent data watches above limit: [junit4] 2> [junit4] 2> 3 /solr/collections/awholynewcollection_3/state.json [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_4/leader_elect/shard9/election/92662336192708623-core_node26-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection1_4/leader_elect/shard3/election/92662336192708619-core_node7-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_4/leader_elect/shard7/election/92662336192708623-core_node20-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_4/leader_elect/shard8/election/92662336192708628-core_node22-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewcollection_3/leader_elect/shard3/election/92662336192708623-core_node11-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_4/leader_elect/shard5/election/92662336192708613-core_node17-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_4/leader_elect/shard4/election/92662336192708619-core_node8-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_4/leader_elect/shard6/election/92662336192708628-core_node15-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_4/leader_elect/shard4/election/92662336192708628-core_node12-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_4/leader_elect/shard1/election/92662336192708628-core_node3-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewcollection_3/leader_elect/shard1/election/92662336192708623-core_node2-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection1_4/leader_elect/shard9/election/92662336192708628-core_node23-n_0000000000 [junit4] 2> 2 /solr/collections/awholynewstresscollection_collection0_4/leader_elect/shard1/election/92662336192708628-core_node1-n_0000000000 [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 43 /solr/live_nodes [junit4] 2> 4 /solr/overseer/collection-queue-work [junit4] 2> 2 /solr/overseer/queue [junit4] 2> [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory. [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=C08FA278D17423F9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=ro_RO -Dtests.timezone=GB-Eire -Dtests.file.encoding=UTF-8 [junit4] ERROR 943s J3 | CollectionsAPIDistributedZkTest.testDistribSearch <<< [junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1 [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:569) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211) [junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583) [junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) [junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=665, name=Thread-281, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest] [junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not fully create collection: awholynewstresscollection_collection2_6 [junit4] > at __randomizedtesting.SeedInfo.seed([C08FA278D17423F9]:0) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:569) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211) [junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1635) [junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1661) [junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:1037) [junit4] 2> 1005008 T58 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-C08FA278D17423F9-001 [junit4] 2> 943178 T57 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {a_i1=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=MockRandom), _version_=PostingsFormat(name=MockRandom), text=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=MockRandom), a_t=PostingsFormat(name=MockRandom), other_tl1=PostingsFormat(name=MockRandom), id=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=Direct)}, docValues:{timestamp=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ro_RO, timezone=GB-Eire [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=204406280,total=430964736 [junit4] 2> NOTE: All tests run in this JVM: [CachingDirectoryFactoryTest, LoggingHandlerTest, HdfsLockFactoryTest, SpellCheckComponentTest, TestMergePolicyConfig, CollectionsAPIDistributedZkTest] [junit4] Completed on J3 in 943.74s, 1 test, 1 error <<< FAILURES! [...truncated 1066 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/build.xml:531: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/build.xml:472: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/build.xml:61: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/extra-targets.xml:39: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/build.xml:189: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/solr/common-build.xml:496: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/lucene/common-build.xml:1359: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk@2/lucene/common-build.xml:961: There were test failures: 437 suites, 1796 tests, 1 error, 55 ignored (10 assumptions) Total time: 203 minutes 7 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #661 Archived 5 artifacts Archive block size is 32768 Received 0 blocks and 548906413 bytes Compression is 0.0% Took 1 min 33 sec Recording test results Email was triggered for: Failure Sending email for trigger: Failure
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org