Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11134/ Java: 64bit/jdk1.9.0-ea-b28 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed. REGRESSION: org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.testDistribSearch Error Message: No live SolrServers available to handle this request:[https://127.0.0.1:46255/x_bn, https://127.0.0.1:55162/x_bn, https://127.0.0.1:52991/x_bn] Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:46255/x_bn, https://127.0.0.1:55162/x_bn, https://127.0.0.1:52991/x_bn] at __randomizedtesting.SeedInfo.seed([B92EEB02FD1E4DE3:38C8651A8A412DDF]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322) at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874) at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658) at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601) at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.removeAndWaitForLastReplicaGone(DeleteLastCustomShardedReplicaTest.java:117) at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.doTest(DeleteLastCustomShardedReplicaTest.java:107) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:484) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not remove replica : customcollreplicadeletion/a/core_node2 at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304) ... 45 more Build Log: [...truncated 14108 lines...] [junit4] Suite: org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest [junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/init-core-data-001 [junit4] 2> 2183148 T6093 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true) [junit4] 2> 2183148 T6093 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /x_bn/ [junit4] 2> 2183151 T6093 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 2183152 T6093 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 2183152 T6094 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 2183252 T6093 oasc.ZkTestServer.run start zk server on port:33243 [junit4] 2> 2183253 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2183253 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2183300 T6100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c350ba1 name:ZooKeeperConnection Watcher:127.0.0.1:33243 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2183301 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2183301 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2183301 T6093 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 2183306 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2183307 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2183308 T6102 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61333d2a name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2183308 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2183309 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2183309 T6093 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 2183312 T6093 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 2183315 T6093 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 2183316 T6093 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 2183319 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 2183319 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 2183323 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 2183324 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 2183327 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 2183330 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 2183334 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 2183335 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 2183338 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 2183341 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 2183344 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 2183345 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 2183348 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 2183349 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 2183351 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 2183352 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 2183355 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 2183356 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 2183358 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 2183359 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 2183361 T6093 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 2183362 T6093 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 2183364 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2183365 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2183365 T6104 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55b36f6a name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2183366 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2183366 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2184188 T6093 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2184192 T6093 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 2184366 T6093 oejs.AbstractConnector.doStart Started [email protected]:46255 [junit4] 2> 2184368 T6093 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2184369 T6093 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2184369 T6093 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002 [junit4] 2> 2184370 T6093 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/' [junit4] 2> 2184392 T6093 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/solr.xml [junit4] 2> 2184398 T6093 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002 [junit4] 2> 2184399 T6093 oasc.CoreContainer.<init> New CoreContainer 1881409865 [junit4] 2> 2184399 T6093 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/] [junit4] 2> 2184399 T6093 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2184399 T6093 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2184400 T6093 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2184400 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2184400 T6093 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2184400 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2184400 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2184401 T6093 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2184401 T6093 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2184402 T6093 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2184402 T6093 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2184403 T6093 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2184403 T6093 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2184403 T6093 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33243/solr [junit4] 2> 2184403 T6093 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2184403 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2184404 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2184405 T6115 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39ce5e52 name:ZooKeeperConnection Watcher:127.0.0.1:33243 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2184406 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2184406 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2184407 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2184407 T6117 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b03e202 name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2184408 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2184408 T6093 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 2184411 T6093 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 2184412 T6093 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 2184413 T6093 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 2184414 T6093 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 2184416 T6093 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 2184416 T6093 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46255_x_bn [junit4] 2> 2184417 T6093 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46255_x_bn [junit4] 2> 2184418 T6093 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 2184418 T6093 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 2184419 T6093 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2184420 T6093 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:46255_x_bn [junit4] 2> 2184421 T6093 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 2184422 T6093 oasc.Overseer.start Overseer (id=92476752476438532-127.0.0.1:46255_x_bn-n_0000000000) starting [junit4] 2> 2184423 T6093 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 2184427 T6093 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 2184428 T6119 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 2184428 T6093 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 2184433 T6093 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 2184434 T6093 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2184437 T6118 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 2184533 T6093 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002 [junit4] 2> 2184541 T6093 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/conf/ [junit4] 2> 2184677 T6093 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2184678 T6093 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1 [junit4] 2> 2184678 T6093 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2184680 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2184680 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2184682 T6123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79407e2f name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2184682 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2184682 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2184684 T6093 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2184687 T6093 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 2184687 T6093 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2 [junit4] 2> 2184688 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2184688 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2184689 T6125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e3972c2 name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2184689 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2184690 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2184691 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2184693 T6118 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1 [junit4] 2> 2184694 T6118 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2] [junit4] 2> 2184694 T6118 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2 [junit4] 2> 2184695 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json [junit4] 2> 2184696 T6126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 2184696 T6127 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 2185490 T6093 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 2185491 T6093 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2185493 T6093 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 2185494 T6093 oejs.AbstractConnector.doStart Started [email protected]:52991 [junit4] 2> 2185497 T6093 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2185498 T6093 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2185499 T6093 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003 [junit4] 2> 2185500 T6093 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/' [junit4] 2> 2185510 T6093 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/solr.xml [junit4] 2> 2185515 T6093 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003 [junit4] 2> 2185516 T6093 oasc.CoreContainer.<init> New CoreContainer 1609177901 [junit4] 2> 2185516 T6093 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/] [junit4] 2> 2185517 T6093 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2185517 T6093 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2185517 T6093 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2185517 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2185517 T6093 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2185518 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2185518 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2185518 T6093 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2185518 T6093 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2185519 T6093 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2185520 T6093 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2185520 T6093 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2185520 T6093 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2185521 T6093 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33243/solr [junit4] 2> 2185521 T6093 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2185521 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2185522 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2185523 T6138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f8dd01e name:ZooKeeperConnection Watcher:127.0.0.1:33243 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2185523 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2185523 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2185524 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2185525 T6140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a43bc36 name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2185525 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2185529 T6093 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2186532 T6093 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52991_x_bn [junit4] 2> 2186533 T6093 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52991_x_bn [junit4] 2> 2186551 T6093 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2186553 T6093 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003 [junit4] 2> 2186559 T6093 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/conf/ [junit4] 2> 2186697 T6093 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2186697 T6093 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1 [junit4] 2> 2186698 T6093 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2186945 T6093 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 2186945 T6093 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2186948 T6093 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 2186948 T6093 oejs.AbstractConnector.doStart Started [email protected]:55162 [junit4] 2> 2186949 T6093 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2186950 T6093 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2186950 T6093 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-004 [junit4] 2> 2186950 T6093 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-004/' [junit4] 2> 2186961 T6093 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-004/solr.xml [junit4] 2> 2186967 T6093 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-004 [junit4] 2> 2186967 T6093 oasc.CoreContainer.<init> New CoreContainer 163314214 [junit4] 2> 2186967 T6093 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-004/] [junit4] 2> 2186968 T6093 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2186968 T6093 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2186968 T6093 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2186969 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2186969 T6093 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2186969 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2186969 T6093 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2186969 T6093 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2186970 T6093 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2186971 T6093 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2186971 T6093 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2186971 T6093 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2186972 T6093 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2186972 T6093 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33243/solr [junit4] 2> 2186972 T6093 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2186972 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2186973 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2186974 T6153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46964bd8 name:ZooKeeperConnection Watcher:127.0.0.1:33243 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2186974 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2186974 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2186975 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2186976 T6155 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3951b18c name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2186976 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2186979 T6093 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2187981 T6093 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55162_x_bn [junit4] 2> 2187982 T6093 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55162_x_bn [junit4] 2> 2187986 T6093 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2187988 T6093 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-004 [junit4] 2> 2187993 T6093 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-004/conf/ [junit4] 2> 2188098 T6093 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2188099 T6093 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1 [junit4] 2> 2188099 T6093 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2188101 T6093 oasc.AbstractFullDistribZkTestBase.createCollection Creating collection with stateFormat=2: customcollreplicadeletion [junit4] 2> 2188102 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2188102 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2188103 T6159 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ee0996e name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2188103 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2188103 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2188104 T6093 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2188191 T6134 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=1&shards=a,b&maxShardsPerNode=5&router.name=implicit&name=customcollreplicadeletion&action=CREATE&numShards=1&stateFormat=2&wt=javabin&version=2 [junit4] 2> 2188192 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged [junit4] 2> 2188192 T6119 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{ [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"customcollreplicadeletion", [junit4] 2> "replicationFactor":"1", [junit4] 2> "numShards":"1", [junit4] 2> "maxShardsPerNode":"5", [junit4] 2> "shards":"a,b", [junit4] 2> "stateFormat":"2", [junit4] 2> "router.name":"implicit"} [junit4] 2> 2188194 T6160 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , { [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"customcollreplicadeletion", [junit4] 2> "replicationFactor":"1", [junit4] 2> "numShards":"1", [junit4] 2> "maxShardsPerNode":"5", [junit4] 2> "shards":"a,b", [junit4] 2> "stateFormat":"2", [junit4] 2> "router.name":"implicit"} [junit4] 2> 2188195 T6160 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1 [junit4] 2> 2188195 T6160 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/customcollreplicadeletion [junit4] 2> 2188195 T6160 oascc.SolrZkClient.makePath makePath: /collections/customcollreplicadeletion [junit4] 2> 2188197 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2188198 T6118 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: customcollreplicadeletion [junit4] 2> 2188198 T6118 oasc.Overseer$ClusterStateUpdater.createCollection Create collection customcollreplicadeletion with shards [a, b] [junit4] 2> 2188198 T6118 oasc.Overseer$ClusterStateUpdater.createCollection state version customcollreplicadeletion 2 [junit4] 2> 2188199 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2188200 T6141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 2188200 T6126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 2188200 T6156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 2188200 T6127 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 2188200 T6161 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 2188297 T6160 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection customcollreplicadeletion, shardNames [a, b] , replicationFactor : 1 [junit4] 2> 2188297 T6160 oasc.OverseerCollectionProcessor.createCollection Creating shard customcollreplicadeletion_a_replica1 as part of slice a of collection customcollreplicadeletion on 127.0.0.1:46255_x_bn [junit4] 2> 2188298 T6160 oasc.OverseerCollectionProcessor.createCollection Creating shard customcollreplicadeletion_b_replica1 as part of slice b of collection customcollreplicadeletion on 127.0.0.1:52991_x_bn [junit4] 2> 2188394 T6133 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=customcollreplicadeletion_b_replica1&action=CREATE&numShards=2&collection=customcollreplicadeletion&shard=b&wt=javabin&version=2 [junit4] 2> 2188396 T6133 oasc.ZkController.publish publishing core=customcollreplicadeletion_b_replica1 state=down collection=customcollreplicadeletion [junit4] 2> 2188399 T6113 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=customcollreplicadeletion_a_replica1&action=CREATE&numShards=2&collection=customcollreplicadeletion&shard=a&wt=javabin&version=2 [junit4] 2> 2188400 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2188400 T6133 oasc.ZkController.preRegister Registering watch for external collection customcollreplicadeletion [junit4] 2> 2188400 T6133 oascc.ZkStateReader.addZkWatch addZkWatch customcollreplicadeletion [junit4] 2> 2188401 T6133 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/customcollreplicadeletion/state.json from ZooKeeper... [junit4] 2> 2188401 T6133 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 0 [junit4] 2> 2188402 T6133 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 2188402 T6113 oasc.ZkController.publish publishing core=customcollreplicadeletion_a_replica1 state=down collection=customcollreplicadeletion [junit4] 2> 2188402 T6118 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "core":"customcollreplicadeletion_b_replica1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52991/x_bn";, [junit4] 2> "node_name":"127.0.0.1:52991_x_bn", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"b", [junit4] 2> "collection":"customcollreplicadeletion", [junit4] 2> "operation":"state"} [junit4] 2> 2188403 T6113 oasc.ZkController.preRegister Registering watch for external collection customcollreplicadeletion [junit4] 2> 2188413 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2188414 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2188414 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 1 [junit4] 2> 2188414 T6113 oascc.ZkStateReader.addZkWatch addZkWatch customcollreplicadeletion [junit4] 2> 2188415 T6113 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/customcollreplicadeletion/state.json from ZooKeeper... [junit4] 2> 2188416 T6113 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 1 [junit4] 2> 2188416 T6113 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 2188417 T6118 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 1 [junit4] 2> 2188418 T6118 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "core":"customcollreplicadeletion_a_replica1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:46255/x_bn";, [junit4] 2> "node_name":"127.0.0.1:46255_x_bn", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"a", [junit4] 2> "collection":"customcollreplicadeletion", [junit4] 2> "operation":"state"} [junit4] 2> 2188419 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2188419 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2188419 T6126 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2188419 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2188420 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 2 [junit4] 2> 2188420 T6126 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 2 [junit4] 2> 2189402 T6133 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for customcollreplicadeletion_b_replica1 [junit4] 2> 2189402 T6133 oasc.ZkController.createCollectionZkNode Check for collection zkNode:customcollreplicadeletion [junit4] 2> 2189403 T6133 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 2189403 T6133 oascc.ZkStateReader.readConfigName Load collection config from:/collections/customcollreplicadeletion [junit4] 2> 2189403 T6133 oascc.ZkStateReader.readConfigName path=/collections/customcollreplicadeletion configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2189404 T6133 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/customcollreplicadeletion_b_replica1/' [junit4] 2> 2189416 T6113 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for customcollreplicadeletion_a_replica1 [junit4] 2> 2189417 T6113 oasc.ZkController.createCollectionZkNode Check for collection zkNode:customcollreplicadeletion [junit4] 2> 2189417 T6113 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 2189417 T6113 oascc.ZkStateReader.readConfigName Load collection config from:/collections/customcollreplicadeletion [junit4] 2> 2189418 T6113 oascc.ZkStateReader.readConfigName path=/collections/customcollreplicadeletion configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2189418 T6113 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/customcollreplicadeletion_a_replica1/' [junit4] 2> 2189422 T6133 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0 [junit4] 2> 2189434 T6133 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 2189435 T6133 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 2189435 T6113 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0 [junit4] 2> 2189445 T6133 oass.IndexSchema.readSchema [customcollreplicadeletion_b_replica1] Schema name=test [junit4] 2> 2189452 T6113 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 2189453 T6113 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 2189457 T6113 oass.IndexSchema.readSchema [customcollreplicadeletion_a_replica1] Schema name=test [junit4] 2> 2189551 T6133 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 2189554 T6133 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 2189556 T6133 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 2189561 T6133 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2189564 T6133 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2189565 T6133 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2189566 T6133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2189566 T6133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2189566 T6133 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2189567 T6133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2189567 T6133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2189567 T6133 oasc.CoreContainer.create Creating SolrCore 'customcollreplicadeletion_b_replica1' using configuration from collection customcollreplicadeletion [junit4] 2> 2189567 T6133 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 2189568 T6133 oasc.SolrCore.<init> [customcollreplicadeletion_b_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/customcollreplicadeletion_b_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/customcollreplicadeletion_b_replica1/data/ [junit4] 2> 2189568 T6133 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@231b3301 [junit4] 2> 2189568 T6133 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/customcollreplicadeletion_b_replica1/data [junit4] 2> 2189569 T6133 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/customcollreplicadeletion_b_replica1/data/index/ [junit4] 2> 2189569 T6133 oasc.SolrCore.initIndex WARN [customcollreplicadeletion_b_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/customcollreplicadeletion_b_replica1/data/index' doesn't exist. Creating new index... [junit4] 2> 2189569 T6133 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-003/customcollreplicadeletion_b_replica1/data/index [junit4] 2> 2189570 T6113 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 2189570 T6133 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0] [junit4] 2> 2189571 T6133 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-4ab2441a)),segFN=segments_1,generation=1} [junit4] 2> 2189571 T6133 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 2189573 T6113 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 2189573 T6133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 2189573 T6133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 2189574 T6133 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 2189574 T6133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 2189574 T6133 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 2189574 T6113 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 2189574 T6133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 2189575 T6133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 2189575 T6133 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 2189575 T6133 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 2189576 T6133 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2189576 T6133 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2189576 T6133 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2189577 T6133 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 2189577 T6133 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4] 2> 2189577 T6133 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4] 2> 2189578 T6133 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 2189578 T6133 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 2189578 T6133 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4] 2> 2189579 T6133 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 2189579 T6133 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 2189580 T6113 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2189582 T6133 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189582 T6113 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2189583 T6133 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189584 T6113 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2189584 T6133 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189584 T6113 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2189584 T6113 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2189585 T6113 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2189585 T6133 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189585 T6113 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2189585 T6113 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2189586 T6113 oasc.CoreContainer.create Creating SolrCore 'customcollreplicadeletion_a_replica1' using configuration from collection customcollreplicadeletion [junit4] 2> 2189586 T6113 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 2189586 T6113 oasc.SolrCore.<init> [customcollreplicadeletion_a_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/customcollreplicadeletion_a_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/customcollreplicadeletion_a_replica1/data/ [junit4] 2> 2189587 T6113 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@231b3301 [junit4] 2> 2189587 T6133 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 2189587 T6133 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 2189588 T6113 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/customcollreplicadeletion_a_replica1/data [junit4] 2> 2189588 T6113 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/customcollreplicadeletion_a_replica1/data/index/ [junit4] 2> 2189588 T6133 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=954583276, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1] [junit4] 2> 2189588 T6113 oasc.SolrCore.initIndex WARN [customcollreplicadeletion_a_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/customcollreplicadeletion_a_replica1/data/index' doesn't exist. Creating new index... [junit4] 2> 2189589 T6133 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-4ab2441a)),segFN=segments_1,generation=1} [junit4] 2> 2189589 T6133 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 2189589 T6113 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001/tempDir-002/customcollreplicadeletion_a_replica1/data/index [junit4] 2> 2189589 T6133 oass.SolrIndexSearcher.<init> Opening Searcher@d7aaad1[customcollreplicadeletion_b_replica1] main [junit4] 2> 2189590 T6113 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0] [junit4] 2> 2189590 T6133 oascc.ZkStateReader.readConfigName Load collection config from:/collections/customcollreplicadeletion [junit4] 2> 2189590 T6113 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-687be81)),segFN=segments_1,generation=1} [junit4] 2> 2189591 T6113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 2189591 T6133 oascc.ZkStateReader.readConfigName path=/collections/customcollreplicadeletion configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2189591 T6133 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 2189592 T6133 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 2189592 T6133 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 2189592 T6133 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2189593 T6133 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 2189593 T6133 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2189593 T6113 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 2189593 T6133 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json [junit4] 2> 2189594 T6113 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 2189594 T6113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 2189594 T6113 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 2189594 T6113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 2189594 T6133 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json [junit4] 2> 2189595 T6133 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2189595 T6113 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 2189595 T6133 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 2189596 T6113 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 2189596 T6113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 2189596 T6113 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 2189597 T6113 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2189597 T6113 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2189597 T6113 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2189598 T6113 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 2189598 T6164 oasc.SolrCore.registerSearcher [customcollreplicadeletion_b_replica1] Registered new searcher Searcher@d7aaad1[customcollreplicadeletion_b_replica1] main{UninvertingDirectoryReader()} [junit4] 2> 2189598 T6113 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4] 2> 2189598 T6113 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4] 2> 2189598 T6113 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 2189599 T6113 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 2189599 T6113 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4] 2> 2189599 T6133 oasc.CoreContainer.registerCore registering core: customcollreplicadeletion_b_replica1 [junit4] 2> 2189599 T6113 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 2189600 T6133 oasc.ZkController.register Register replica - core:customcollreplicadeletion_b_replica1 address:https://127.0.0.1:52991/x_bn collection:customcollreplicadeletion shard:b [junit4] 2> 2189600 T6113 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 2189600 T6133 oascc.SolrZkClient.makePath makePath: /collections/customcollreplicadeletion/leader_elect/b/election [junit4] 2> 2189601 T6113 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189602 T6113 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189603 T6113 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189603 T6133 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard b [junit4] 2> 2189603 T6113 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2189604 T6133 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 2189604 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189605 T6133 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C1885 name=customcollreplicadeletion_b_replica1 org.apache.solr.core.SolrCore@7494099 url=https://127.0.0.1:52991/x_bn/customcollreplicadeletion_b_replica1 node=127.0.0.1:52991_x_bn C1885_STATE=coll:customcollreplicadeletion core:customcollreplicadeletion_b_replica1 props:{core=customcollreplicadeletion_b_replica1, base_url=https://127.0.0.1:52991/x_bn, node_name=127.0.0.1:52991_x_bn, state=down} [junit4] 2> 2189605 T6133 C1885 P52991 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52991/x_bn/customcollreplicadeletion_b_replica1/ [junit4] 2> 2189605 T6133 C1885 P52991 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 2189606 T6133 C1885 P52991 oasc.SyncStrategy.syncToMe https://127.0.0.1:52991/x_bn/customcollreplicadeletion_b_replica1/ has no replicas [junit4] 2> 2189606 T6133 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52991/x_bn/customcollreplicadeletion_b_replica1/ b [junit4] 2> 2189606 T6133 oascc.SolrZkClient.makePath makePath: /collections/customcollreplicadeletion/leaders/b [junit4] 2> 2189607 T6118 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 2 [junit4] 2> 2189607 T6113 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 2189608 T6113 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 2189609 T6113 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=954583276, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1] [junit4] 2> 2189610 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2189610 T6113 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-687be81)),segFN=segments_1,generation=1} [junit4] 2> 2189610 T6113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 2189610 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189610 T6126 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189610 T6113 oass.SolrIndexSearcher.<init> Opening Searcher@6020c691[customcollreplicadeletion_a_replica1] main [junit4] 2> 2189611 T6113 oascc.ZkStateReader.readConfigName Load collection config from:/collections/customcollreplicadeletion [junit4] 2> 2189611 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 3 [junit4] 2> 2189612 T6113 oascc.ZkStateReader.readConfigName path=/collections/customcollreplicadeletion configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2189612 T6113 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 2189612 T6126 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 3 [junit4] 2> 2189612 T6113 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 2189613 T6113 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 2189613 T6113 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2189613 T6113 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json [junit4] 2> 2189614 T6113 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2189614 T6118 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 3 [junit4] 2> 2189614 T6113 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed [junit4] 2> 2189614 T6113 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 2189615 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189615 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2189616 T6126 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189616 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189616 T6166 oasc.SolrCore.registerSearcher [customcollreplicadeletion_a_replica1] Registered new searcher Searcher@6020c691[customcollreplicadeletion_a_replica1] main{UninvertingDirectoryReader()} [junit4] 2> 2189616 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 4 [junit4] 2> 2189617 T6126 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 4 [junit4] 2> 2189617 T6113 oasc.CoreContainer.registerCore registering core: customcollreplicadeletion_a_replica1 [junit4] 2> 2189618 T6113 oasc.ZkController.register Register replica - core:customcollreplicadeletion_a_replica1 address:https://127.0.0.1:46255/x_bn collection:customcollreplicadeletion shard:a [junit4] 2> 2189618 T6113 oascc.SolrZkClient.makePath makePath: /collections/customcollreplicadeletion/leader_elect/a/election [junit4] 2> 2189620 T6113 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard a [junit4] 2> 2189621 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189621 T6113 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 2189621 T6113 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C1886 name=customcollreplicadeletion_a_replica1 org.apache.solr.core.SolrCore@48420e70 url=https://127.0.0.1:46255/x_bn/customcollreplicadeletion_a_replica1 node=127.0.0.1:46255_x_bn C1886_STATE=coll:customcollreplicadeletion core:customcollreplicadeletion_a_replica1 props:{core=customcollreplicadeletion_a_replica1, base_url=https://127.0.0.1:46255/x_bn, node_name=127.0.0.1:46255_x_bn, state=down} [junit4] 2> 2189622 T6113 C1886 P46255 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:46255/x_bn/customcollreplicadeletion_a_replica1/ [junit4] 2> 2189622 T6113 C1886 P46255 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 2189622 T6113 C1886 P46255 oasc.SyncStrategy.syncToMe https://127.0.0.1:46255/x_bn/customcollreplicadeletion_a_replica1/ has no replicas [junit4] 2> 2189622 T6113 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:46255/x_bn/customcollreplicadeletion_a_replica1/ a [junit4] 2> 2189623 T6113 oascc.SolrZkClient.makePath makePath: /collections/customcollreplicadeletion/leaders/a [junit4] 2> 2189623 T6118 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 4 [junit4] 2> 2189624 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2189625 T6126 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189625 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189625 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 5 [junit4] 2> 2189626 T6126 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 5 [junit4] 2> 2189627 T6118 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 5 [junit4] 2> 2189628 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189628 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2189629 T6126 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189629 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189629 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 6 [junit4] 2> 2189630 T6126 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 6 [junit4] 2> 2189661 T6133 oasc.ZkController.register We are https://127.0.0.1:52991/x_bn/customcollreplicadeletion_b_replica1/ and leader is https://127.0.0.1:52991/x_bn/customcollreplicadeletion_b_replica1/ [junit4] 2> 2189662 T6133 oasc.ZkController.register No LogReplay needed for core=customcollreplicadeletion_b_replica1 baseURL=https://127.0.0.1:52991/x_bn [junit4] 2> 2189662 T6133 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 2189662 T6133 oasc.ZkController.publish publishing core=customcollreplicadeletion_b_replica1 state=active collection=customcollreplicadeletion [junit4] 2> 2189664 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189666 T6133 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 6 [junit4] 2> 2189667 T6118 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 6 [junit4] 2> 2189667 T6118 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "core":"customcollreplicadeletion_b_replica1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52991/x_bn";, [junit4] 2> "node_name":"127.0.0.1:52991_x_bn", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"b", [junit4] 2> "collection":"customcollreplicadeletion", [junit4] 2> "operation":"state"} [junit4] 2> 2189667 T6133 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=customcollreplicadeletion_b_replica1&action=CREATE&numShards=2&collection=customcollreplicadeletion&shard=b&wt=javabin&version=2} status=0 QTime=1273 [junit4] 2> 2189669 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2189669 T6126 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189669 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189670 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 7 [junit4] 2> 2189671 T6126 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 7 [junit4] 2> 2189675 T6113 oasc.ZkController.register We are https://127.0.0.1:46255/x_bn/customcollreplicadeletion_a_replica1/ and leader is https://127.0.0.1:46255/x_bn/customcollreplicadeletion_a_replica1/ [junit4] 2> 2189676 T6113 oasc.ZkController.register No LogReplay needed for core=customcollreplicadeletion_a_replica1 baseURL=https://127.0.0.1:46255/x_bn [junit4] 2> 2189676 T6113 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 2189676 T6113 oasc.ZkController.publish publishing core=customcollreplicadeletion_a_replica1 state=active collection=customcollreplicadeletion [junit4] 2> 2189677 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189679 T6113 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 7 [junit4] 2> 2189680 T6113 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=customcollreplicadeletion_a_replica1&action=CREATE&numShards=2&collection=customcollreplicadeletion&shard=a&wt=javabin&version=2} status=0 QTime=1281 [junit4] 2> 2189681 T6160 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: customcollreplicadeletion [junit4] 2> 2189681 T6118 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 7 [junit4] 2> 2189682 T6160 oasc.OverseerCollectionProcessor$Runner.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=1273},core=customcollreplicadeletion_b_replica1},null={responseHeader={status=0,QTime=1281},core=customcollreplicadeletion_a_replica1}}} [junit4] 2> 2189683 T6141 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged [junit4] 2> 2189683 T6118 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "core":"customcollreplicadeletion_a_replica1", [junit4] 2> "core_node_name":"core_node2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:46255/x_bn";, [junit4] 2> "node_name":"127.0.0.1:46255_x_bn", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"a", [junit4] 2> "collection":"customcollreplicadeletion", [junit4] 2> "operation":"state"} [junit4] 2> 2189684 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189684 T6134 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={replicationFactor=1&shards=a,b&maxShardsPerNode=5&router.name=implicit&name=customcollreplicadeletion&action=CREATE&numShards=1&stateFormat=2&wt=javabin&version=2} status=0 QTime=1493 [junit4] 2> 2189685 T6118 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/customcollreplicadeletion/state.json [junit4] 2> 2189685 T6126 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189685 T6093 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: customcollreplicadeletion failOnTimeout:true timeout (sec):330 [junit4] 2> 2189685 T6141 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/customcollreplicadeletion/state.json, has occurred - updating... [junit4] 2> 2189686 T6141 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 8 [junit4] 2> 2189686 T6126 oascc.ZkStateReader.updateWatchedCollection Updating data for customcollreplicadeletion to ver 8 [junit4] 2> 2189687 T6093 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: customcollreplicadeletion [junit4] 2> 2189688 T6093 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2189689 T6093 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2189689 T6169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c1fa9d5 name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2189690 T6093 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2189690 T6093 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2189690 T6093 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2189696 T6134 oasha.CollectionsHandler.handleRemoveReplica Remove replica: replica=core_node2&action=deletereplica&collection=customcollreplicadeletion&shard=a&wt=javabin&version=2 [junit4] 2> 2189697 T6126 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged [junit4] 2> 2189698 T6119 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{ [junit4] 2> "operation":"deletereplica", [junit4] 2> "collection":"customcollreplicadeletion", [junit4] 2> "shard":"a", [junit4] 2> "replica":"core_node2"} [junit4] 2> 2189698 T6170 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : deletereplica , { [junit4] 2> "operation":"deletereplica", [junit4] 2> "collection":"customcollreplicadeletion", [junit4] 2> "shard":"a", [junit4] 2> "replica":"core_node2"} [junit4] 2> 2189760 T6112 oasc.SolrCore.close [customcollreplicadeletion_a_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@48420e70 [junit4] 2> 2189765 T6112 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0} [junit4] 2> 2189765 T6112 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState [junit4] 2> 2189765 T6112 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter [junit4] 2> 2189765 T6112 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWr [...truncated too long message...] entCnxn$EventThread.processEvent(ClientCnxn.java:522) [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) [junit4] 2> [junit4] 2> 2313935 T6159 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@38e644c2 rejected from java.util.concurrent.ThreadPoolExecutor@9a58fe8[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 5] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2049) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) [junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:263) [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522) [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) [junit4] 2> [junit4] 2> 2313936 T6159 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ee0996e name:ZooKeeperConnection Watcher:127.0.0.1:33243/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None [junit4] 2> 2313936 T6159 oascc.ConnectionManager.process Client->ZooKeeper status change trigger but we are already closed [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DeleteLastCustomShardedReplicaTest -Dtests.method=testDistribSearch -Dtests.seed=B92EEB02FD1E4DE3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_JO -Dtests.timezone=Asia/Pontianak -Dtests.file.encoding=US-ASCII [junit4] ERROR 131s J1 | DeleteLastCustomShardedReplicaTest.testDistribSearch <<< [junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:46255/x_bn, https://127.0.0.1:55162/x_bn, https://127.0.0.1:52991/x_bn] [junit4] > at __randomizedtesting.SeedInfo.seed([B92EEB02FD1E4DE3:38C8651A8A412DDF]:0) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601) [junit4] > at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.removeAndWaitForLastReplicaGone(DeleteLastCustomShardedReplicaTest.java:117) [junit4] > at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.doTest(DeleteLastCustomShardedReplicaTest.java:107) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not remove replica : customcollreplicadeletion/a/core_node2 [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304) [junit4] > ... 45 more [junit4] 2> 2313953 T6093 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-B92EEB02FD1E4DE3-001 [junit4] 2> 130809 T6092 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=ar_JO, timezone=Asia/Pontianak [junit4] 2> NOTE: Linux 3.13.0-35-generic amd64/Oracle Corporation 1.9.0-ea (64-bit)/cpus=8,threads=1,free=330974048,total=472383488 [junit4] 2> NOTE: All tests run in this JVM: [ModifyConfFileTest, TestQueryUtils, TestLFUCache, DistributedIntervalFacetingTest, CursorMarkTest, TestReversedWildcardFilterFactory, RequestHandlersTest, SharedFSAutoReplicaFailoverUtilsTest, CircularListTest, CoreMergeIndexesAdminHandlerTest, DistributedFacetPivotLongTailTest, TestCSVLoader, QueryElevationComponentTest, BlockCacheTest, HdfsUnloadDistributedZkTest, LukeRequestHandlerTest, CustomCollectionTest, OverriddenZkACLAndCredentialsProvidersTest, CoreAdminCreateDiscoverTest, TestIndexSearcher, TestShortCircuitedRequests, TestPseudoReturnFields, LeaderElectionIntegrationTest, TestIndexingPerformance, TestGroupingSearch, TestSchemaSimilarityResource, TestBM25SimilarityFactory, TestDistributedGrouping, BJQParserTest, TestBinaryField, DistanceFunctionTest, UUIDUpdateProcessorFallbackTest, RankQueryTest, MergeStrategyTest, CursorPagingTest, DeleteInactiveReplicaTest, StressHdfsTest, SolrTestCaseJ4Test, HdfsSyncSliceTest, ChaosMonkeySafeLeaderTest, OpenCloseCoreStressTest, TestQuerySenderNoQuery, TestMissingGroups, QueryParsingTest, DistribCursorPagingTest, FieldAnalysisRequestHandlerTest, TestBlendedInfixSuggestions, DirectUpdateHandlerOptimizeTest, TestSchemaResource, PolyFieldTest, HdfsBasicDistributedZkTest, CSVRequestHandlerTest, SoftAutoCommitTest, DirectSolrConnectionTest, DocValuesTest, AsyncMigrateRouteKeyTest, SchemaVersionSpecificBehaviorTest, RemoteQueryErrorTest, TestSolrQueryParserResource, TestNonNRTOpen, TestExceedMaxTermLength, DistributedDebugComponentTest, TestCSVResponseWriter, TestOrdValues, TestExpandComponent, HighlighterMaxOffsetTest, SolrInfoMBeanTest, SortByFunctionTest, TestSearcherReuse, ShardSplitTest, TestMultiCoreConfBootstrap, TestBadConfig, TestPerFieldSimilarity, DateFieldTest, StatsComponentTest, LoggingHandlerTest, BinaryUpdateRequestHandlerTest, PreAnalyzedFieldTest, ClusterStateTest, HttpPartitionTest, SolrPluginUtilsTest, FullSolrCloudDistribCmdsTest, AddSchemaFieldsUpdateProcessorFactoryTest, RAMDirectoryFactoryTest, ChangedSchemaMergeTest, TestZkChroot, TestJmxMonitoredMap, PreAnalyzedUpdateProcessorTest, TestCopyFieldCollectionResource, LegacyHTMLStripCharFilterTest, TestMergePolicyConfig, TestUtils, PrimUtilsTest, SliceStateTest, SimplePostToolTest, TestRealTimeGet, TestFieldSortValues, TestCloudManagedSchemaConcurrent, BasicDistributedZk2Test, DistributedFacetPivotSmallTest, TestSolrQueryParser, TestManagedResource, CollectionsAPIDistributedZkTest, BasicZkTest, RecoveryZkTest, ZkSolrClientTest, TestRandomDVFaceting, TestDistributedSearch, TermVectorComponentDistributedTest, HardAutoCommitTest, DistributedTermsComponentTest, PeerSyncTest, TestSort, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, QueryEqualityTest, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, FastVectorHighlighterTest, RegexBoostProcessorTest, ReturnFieldsTest, JSONWriterTest, UniqFieldsUpdateProcessorFactoryTest, SearchHandlerTest, SOLR749Test, UpdateParamsTest, CopyFieldTest, SolrIndexConfigTest, MultiTermTest, TestDocSet, NumericFieldsTest, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, ScriptEngineTest, DateMathParserTest, ZkNodePropsTest, FileUtilsTest, TestDistributedMissingSort, AssignTest, DeleteLastCustomShardedReplicaTest] [junit4] Completed on J1 in 131.34s, 1 test, 1 error <<< FAILURES! [...truncated 245 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:491: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:471: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:60: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:189: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:496: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1358: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:961: There were test failures: 429 suites, 1772 tests, 1 error, 45 ignored (23 assumptions) Total time: 91 minutes 43 seconds Build step 'Invoke Ant' marked build as failure [description-setter] Description set: Java: 64bit/jdk1.9.0-ea-b28 -XX:-UseCompressedOops -XX:+UseParallelGC Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
