Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4849/
7 tests failed. REGRESSION: org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch Error Message: No live SolrServers available to handle this request:[https://127.0.0.1:31541/ln/ky, https://127.0.0.1:31530/ln/ky, https://127.0.0.1:31511/ln/ky, https://127.0.0.1:31538/ln/ky, https://127.0.0.1:31519/ln/ky] Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:31541/ln/ky, https://127.0.0.1:31530/ln/ky, https://127.0.0.1:31511/ln/ky, https://127.0.0.1:31538/ln/ky, https://127.0.0.1:31519/ln/ky] at __randomizedtesting.SeedInfo.seed([293DA6A76D018679:A8DB28BF1A5EE645]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322) at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874) at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658) at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601) at org.apache.solr.cloud.DeleteReplicaTest.removeAndWaitForReplicaGone(DeleteReplicaTest.java:139) at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:123) at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:87) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not remove replica : delLiveColl/shard2/core_node4 at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304) ... 47 more FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot Error Message: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=5070, name=coreLoadExecutor-1409-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below. Stack Trace: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=5070, name=coreLoadExecutor-1409-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below. at java.lang.Thread.getStackTrace(Thread.java:1589) at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166) at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731) at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157) at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@146e48e8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d056f74) at org.junit.Assert.fail(Assert.java:93) at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696) at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) ... 1 more FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot Error Message: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=5088, name=coreLoadExecutor-1416-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below. Stack Trace: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=5088, name=coreLoadExecutor-1416-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below. at java.lang.Thread.getStackTrace(Thread.java:1589) at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166) at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731) at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157) at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@400e58fc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@664933d7) at org.junit.Assert.fail(Assert.java:93) at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696) at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) ... 1 more FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot Error Message: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=5104, name=coreLoadExecutor-1423-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below. Stack Trace: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=5104, name=coreLoadExecutor-1423-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below. at java.lang.Thread.getStackTrace(Thread.java:1589) at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166) at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731) at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157) at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@31932b4c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4fea9cb7) at org.junit.Assert.fail(Assert.java:93) at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696) at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) ... 1 more REGRESSION: org.apache.solr.core.CachingDirectoryFactoryTest.stressTest Error Message: Captured an uncaught exception in thread: Thread[id=3386, name=Thread-1482, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3386, name=Thread-1482, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest] Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path3" "write") at __randomizedtesting.SeedInfo.seed([293DA6A76D018679]:0) at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372) at java.security.AccessController.checkPermission(AccessController.java:559) at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801) at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376) at java.nio.file.Files.createDirectory(Files.java:630) at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734) at java.nio.file.Files.createDirectories(Files.java:683) at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46) at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81) at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357) at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190) REGRESSION: org.apache.solr.core.RAMDirectoryFactoryTest.test Error Message: access denied ("java.io.FilePermission" "/fake/path" "write") Stack Trace: java.security.AccessControlException: access denied ("java.io.FilePermission" "/fake/path" "write") at __randomizedtesting.SeedInfo.seed([293DA6A76D018679:A169997DC3FDEB81]:0) at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372) at java.security.AccessController.checkPermission(AccessController.java:559) at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801) at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376) at java.nio.file.Files.createDirectory(Files.java:630) at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734) at java.nio.file.Files.createDirectories(Files.java:683) at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46) at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81) at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357) at org.apache.solr.core.RAMDirectoryFactoryTest.dotestOpenReturnsTheSameForSamePath(RAMDirectoryFactoryTest.java:46) at org.apache.solr.core.RAMDirectoryFactoryTest.test(RAMDirectoryFactoryTest.java:33) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.testDistribSearch Error Message: No live SolrServers available to handle this request:[https://127.0.0.1:27625, https://127.0.0.1:27620, https://127.0.0.1:27628] Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:27625, https://127.0.0.1:27620, https://127.0.0.1:27628] at __randomizedtesting.SeedInfo.seed([293DA6A76D018679:A8DB28BF1A5EE645]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322) at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874) at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658) at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601) at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.removeAndWaitForLastReplicaGone(DeleteLastCustomShardedReplicaTest.java:116) at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.doTest(DeleteLastCustomShardedReplicaTest.java:106) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not remove replica : customcollreplicadeletion/a/core_node2 at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304) ... 46 more Build Log: [...truncated 11520 lines...] [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest [junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/init-core-data-001 [junit4] 2> 486022 T1841 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true) [junit4] 2> 486023 T1841 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ln/ky [junit4] 2> 486030 T1841 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 486031 T1841 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 486031 T1842 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 486131 T1841 oasc.ZkTestServer.run start zk server on port:31505 [junit4] 2> 486132 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 486133 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 486172 T1848 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45684e80 name:ZooKeeperConnection Watcher:127.0.0.1:31505 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 486173 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 486173 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 486174 T1841 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 486176 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 486178 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 486179 T1850 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17d6c198 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 486180 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 486180 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 486181 T1841 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 486184 T1841 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 486187 T1841 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 486189 T1841 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 486192 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 486192 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 486196 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 486197 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 486201 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 486202 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 486205 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 486206 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 486209 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 486210 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 486212 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 486213 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 486216 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 486217 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 486219 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 486220 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 486223 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 486224 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 486227 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 486228 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 486231 T1841 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 486232 T1841 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 486236 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 486237 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 486239 T1852 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1095ef5 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 486239 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 486240 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 487947 T1841 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 487951 T1841 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 487956 T1841 oejs.AbstractConnector.doStart Started [email protected]:31511 [junit4] 2> 487958 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 487958 T1841 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 487959 T1841 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-002 [junit4] 2> 487959 T1841 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-002/' [junit4] 2> 487989 T1841 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-002/solr.xml [junit4] 2> 488042 T1841 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-002 [junit4] 2> 488042 T1841 oasc.CoreContainer.<init> New CoreContainer 726849363 [junit4] 2> 488042 T1841 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-002/] [junit4] 2> 488043 T1841 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 488043 T1841 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 488044 T1841 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 488044 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 488044 T1841 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 488044 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 488045 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 488045 T1841 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 488045 T1841 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 488047 T1841 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 488049 T1841 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 488049 T1841 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 488049 T1841 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 488050 T1841 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:31505/solr [junit4] 2> 488050 T1841 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 488050 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 488051 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 488052 T1863 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11f5835d name:ZooKeeperConnection Watcher:127.0.0.1:31505 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 488053 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 488053 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 488055 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 488056 T1865 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41d98752 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 488056 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 488057 T1841 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 488059 T1841 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 488060 T1841 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 488062 T1841 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 488063 T1841 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 488065 T1841 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 488066 T1841 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31511_ln%2Fky [junit4] 2> 488067 T1841 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31511_ln%2Fky [junit4] 2> 488068 T1841 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 488069 T1841 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 488070 T1841 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 488071 T1841 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:31511_ln%2Fky [junit4] 2> 488072 T1841 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 488073 T1841 oasc.Overseer.start Overseer (id=92449841279729668-127.0.0.1:31511_ln%2Fky-n_0000000000) starting [junit4] 2> 488074 T1841 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 488079 T1841 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 488080 T1867 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 488081 T1841 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 488082 T1841 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 488083 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 488085 T1866 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 488087 T1841 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-002 [junit4] 2> 488261 T1841 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-002/conf/ [junit4] 2> 489163 T1841 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 489164 T1841 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3 [junit4] 2> 489164 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 489168 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 489168 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 489169 T1871 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70ae06b6 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 489169 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 489170 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 489170 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 489172 T1841 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 489172 T1841 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2 [junit4] 2> 489172 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 489173 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 489173 T1873 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@502a8edf name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 489174 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 489174 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 489176 T1874 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 489177 T1866 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1 [junit4] 2> 489178 T1866 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2] [junit4] 2> 489178 T1866 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2 [junit4] 2> 489180 T1866 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json [junit4] 2> 489181 T1874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 489181 T1875 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 490357 T1841 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 490358 T1841 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 490362 T1841 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 490364 T1841 oejs.AbstractConnector.doStart Started [email protected]:31519 [junit4] 2> 490366 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 490366 T1841 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 490367 T1841 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-003 [junit4] 2> 490367 T1841 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-003/' [junit4] 2> 490408 T1841 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-003/solr.xml [junit4] 2> 490464 T1841 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-003 [junit4] 2> 490464 T1841 oasc.CoreContainer.<init> New CoreContainer 1352746685 [junit4] 2> 490465 T1841 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-003/] [junit4] 2> 490465 T1841 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 490466 T1841 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 490466 T1841 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 490466 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 490466 T1841 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 490467 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 490467 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 490467 T1841 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 490467 T1841 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 490470 T1841 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 490471 T1841 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 490472 T1841 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 490472 T1841 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 490472 T1841 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:31505/solr [junit4] 2> 490472 T1841 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 490473 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 490473 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 490475 T1886 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@249905e4 name:ZooKeeperConnection Watcher:127.0.0.1:31505 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 490476 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 490476 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 490478 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 490480 T1888 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7469bea5 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 490480 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 490484 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 491490 T1841 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31519_ln%2Fky [junit4] 2> 491491 T1841 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31519_ln%2Fky [junit4] 2> 491495 T1841 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 491499 T1841 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-003 [junit4] 2> 491503 T1841 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-003/conf/ [junit4] 2> 492625 T1841 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 492626 T1841 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3 [junit4] 2> 492626 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 493877 T1841 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 493878 T1841 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 493883 T1841 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 493884 T1841 oejs.AbstractConnector.doStart Started [email protected]:31530 [junit4] 2> 493887 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 493888 T1841 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 493888 T1841 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-004 [junit4] 2> 493888 T1841 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-004/' [junit4] 2> 493921 T1841 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-004/solr.xml [junit4] 2> 493969 T1841 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-004 [junit4] 2> 493969 T1841 oasc.CoreContainer.<init> New CoreContainer 1689428548 [junit4] 2> 493969 T1841 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-004/] [junit4] 2> 493970 T1841 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 493970 T1841 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 493971 T1841 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 493971 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 493971 T1841 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 493971 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 493972 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 493972 T1841 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 493972 T1841 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 493974 T1841 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 493976 T1841 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 493976 T1841 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 493976 T1841 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 493976 T1841 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:31505/solr [junit4] 2> 493977 T1841 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 493977 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 493978 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 493979 T1901 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@295f057c name:ZooKeeperConnection Watcher:127.0.0.1:31505 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 493980 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 493980 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 493982 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 493982 T1903 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3eaea4ce name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 493983 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 493986 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 494991 T1841 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31530_ln%2Fky [junit4] 2> 494992 T1841 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31530_ln%2Fky [junit4] 2> 494995 T1841 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 494999 T1841 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-004 [junit4] 2> 496751 T1841 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-004/conf/ [junit4] 2> 496777 T1841 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 496779 T1841 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3 [junit4] 2> 496779 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 498223 T1841 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3 [junit4] 2> 498224 T1841 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 498228 T1841 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 498230 T1841 oejs.AbstractConnector.doStart Started [email protected]:31538 [junit4] 2> 498232 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 498232 T1841 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 498232 T1841 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-005 [junit4] 2> 498233 T1841 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-005/' [junit4] 2> 498261 T1841 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-005/solr.xml [junit4] 2> 498303 T1841 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-005 [junit4] 2> 498303 T1841 oasc.CoreContainer.<init> New CoreContainer 962354436 [junit4] 2> 498303 T1841 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-005/] [junit4] 2> 498304 T1841 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 498305 T1841 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 498305 T1841 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 498305 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 498305 T1841 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 498305 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 498306 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 498306 T1841 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 498306 T1841 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 498308 T1841 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 498310 T1841 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 498310 T1841 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 498310 T1841 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 498311 T1841 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:31505/solr [junit4] 2> 498311 T1841 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 498311 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 498312 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 498313 T1916 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23b8710a name:ZooKeeperConnection Watcher:127.0.0.1:31505 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 498314 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 498314 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 498316 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 498316 T1918 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76c834e7 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 498317 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 498320 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 499324 T1841 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31538_ln%2Fky [junit4] 2> 499325 T1841 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31538_ln%2Fky [junit4] 2> 499329 T1841 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 499333 T1841 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-005 [junit4] 2> 499336 T1841 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-005/conf/ [junit4] 2> 500394 T1841 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 500396 T1841 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3 [junit4] 2> 500396 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 501783 T1841 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4 [junit4] 2> 501783 T1841 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 501788 T1841 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 501789 T1841 oejs.AbstractConnector.doStart Started [email protected]:31541 [junit4] 2> 501792 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 501792 T1841 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 501792 T1841 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-006 [junit4] 2> 501793 T1841 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-006/' [junit4] 2> 501833 T1841 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-006/solr.xml [junit4] 2> 501891 T1841 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-006 [junit4] 2> 501891 T1841 oasc.CoreContainer.<init> New CoreContainer 790565428 [junit4] 2> 501892 T1841 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-006/] [junit4] 2> 501893 T1841 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 501893 T1841 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 501893 T1841 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 501893 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 501894 T1841 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 501894 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 501894 T1841 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 501894 T1841 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 501894 T1841 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 501897 T1841 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 501898 T1841 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 501899 T1841 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 501899 T1841 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 501899 T1841 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:31505/solr [junit4] 2> 501899 T1841 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 501899 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 501900 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 501901 T1931 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8d93987 name:ZooKeeperConnection Watcher:127.0.0.1:31505 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 501902 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 501902 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 501904 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 502138 T1933 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d9199a3 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 502138 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 502142 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 503145 T1841 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31541_ln%2Fky [junit4] 2> 503146 T1841 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31541_ln%2Fky [junit4] 2> 503150 T1841 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 503155 T1841 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-006 [junit4] 2> 503166 T1841 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.DeleteReplicaTest-293DA6A76D018679-001/tempDir-006/conf/ [junit4] 2> 504490 T1841 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 504491 T1841 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J3 [junit4] 2> 504491 T1841 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 504498 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 504498 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 504499 T1937 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@65ec88db name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 504500 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 504500 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 504501 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 504502 T1841 oasc.AbstractFullDistribZkTestBase.createCollection Creating collection with stateFormat=2: delLiveColl [junit4] 2> 504503 T1841 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 504503 T1841 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 504504 T1939 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a9bd495 name:ZooKeeperConnection Watcher:127.0.0.1:31505/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 504504 T1841 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 504504 T1841 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 504506 T1841 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 504633 T1878 oasha.CollectionsHandler.handleCreateAction Creating Collection : version=2&action=CREATE&stateFormat=2&name=delLiveColl&replicationFactor=2&numShards=2&wt=javabin&maxShardsPerNode=2 [junit4] 2> 504634 T1874 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged [junit4] 2> 504635 T1867 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{ [junit4] 2> "operation":"createcollection", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"delLiveColl", [junit4] 2> "replicationFactor":"2", [junit4] 2> "numShards":"2", [junit4] 2> "maxShardsPerNode":"2", [junit4] 2> "stateFormat":"2"} [junit4] 2> 504636 T1941 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , { [junit4] 2> "operation":"createcollection", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"delLiveColl", [junit4] 2> "replicationFactor":"2", [junit4] 2> "numShards":"2", [junit4] 2> "maxShardsPerNode":"2", [junit4] 2> "stateFormat":"2"} [junit4] 2> 504637 T1941 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1 [junit4] 2> 504637 T1941 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delLiveColl [junit4] 2> 504638 T1941 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl [junit4] 2> 504640 T1874 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 504642 T1866 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: delLiveColl [junit4] 2> 504642 T1866 oasc.Overseer$ClusterStateUpdater.createCollection Create collection delLiveColl with shards [shard1, shard2] [junit4] 2> 504642 T1866 oasc.Overseer$ClusterStateUpdater.createCollection state version delLiveColl 2 [junit4] 2> 504643 T1866 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/delLiveColl/state.json [junit4] 2> 504644 T1874 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504644 T1889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504644 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504644 T1934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504644 T1904 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504644 T1919 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504644 T1875 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504645 T1943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 504740 T1941 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delLiveColl, shardNames [shard1, shard2] , replicationFactor : 2 [junit4] 2> 504741 T1941 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:31541_ln%2Fky [junit4] 2> 504741 T1941 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:31519_ln%2Fky [junit4] 2> 504743 T1941 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:31511_ln%2Fky [junit4] 2> 504744 T1941 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:31538_ln%2Fky [junit4] 2> 504865 T1923 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&version=2&collection=delLiveColl&collection.configName=conf1&action=CREATE&shard=shard1&name=delLiveColl_shard1_replica1&numShards=2&wt=javabin [junit4] 2> 504867 T1909 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&version=2&collection=delLiveColl&collection.configName=conf1&action=CREATE&shard=shard2&name=delLiveColl_shard2_replica2&numShards=2&wt=javabin [junit4] 2> 504868 T1923 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down collection=delLiveColl [junit4] 2> 504869 T1874 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 504869 T1923 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 504870 T1923 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 504870 T1923 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 504870 T1909 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down collection=delLiveColl [junit4] 2> 504871 T1923 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 [junit4] 2> 504871 T1866 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "base_url":"https://127.0.0.1:31541/ln/ky";, [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "collection":"delLiveColl", [junit4] 2> "shard":"shard1", [junit4] 2> "numShards":"2", [junit4] 2> "node_name":"127.0.0.1:31541_ln%2Fky", [junit4] 2> "core":"delLiveColl_shard1_replica1", [junit4] 2> "operation":"state"} [junit4] 2> 504871 T1923 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 504873 T1909 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 504873 T1909 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 504874 T1879 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&version=2&collection=delLiveColl&collection.configName=conf1&action=CREATE&shard=shard1&name=delLiveColl_shard1_replica2&numShards=2&wt=javabin [junit4] 2> 504874 T1909 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 504875 T1909 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 [junit4] 2> 504875 T1909 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 504876 T1879 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down collection=delLiveColl [junit4] 2> 504877 T1866 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 504877 T1860 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&version=2&collection=delLiveColl&collection.configName=conf1&action=CREATE&shard=shard2&name=delLiveColl_shard2_replica1&numShards=2&wt=javabin [junit4] 2> 504877 T1934 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504877 T1919 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504878 T1879 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 504878 T1934 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 504878 T1879 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 504878 T1919 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 504878 T1879 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 504879 T1879 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 504880 T1879 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 504880 T1860 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down collection=delLiveColl [junit4] 2> 504880 T1866 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "base_url":"https://127.0.0.1:31538/ln/ky";, [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "collection":"delLiveColl", [junit4] 2> "shard":"shard2", [junit4] 2> "numShards":"2", [junit4] 2> "node_name":"127.0.0.1:31538_ln%2Fky", [junit4] 2> "core":"delLiveColl_shard2_replica2", [junit4] 2> "operation":"state"} [junit4] 2> 504881 T1874 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 504882 T1860 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 504884 T1866 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 504885 T1919 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504885 T1889 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504885 T1934 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504885 T1919 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 504885 T1889 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 504886 T1934 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 504886 T1860 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 504886 T1860 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 504887 T1860 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 504887 T1860 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 504889 T1866 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 504889 T1866 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "base_url":"https://127.0.0.1:31519/ln/ky";, [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "collection":"delLiveColl", [junit4] 2> "shard":"shard1", [junit4] 2> "numShards":"2", [junit4] 2> "node_name":"127.0.0.1:31519_ln%2Fky", [junit4] 2> "core":"delLiveColl_shard1_replica2", [junit4] 2> "operation":"state"} [junit4] 2> 504890 T1874 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 504890 T1866 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 504891 T1874 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504891 T1919 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504891 T1934 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504891 T1889 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504891 T1919 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 504892 T1934 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 504892 T1889 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 504892 T1874 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 504894 T1866 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 504894 T1866 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "base_url":"https://127.0.0.1:31511/ln/ky";, [junit4] 2> "state":"down", [junit4] 2> "roles":null, [junit4] 2> "collection":"delLiveColl", [junit4] 2> "shard":"shard2", [junit4] 2> "numShards":"2", [junit4] 2> "node_name":"127.0.0.1:31511_ln%2Fky", [junit4] 2> "core":"delLiveColl_shard2_replica1", [junit4] 2> "operation":"state"} [junit4] 2> 504895 T1874 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 504896 T1866 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 504896 T1874 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504896 T1889 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504896 T1934 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504896 T1919 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 504897 T1889 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 504897 T1934 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 504897 T1919 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 504897 T1874 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 505874 T1923 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1 [junit4] 2> 505875 T1923 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 505875 T1923 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 505875 T1909 oasc.ZkController.waitFor [...truncated too long message...] e.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46) [junit4] > at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357) [junit4] > at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3393, name=Thread-1489, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest] [junit4] > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path3" "write") [junit4] > at __randomizedtesting.SeedInfo.seed([293DA6A76D018679]:0) [junit4] > at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372) [junit4] > at java.security.AccessController.checkPermission(AccessController.java:559) [junit4] > at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) [junit4] > at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) [junit4] > at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801) [junit4] > at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376) [junit4] > at java.nio.file.Files.createDirectory(Files.java:630) [junit4] > at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734) [junit4] > at java.nio.file.Files.createDirectories(Files.java:683) [junit4] > at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46) [junit4] > at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357) [junit4] > at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #10: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3395, name=Thread-1491, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest] [junit4] > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path3" "write") [junit4] > at __randomizedtesting.SeedInfo.seed([293DA6A76D018679]:0) [junit4] > at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372) [junit4] > at java.security.AccessController.checkPermission(AccessController.java:559) [junit4] > at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) [junit4] > at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) [junit4] > at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801) [junit4] > at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376) [junit4] > at java.nio.file.Files.createDirectory(Files.java:630) [junit4] > at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734) [junit4] > at java.nio.file.Files.createDirectories(Files.java:683) [junit4] > at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46) [junit4] > at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357) [junit4] > at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #11: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3394, name=Thread-1490, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest] [junit4] > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path3" "write") [junit4] > at __randomizedtesting.SeedInfo.seed([293DA6A76D018679]:0) [junit4] > at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372) [junit4] > at java.security.AccessController.checkPermission(AccessController.java:559) [junit4] > at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) [junit4] > at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) [junit4] > at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801) [junit4] > at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376) [junit4] > at java.nio.file.Files.createDirectory(Files.java:630) [junit4] > at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734) [junit4] > at java.nio.file.Files.createDirectories(Files.java:683) [junit4] > at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46) [junit4] > at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500) [junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357) [junit4] > at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190) [junit4] 2> 1612869 T3384 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: test params are: codec=Asserting(Lucene410), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=sr__#Latn, timezone=Asia/Gaza [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=94692312,total=301465600 [junit4] 2> NOTE: All tests run in this JVM: [DocValuesTest, RegexBoostProcessorTest, DefaultValueUpdateProcessorTest, XmlUpdateRequestHandlerTest, TestStressLucene, AnalysisErrorHandlingTest, TestReplicationHandler, MultiTermTest, InfoHandlerTest, HdfsCollectionsAPIDistributedZkTest, HighlighterConfigTest, TestRandomDVFaceting, HdfsRecoveryZkTest, TestManagedSchema, TestCloudManagedSchema, AnalysisAfterCoreReloadTest, CursorMarkTest, TestSolr4Spatial, SSLMigrationTest, TestRealTimeGet, DistributedIntervalFacetingTest, TestSolrQueryParserDefaultOperatorResource, TestMissingGroups, OutOfBoxZkACLAndCredentialsProvidersTest, StatsComponentTest, TestManagedStopFilterFactory, LegacyHTMLStripCharFilterTest, FieldFacetTest, AbstractAnalyticsStatsTest, ExternalCollectionsTest, TestPseudoReturnFields, DistributedFacetPivotLongTailTest, AddSchemaFieldsUpdateProcessorFactoryTest, RemoteQueryErrorTest, TestQuerySenderListener, TestReversedWildcardFilterFactory, OpenExchangeRatesOrgProviderTest, TestFieldResource, TestDistributedSearch, DistributedDebugComponentTest, DistribDocExpirationUpdateProcessorTest, TestSchemaVersionResource, SolrXmlInZkTest, BinaryUpdateRequestHandlerTest, TestLeaderElectionZkExpiry, TestDFRSimilarityFactory, PingRequestHandlerTest, NoFacetTest, CopyFieldTest, HighlighterTest, UpdateRequestProcessorFactoryTest, IndexSchemaTest, TestSolrQueryParser, TestRawResponseWriter, TestStressReorder, TestMultiCoreConfBootstrap, TestReload, HardAutoCommitTest, DistributedTermsComponentTest, TestBadConfig, DistributedQueryElevationComponentTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocumentBuilderTest, TestIndexingPerformance, JsonLoaderTest, TestQuerySenderNoQuery, TestSolrDeletionPolicy2, TestElisionMultitermQuery, DirectSolrConnectionTest, TestLuceneMatchVersion, TestCodecSupport, TestSweetSpotSimilarityFactory, TestBM25SimilarityFactory, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, OverseerStatusTest, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsBasicDistributedZkTest, HdfsChaosMonkeySafeLeaderTest, HdfsSyncSliceTest, HdfsUnloadDistributedZkTest, StressHdfsTest, CachingDirectoryFactoryTest] [junit4] Completed on J2 in 8.11s, 1 test, 1 error <<< FAILURES! [...truncated 221 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:491: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:471: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:60: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:39: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1358: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:961: There were test failures: 438 suites, 1830 tests, 3 suite-level errors, 4 errors, 59 ignored (24 assumptions) Total time: 56 minutes 30 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4846 Archived 5 artifacts Archive block size is 32768 Received 0 blocks and 166439907 bytes Compression is 0.0% Took 30 sec Recording test results Email was triggered for: Failure Sending email for trigger: Failure
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
