Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/917/
3 tests failed. FAILED: org.apache.solr.cloud.BasicDistributedZkTest.test Error Message: Error from server at http://127.0.0.1:41610/_hr/collection1: Bad Request request: http://127.0.0.1:33848/_hr/collection1/update?update.chain=distrib-dup-test-chain-explicit&update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A41610%2F_hr%2Fcollection1%2F&wt=javabin&version=2 Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41610/_hr/collection1: Bad Request request: http://127.0.0.1:33848/_hr/collection1/update?update.chain=distrib-dup-test-chain-explicit&update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A41610%2F_hr%2Fcollection1%2F&wt=javabin&version=2 at __randomizedtesting.SeedInfo.seed([DF7B7EA88C3E888F:572F417222C2E577]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:229) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167) at org.apache.solr.BaseDistributedSearchTestCase.add(BaseDistributedSearchTestCase.java:512) at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:623) at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:364) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:965) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:940) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(Thread.java:745) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest Error Message: ObjectTracker found 1 object(s) that were not released!!! [TransactionLog] Stack Trace: java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [TransactionLog] at __randomizedtesting.SeedInfo.seed([DF7B7EA88C3E888F]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertNull(Assert.java:551) at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:229) at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(Thread.java:745) FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload Error Message: expected:<[{indexVersion=1453623651052,generation=2,filelist=[_18nw.cfe, _18nw.cfs, _18nw.si, _1jt0.cfe, _1jt0.cfs, _1jt0.si, _1uym.cfe, _1uym.cfs, _1uym.si, _263z.cfe, _263z.cfs, _263z.si, _2g4x.cfe, _2g4x.cfs, _2g4x.si, _2g5s.cfe, _2g5s.cfs, _2g5s.si, _2g5t.cfe, _2g5t.cfs, _2g5t.si, _2g5u.cfe, _2g5u.cfs, _2g5u.si, _2g5v.cfe, _2g5v.cfs, _2g5v.si, _2g5w.cfe, _2g5w.cfs, _2g5w.si, _2g5x.cfe, _2g5x.cfs, _2g5x.si, _2g5y.cfe, _2g5y.cfs, _2g5y.si, _2g6g.cfe, _2g6g.cfs, _2g6g.si, _2g6h.cfe, _2g6h.cfs, _2g6h.si, _2g6i.cfe, _2g6i.cfs, _2g6i.si, _2g6j.cfe, _2g6j.cfs, _2g6j.si, _b8h.fdt, _b8h.fdx, _b8h.fnm, _b8h.nvd, _b8h.nvm, _b8h.si, _b8h_LuceneVarGapFixedInterval_0.doc, _b8h_LuceneVarGapFixedInterval_0.tib, _b8h_LuceneVarGapFixedInterval_0.tiv, _mde.fdt, _mde.fdx, _mde.fnm, _mde.nvd, _mde.nvm, _mde.si, _mde_LuceneVarGapFixedInterval_0.doc, _mde_LuceneVarGapFixedInterval_0.tib, _mde_LuceneVarGapFixedInterval_0.tiv, _xir.fdt, _xir.fdx, _xir.fnm, _xir.nvd, _xir.nvm, _xir.si, _xir_LuceneVarGapFixedInterval_0.doc, _xir_LuceneVarGapFixedInterval_0.tib, _xir_LuceneVarGapFixedInterval_0.tiv, segments_2]}]> but was:<[{indexVersion=1453623651052,generation=2,filelist=[_18nw.cfe, _18nw.cfs, _18nw.si, _1jt0.cfe, _1jt0.cfs, _1jt0.si, _1uym.cfe, _1uym.cfs, _1uym.si, _263z.cfe, _263z.cfs, _263z.si, _2g4x.cfe, _2g4x.cfs, _2g4x.si, _2g5s.cfe, _2g5s.cfs, _2g5s.si, _2g5t.cfe, _2g5t.cfs, _2g5t.si, _2g5u.cfe, _2g5u.cfs, _2g5u.si, _2g5v.cfe, _2g5v.cfs, _2g5v.si, _2g5w.cfe, _2g5w.cfs, _2g5w.si, _2g5x.cfe, _2g5x.cfs, _2g5x.si, _2g5y.cfe, _2g5y.cfs, _2g5y.si, _2g6g.cfe, _2g6g.cfs, _2g6g.si, _2g6h.cfe, _2g6h.cfs, _2g6h.si, _2g6i.cfe, _2g6i.cfs, _2g6i.si, _2g6j.cfe, _2g6j.cfs, _2g6j.si, _b8h.fdt, _b8h.fdx, _b8h.fnm, _b8h.nvd, _b8h.nvm, _b8h.si, _b8h_LuceneVarGapFixedInterval_0.doc, _b8h_LuceneVarGapFixedInterval_0.tib, _b8h_LuceneVarGapFixedInterval_0.tiv, _mde.fdt, _mde.fdx, _mde.fnm, _mde.nvd, _mde.nvm, _mde.si, _mde_LuceneVarGapFixedInterval_0.doc, _mde_LuceneVarGapFixedInterval_0.tib, _mde_LuceneVarGapFixedInterval_0.tiv, _xir.fdt, _xir.fdx, _xir.fnm, _xir.nvd, _xir.nvm, _xir.si, _xir_LuceneVarGapFixedInterval_0.doc, _xir_LuceneVarGapFixedInterval_0.tib, _xir_LuceneVarGapFixedInterval_0.tiv, segments_2]}, {indexVersion=1453623651052,generation=3,filelist=[_18nw.cfe, _18nw.cfs, _18nw.si, _1jt0.cfe, _1jt0.cfs, _1jt0.si, _1uym.cfe, _1uym.cfs, _1uym.si, _263z.cfe, _263z.cfs, _263z.si, _2g5z.cfe, _2g5z.cfs, _2g5z.si, _2g6g.cfe, _2g6g.cfs, _2g6g.si, _2g6h.cfe, _2g6h.cfs, _2g6h.si, _2g6i.cfe, _2g6i.cfs, _2g6i.si, _2g6j.cfe, _2g6j.cfs, _2g6j.si, _b8h.fdt, _b8h.fdx, _b8h.fnm, _b8h.nvd, _b8h.nvm, _b8h.si, _b8h_LuceneVarGapFixedInterval_0.doc, _b8h_LuceneVarGapFixedInterval_0.tib, _b8h_LuceneVarGapFixedInterval_0.tiv, _mde.fdt, _mde.fdx, _mde.fnm, _mde.nvd, _mde.nvm, _mde.si, _mde_LuceneVarGapFixedInterval_0.doc, _mde_LuceneVarGapFixedInterval_0.tib, _mde_LuceneVarGapFixedInterval_0.tiv, _xir.fdt, _xir.fdx, _xir.fnm, _xir.nvd, _xir.nvm, _xir.si, _xir_LuceneVarGapFixedInterval_0.doc, _xir_LuceneVarGapFixedInterval_0.tib, _xir_LuceneVarGapFixedInterval_0.tiv, segments_3]}]> Stack Trace: java.lang.AssertionError: expected:<[{indexVersion=1453623651052,generation=2,filelist=[_18nw.cfe, _18nw.cfs, _18nw.si, _1jt0.cfe, _1jt0.cfs, _1jt0.si, _1uym.cfe, _1uym.cfs, _1uym.si, _263z.cfe, _263z.cfs, _263z.si, _2g4x.cfe, _2g4x.cfs, _2g4x.si, _2g5s.cfe, _2g5s.cfs, _2g5s.si, _2g5t.cfe, _2g5t.cfs, _2g5t.si, _2g5u.cfe, _2g5u.cfs, _2g5u.si, _2g5v.cfe, _2g5v.cfs, _2g5v.si, _2g5w.cfe, _2g5w.cfs, _2g5w.si, _2g5x.cfe, _2g5x.cfs, _2g5x.si, _2g5y.cfe, _2g5y.cfs, _2g5y.si, _2g6g.cfe, _2g6g.cfs, _2g6g.si, _2g6h.cfe, _2g6h.cfs, _2g6h.si, _2g6i.cfe, _2g6i.cfs, _2g6i.si, _2g6j.cfe, _2g6j.cfs, _2g6j.si, _b8h.fdt, _b8h.fdx, _b8h.fnm, _b8h.nvd, _b8h.nvm, _b8h.si, _b8h_LuceneVarGapFixedInterval_0.doc, _b8h_LuceneVarGapFixedInterval_0.tib, _b8h_LuceneVarGapFixedInterval_0.tiv, _mde.fdt, _mde.fdx, _mde.fnm, _mde.nvd, _mde.nvm, _mde.si, _mde_LuceneVarGapFixedInterval_0.doc, _mde_LuceneVarGapFixedInterval_0.tib, _mde_LuceneVarGapFixedInterval_0.tiv, _xir.fdt, _xir.fdx, _xir.fnm, _xir.nvd, _xir.nvm, _xir.si, _xir_LuceneVarGapFixedInterval_0.doc, _xir_LuceneVarGapFixedInterval_0.tib, _xir_LuceneVarGapFixedInterval_0.tiv, segments_2]}]> but was:<[{indexVersion=1453623651052,generation=2,filelist=[_18nw.cfe, _18nw.cfs, _18nw.si, _1jt0.cfe, _1jt0.cfs, _1jt0.si, _1uym.cfe, _1uym.cfs, _1uym.si, _263z.cfe, _263z.cfs, _263z.si, _2g4x.cfe, _2g4x.cfs, _2g4x.si, _2g5s.cfe, _2g5s.cfs, _2g5s.si, _2g5t.cfe, _2g5t.cfs, _2g5t.si, _2g5u.cfe, _2g5u.cfs, _2g5u.si, _2g5v.cfe, _2g5v.cfs, _2g5v.si, _2g5w.cfe, _2g5w.cfs, _2g5w.si, _2g5x.cfe, _2g5x.cfs, _2g5x.si, _2g5y.cfe, _2g5y.cfs, _2g5y.si, _2g6g.cfe, _2g6g.cfs, _2g6g.si, _2g6h.cfe, _2g6h.cfs, _2g6h.si, _2g6i.cfe, _2g6i.cfs, _2g6i.si, _2g6j.cfe, _2g6j.cfs, _2g6j.si, _b8h.fdt, _b8h.fdx, _b8h.fnm, _b8h.nvd, _b8h.nvm, _b8h.si, _b8h_LuceneVarGapFixedInterval_0.doc, _b8h_LuceneVarGapFixedInterval_0.tib, _b8h_LuceneVarGapFixedInterval_0.tiv, _mde.fdt, _mde.fdx, _mde.fnm, _mde.nvd, _mde.nvm, _mde.si, _mde_LuceneVarGapFixedInterval_0.doc, _mde_LuceneVarGapFixedInterval_0.tib, _mde_LuceneVarGapFixedInterval_0.tiv, _xir.fdt, _xir.fdx, _xir.fnm, _xir.nvd, _xir.nvm, _xir.si, _xir_LuceneVarGapFixedInterval_0.doc, _xir_LuceneVarGapFixedInterval_0.tib, _xir_LuceneVarGapFixedInterval_0.tiv, segments_2]}, {indexVersion=1453623651052,generation=3,filelist=[_18nw.cfe, _18nw.cfs, _18nw.si, _1jt0.cfe, _1jt0.cfs, _1jt0.si, _1uym.cfe, _1uym.cfs, _1uym.si, _263z.cfe, _263z.cfs, _263z.si, _2g5z.cfe, _2g5z.cfs, _2g5z.si, _2g6g.cfe, _2g6g.cfs, _2g6g.si, _2g6h.cfe, _2g6h.cfs, _2g6h.si, _2g6i.cfe, _2g6i.cfs, _2g6i.si, _2g6j.cfe, _2g6j.cfs, _2g6j.si, _b8h.fdt, _b8h.fdx, _b8h.fnm, _b8h.nvd, _b8h.nvm, _b8h.si, _b8h_LuceneVarGapFixedInterval_0.doc, _b8h_LuceneVarGapFixedInterval_0.tib, _b8h_LuceneVarGapFixedInterval_0.tiv, _mde.fdt, _mde.fdx, _mde.fnm, _mde.nvd, _mde.nvm, _mde.si, _mde_LuceneVarGapFixedInterval_0.doc, _mde_LuceneVarGapFixedInterval_0.tib, _mde_LuceneVarGapFixedInterval_0.tiv, _xir.fdt, _xir.fdx, _xir.fnm, _xir.nvd, _xir.nvm, _xir.si, _xir_LuceneVarGapFixedInterval_0.doc, _xir_LuceneVarGapFixedInterval_0.tib, _xir_LuceneVarGapFixedInterval_0.tiv, segments_3]}]> at __randomizedtesting.SeedInfo.seed([DF7B7EA88C3E888F:FAAC6598FC76868C]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1140) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 10068 lines...] [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/init-core-data-001 [junit4] 2> 307691 INFO (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[DF7B7EA88C3E888F]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: / [junit4] 2> 307697 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 307702 INFO (Thread-1364) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 307702 INFO (Thread-1364) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 307802 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.ZkTestServer start zk server on port:38928 [junit4] 2> 307803 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 307803 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 307813 INFO (zkCallback-317-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@796e8f73 name:ZooKeeperConnection Watcher:127.0.0.1:38928 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 307813 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 307813 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 307813 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr [junit4] 2> 307816 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 307823 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 307825 INFO (zkCallback-318-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b7c1972 name:ZooKeeperConnection Watcher:127.0.0.1:38928/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 307825 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 307826 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 307826 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1 [junit4] 2> 307829 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards [junit4] 2> 307831 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection [junit4] 2> 307832 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards [junit4] 2> 307834 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 307834 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml [junit4] 2> 307836 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml [junit4] 2> 307836 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml [junit4] 2> 307838 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 307838 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 307839 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 307839 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt [junit4] 2> 307840 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 307841 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt [junit4] 2> 307842 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 307842 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml [junit4] 2> 307843 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 307843 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 307844 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 307844 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 307845 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 307846 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 307847 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 307847 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 307848 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 307848 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt [junit4] 2> 307950 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1 [junit4] 2> 307952 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.Server jetty-9.2.13.v20150730 [junit4] 2> 307963 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34990d62{/,null,AVAILABLE} [junit4] 2> 307969 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.ServerConnector Started ServerConnector@4bfb57cb{HTTP/1.1}{127.0.0.1:41367} [junit4] 2> 307969 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.Server Started @311334ms [junit4] 2> 307969 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/tempDir-001/control/data, hostContext=/, hostPort=41367, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores} [junit4] 2> 307969 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a [junit4] 2> 307969 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001' [junit4] 2> 307970 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 307970 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 307973 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 307974 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 307977 INFO (zkCallback-319-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73065c9e name:ZooKeeperConnection Watcher:127.0.0.1:38928/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 307977 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 307978 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 307979 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper) [junit4] 2> 307980 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/solr.xml [junit4] 2> 307985 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores [junit4] 2> 307985 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CoreContainer New CoreContainer 706770019 [junit4] 2> 307985 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001] [junit4] 2> 307985 WARN (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/lib [junit4] 2> 307986 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 307987 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 307988 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 307988 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 307988 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38928/solr [junit4] 2> 307988 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 307988 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 307998 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 308006 INFO (zkCallback-322-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@316ec176 name:ZooKeeperConnection Watcher:127.0.0.1:38928 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 308006 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 308007 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 308009 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 308023 INFO (zkCallback-323-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@454c7fa9 name:ZooKeeperConnection Watcher:127.0.0.1:38928/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 308023 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 308025 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue [junit4] 2> 308026 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work [junit4] 2> 308029 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running [junit4] 2> 308030 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed [junit4] 2> 308031 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure [junit4] 2> 308033 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes [junit4] 2> 308034 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json [junit4] 2> 308035 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json [junit4] 2> 308035 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /security.json [junit4] 2> 308036 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41367_ [junit4] 2> 308036 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41367_ [junit4] 2> 308037 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 308038 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 308039 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 308040 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95264457118842884-127.0.0.1:41367_-n_0000000000 [junit4] 2> 308040 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41367_ [junit4] 2> 308040 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader [junit4] 2> 308041 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer Overseer (id=95264457118842884-127.0.0.1:41367_-n_0000000000) starting [junit4] 2> 308042 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work [junit4] 2> 308046 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 308050 INFO (OverseerCollectionConfigSetProcessor-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations [junit4] 2> 308051 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 308051 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer Starting to work on the main queue [junit4] 2> 308076 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 308076 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 308077 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores [junit4] 2> 308078 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 308078 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1 [junit4] 2> 308078 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions [junit4] 2> 308088 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.ZkController publishing state=down [junit4] 2> 308088 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property [junit4] 2> 308088 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1 [junit4] 2> 308088 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:41367_ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 308090 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41367";, [junit4] 2> "node_name":"127.0.0.1:41367_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 0 [junit4] 2> 308090 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.ZkController look for our core node name [junit4] 2> 308090 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41367";, [junit4] 2> "node_name":"127.0.0.1:41367_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 308091 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection [junit4] 2> 308091 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1 [junit4] 2> 308092 INFO (zkCallback-323-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1]) [junit4] 2> 309091 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1 [junit4] 2> 309091 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection [junit4] 2> 309091 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.ZkController Collection zkNode exists [junit4] 2> 309091 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection] [junit4] 2> 309092 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper [junit4] 2> 309092 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1' [junit4] 2> 309092 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 309092 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 309093 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 [junit4] 2> 309098 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1 [junit4] 2> 309129 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0 [junit4] 2> 309139 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml [junit4] 2> 309144 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test [junit4] 2> 309226 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.s.IndexSchema default search field in schema is text [junit4] 2> 309228 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.s.IndexSchema unique key field: id [junit4] 2> 309229 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml [junit4] 2> 309232 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml [junit4] 2> 309246 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1 [junit4] 2> 309247 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 309247 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore solr.StandardDirectoryFactory [junit4] 2> 309255 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data/] [junit4] 2> 309255 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6cf106c8 [junit4] 2> 309256 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data [junit4] 2> 309256 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data/index/ [junit4] 2> 309256 WARN (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 309257 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data/index [junit4] 2> 309257 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3c1adec2 [junit4] 2> 309266 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1} [junit4] 2> 309266 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1 [junit4] 2> 309273 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 309274 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 309274 WARN (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}} [junit4] 2> 309277 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 309277 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 309278 INFO (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data/ [junit4] 2> 309279 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 309279 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 309295 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/dump,/update/csv,/sql,/update/json/docs,/admin/luke,/export,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/stream,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file [junit4] 2> 309295 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 309307 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog [junit4] 2> 309307 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 309307 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; [junit4] 2> 309307 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 309308 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=2.0537109375, floorSegmentMB=1.2041015625, forceMergeDeletesPctAllowed=29.77406678033875, segmentsPerTier=38.0, maxCFSSegmentSizeMB=0.5595703125, noCFSRatio=1.0 [junit4] 2> 309309 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1} [junit4] 2> 309309 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1 [junit4] 2> 309309 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1de90a4e[collection1] main] [junit4] 2> 309309 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection] [junit4] 2> 309311 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper [junit4] 2> 309311 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 309312 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 309312 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {} [junit4] 2> 309312 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 309312 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 309312 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 309312 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources [junit4] 2> 309312 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 [junit4] 2> 309313 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets [junit4] 2> 309313 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index [junit4] 2> 309313 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index [junit4] 2> 309313 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1524231315577110528 [junit4] 2> 309314 INFO (searcherExecutor-1262-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1de90a4e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 309315 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1524231315577110528 [junit4] 2> 309315 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1 [junit4] 2> 309316 INFO (coreLoadExecutor-1261-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1 [junit4] 2> 309329 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:41367 collection:control_collection shard:shard1 [junit4] 2> 309329 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election [junit4] 2> 309332 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/95264457118842884-core_node1-n_0000000000 [junit4] 2> 309332 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000 [junit4] 2> 309334 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection"} current state version: 1 [junit4] 2> 309335 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. [junit4] 2> 309335 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync [junit4] 2> 309335 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41367/collection1/ [junit4] 2> 309335 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me [junit4] 2> 309335 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:41367/collection1/ has no replicas [junit4] 2> 309337 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1 [junit4] 2> 309339 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node [junit4] 2> 309340 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41367/collection1/ shard1 [junit4] 2> 309341 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "base_url":"http://127.0.0.1:41367";, [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 1 [junit4] 2> 309443 INFO (zkCallback-323-thread-1-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1]) [junit4] 2> 309493 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:41367/collection1/ and leader is http://127.0.0.1:41367/collection1/ [junit4] 2> 309493 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41367 [junit4] 2> 309493 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary [junit4] 2> 309493 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active [junit4] 2> 309493 INFO (coreZkRegister-1255-thread-1-processing-n:127.0.0.1:41367_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:41367_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property [junit4] 2> 309497 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41367";, [junit4] 2> "node_name":"127.0.0.1:41367_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 309497 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:41367";, [junit4] 2> "node_name":"127.0.0.1:41367_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 309623 INFO (zkCallback-323-thread-2-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1]) [junit4] 2> 309624 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 309627 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 309632 INFO (zkCallback-325-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@150d401f name:ZooKeeperConnection Watcher:127.0.0.1:38928/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 309632 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 309633 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 309633 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 309637 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 309729 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/cores/collection1 [junit4] 2> 309730 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001 [junit4] 2> 309738 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.Server jetty-9.2.13.v20150730 [junit4] 2> 309739 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b21d64c{/,null,AVAILABLE} [junit4] 2> 309740 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.ServerConnector Started ServerConnector@3a45107c{HTTP/1.1}{127.0.0.1:52194} [junit4] 2> 309740 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.Server Started @313105ms [junit4] 2> 309740 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=52194, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/cores} [junit4] 2> 309740 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a [junit4] 2> 309740 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001' [junit4] 2> 309741 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 309741 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 309741 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 309742 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 309746 INFO (zkCallback-326-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5de935c8 name:ZooKeeperConnection Watcher:127.0.0.1:38928/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 309746 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 309747 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 309749 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper) [junit4] 2> 309749 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/solr.xml [junit4] 2> 309754 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/cores [junit4] 2> 309754 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CoreContainer New CoreContainer 1954637927 [junit4] 2> 309754 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001] [junit4] 2> 309755 WARN (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/lib [junit4] 2> 309756 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 309758 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 309758 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 309758 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 309758 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38928/solr [junit4] 2> 309758 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 309758 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 309759 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 309761 INFO (zkCallback-329-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54e1518b name:ZooKeeperConnection Watcher:127.0.0.1:38928 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 309761 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 309762 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 309770 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 309779 INFO (zkCallback-330-thread-1-processing-n:127.0.0.1:52194_) [n:127.0.0.1:52194_ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53616e90 name:ZooKeeperConnection Watcher:127.0.0.1:38928/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 309780 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 309784 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 310788 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52194_ [junit4] 2> 310789 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52194_ [junit4] 2> 310790 INFO (zkCallback-330-thread-1-processing-n:127.0.0.1:52194_) [n:127.0.0.1:52194_ ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1]) [junit4] 2> 310790 INFO (zkCallback-325-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1]) [junit4] 2> 310790 INFO (zkCallback-323-thread-2-processing-n:127.0.0.1:41367_) [n:127.0.0.1:41367_ ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1]) [junit4] 2> 310792 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 310792 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/95264457118842888-127.0.0.1:52194_-n_0000000001 [junit4] 2> 310793 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/95264457118842884-127.0.0.1:41367_-n_0000000000 to know if I could be the leader [junit4] 2> 310806 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 310806 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 310807 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/cores [junit4] 2> 310808 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 310808 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_DF7B7EA88C3E888F-001/shard-1-001/cores/collection1 [junit4] 2> 310808 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions [junit4] 2> 310809 INFO (coreLoadExecutor-1272-thread-1-processing-n:127.0.0.1:52194_) [n:127.0.0.1:52194_ c:collection1 x:collection1] o.a.s.c.ZkController publishing state=down [junit4] 2> 310809 INFO (coreLoadExecutor-1272-thread-1-processing-n:127.0.0.1:52194_) [n:127.0.0.1:52194_ c:collection1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property [junit4] 2> 310810 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1 [junit4] 2> 310810 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[DF7B7EA88C3E888F]) [n:127.0.0.1:52194_ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 310811 INFO (OverseerStateUpdate-95264457118842884-127.0.0.1:41367_-n_0000000000) [n:127.0.0.1:41367_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:52194";, [junit4] 2> "node_name":"127.0.0.1:52194_", [junit4] 2> "numShards":"3", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 3 [junit4] 2> 310811 INFO (coreLoadExecutor-1272-thread-1-processing-n:127.0.0.1:52194_) [n:127.0.0.1:52194_ c:collection1 x:collection1] o.a.s.c.ZkController look for our core node name [junit4] 2> 310811 I [...truncated too long message...] nk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-027/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-027/./collection1/data;done=false>>] [junit4] 2> 3707648 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-027/./collection1/data [junit4] 2> 3707649 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@53146bd2{/solr,null,UNAVAILABLE} [junit4] 2> 3707652 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@35a3f9b3{HTTP/1.1}{127.0.0.1:0} [junit4] 2> 3707653 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1097079382 [junit4] 2> 3707653 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@ade597e [junit4] 2> 3707653 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0} [junit4] 2> 3707653 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState [junit4] 2> 3707653 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter [junit4] 2> 3707653 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser [junit4] 2> 3707660 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request. [junit4] 2> 3707660 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked [junit4] 2> 3707660 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-028/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-028/./collection1/data;done=false>>] [junit4] 2> 3707660 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-028/./collection1/data [junit4] 2> 3707660 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-028/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-028/./collection1/data/index;done=false>>] [junit4] 2> 3707661 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001/solr-instance-028/./collection1/data/index [junit4] 2> 3707661 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[DF7B7EA88C3E888F]) [ x:collection1] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@725cca78{/solr,null,UNAVAILABLE} [junit4] 2> 3707663 INFO (SUITE-TestReplicationHandler-seed#[DF7B7EA88C3E888F]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_DF7B7EA88C3E888F-001 [junit4] 2> NOTE: test params are: codec=Asserting(Lucene60): {name=PostingsFormat(name=LuceneVarGapFixedInterval), id=PostingsFormat(name=LuceneVarGapFixedInterval), newname=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{}, sim=ClassicSimilarity, locale=ar-SA, timezone=America/Argentina/Catamarca [junit4] 2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_66 (64-bit)/cpus=4,threads=1,free=155886888,total=531103744 [junit4] 2> NOTE: All tests run in this JVM: [DistributedDebugComponentTest, ResponseLogComponentTest, DistribJoinFromCollectionTest, SynonymTokenizerTest, CollectionTooManyReplicasTest, OverseerTaskQueueTest, TestComponentsName, DistributedFacetPivotLongTailTest, TestFunctionQuery, SystemInfoHandlerTest, TestRandomCollapseQParserPlugin, TestRandomDVFaceting, DateRangeFieldTest, DateFieldTest, TestManagedSchemaFieldResource, SOLR749Test, URLClassifyProcessorTest, TestJsonRequest, TestLuceneMatchVersion, TestLMDirichletSimilarityFactory, ReplicationFactorTest, DistributedSuggestComponentTest, TestConfigSetProperties, TestCloudInspectUtil, TestImplicitCoreProperties, TestFieldTypeCollectionResource, XsltUpdateRequestHandlerTest, DocumentAnalysisRequestHandlerTest, PreAnalyzedFieldTest, IndexSchemaTest, TestJoin, TestRandomFaceting, AnalysisAfterCoreReloadTest, TestRangeQuery, TestSolrCLIRunExample, TestSimpleQParserPlugin, TestRestManager, PeerSyncTest, DistributedFacetPivotWhiteBoxTest, InfoHandlerTest, TestRemoteStreaming, PathHierarchyTokenizerFactoryTest, PrimUtilsTest, SortSpecParsingTest, TestNRTOpen, VMParamsZkACLAndCredentialsProvidersTest, TestOmitPositions, TestNoOpRegenerator, HdfsChaosMonkeyNothingIsSafeTest, TestDefaultStatsCache, TestAtomicUpdateErrorCases, TestPhraseSuggestions, HdfsDirectoryTest, UUIDFieldTest, TestManagedSynonymFilterFactory, BasicZkTest, SuggestComponentContextFilterQueryTest, DeleteInactiveReplicaTest, LeaderElectionIntegrationTest, TestSolr4Spatial, TestSchemaVersionResource, CacheHeaderTest, TestLRUCache, TestExactStatsCache, TestStressLucene, QueryResultKeyTest, TestFieldSortValues, TestRuleBasedAuthorizationPlugin, TestInitQParser, CursorPagingTest, TestOrdValues, GraphQueryTest, BlockJoinFacetDistribTest, DirectoryFactoryTest, SparseHLLTest, VersionInfoTest, TestRTGBase, AnalysisErrorHandlingTest, TestExpandComponent, RequestLoggingTest, HdfsUnloadDistributedZkTest, TestReplicationHandlerBackup, TestAnalyzedSuggestions, PingRequestHandlerTest, ParsingFieldUpdateProcessorsTest, SolrCoreTest, MultiThreadedOCPTest, CdcrVersionReplicationTest, TestSQLHandler, AsyncCallRequestStatusResponseTest, ConnectionReuseTest, FileBasedSpellCheckerTest, OverseerCollectionConfigSetProcessorTest, TestShortCircuitedRequests, ResourceLoaderTest, EchoParamsTest, ShardRoutingTest, RecoveryZkTest, TestReplicationHandler] [junit4] Completed [415/577 (3!)] on J0 in 1142.84s, 13 tests, 1 failure <<< FAILURES! [...truncated 644 lines...] BUILD FAILED /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:682: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:619: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:57: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:524: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1424: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:981: There were test failures: 577 suites (8 ignored), 2316 tests, 1 suite-level error, 1 error, 1 failure, 33 ignored (18 assumptions) [seed: DF7B7EA88C3E888F] Total time: 272 minutes 56 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts No prior successful build to compare, so performing full copy of artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
