I hate the black hole :)

I can repro this on my freebsd machine - it's due to the unlimited connection 
timeout and black hole.

I need to modify the test to explicitly wait to make sure it doesn't try to 
query when it will be directed to a downed node. I hate that because I'd like 
to test that sort of thing, but it's very not blackhole friendly. Really a pain.

On Jul 23, 2012, at 2:23 PM, Apache Jenkins Server wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x-java7/183/
> 
> 2 tests failed.
> FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.SyncSliceTest
> 
> Error Message:
> ERROR: SolrIndexSearcher opens=24 closes=23
> 
> Stack Trace:
> java.lang.AssertionError: ERROR: SolrIndexSearcher opens=24 closes=23
>       at __randomizedtesting.SeedInfo.seed([60281BBAEEA74A2D]:0)
>       at org.junit.Assert.fail(Assert.java:93)
>       at 
> org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:216)
>       at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:82)
>       at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:601)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:754)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>       at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>       at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>       at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>       at 
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>       at 
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
>       at 
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
>       at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
>       at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>       at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>       at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
> 
> 
> FAILED:  org.apache.solr.cloud.FullSolrCloudTest.testDistribSearch
> 
> Error Message:
> Timeout occured while waiting response from server at: 
> http://localhost:40496/solr/collection1
> 
> Stack Trace:
> org.apache.solr.client.solrj.SolrServerException: Timeout occured while 
> waiting response from server at: http://localhost:40496/solr/collection1
>       at 
> __randomizedtesting.SeedInfo.seed([60281BBAEEA74A2D:E1CE95A299F82A11]:0)
>       at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:410)
>       at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:182)
>       at 
> org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
>       at 
> org.apache.solr.cloud.FullSolrCloudTest.index_specific(FullSolrCloudTest.java:504)
>       at 
> org.apache.solr.cloud.FullSolrCloudTest.brindDownShardIndexSomeDocsAndRecover(FullSolrCloudTest.java:722)
>       at 
> org.apache.solr.cloud.FullSolrCloudTest.doTest(FullSolrCloudTest.java:559)
>       at 
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679)
>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>       at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:601)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>       at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>       at 
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>       at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>       at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>       at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>       at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>       at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>       at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>       at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>       at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>       at 
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
>       at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>       at 
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
>       at 
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
>       at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
>       at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>       at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>       at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
>       at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
> Caused by: java.net.SocketTimeoutException: Read timed out
>       at java.net.SocketInputStream.socketRead0(Native Method)
>       at java.net.SocketInputStream.read(SocketInputStream.java:150)
>       at java.net.SocketInputStream.read(SocketInputStream.java:121)
>       at 
> org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
>       at 
> org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
>       at 
> org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
>       at 
> org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
>       at 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
>       at 
> org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
>       at 
> org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
>       at 
> org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
>       at 
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
>       at 
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
>       at 
> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
>       at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
>       at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
>       at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
>       at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
>       at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
>       ... 45 more
> 
> 
> 
> 
> Build Log:
> [...truncated 12618 lines...]
> [junit4:junit4]   2> 131579 T1223 oaz.ZooKeeper.close Session: 
> 0x138b509b89c0005 closed
> [junit4:junit4]   2> 131581 T1223 oejsh.ContextHandler.doStop stopped 
> o.e.j.s.ServletContextHandler{/solr,null}
> [junit4:junit4]   2> 131633 T1223 oasc.ChaosMonkey.monkeyLog monkey: stop 
> shard! 40496
> [junit4:junit4]   2> 131633 T1223 oasc.CoreContainer.shutdown Shutting down 
> CoreContainer instance=1760137095
> [junit4:junit4]   2> 131634 T1223 oasc.RecoveryStrategy.close WARNING 
> Stopping recovery for core collection1 
> zkNodeName=127.0.0.1:40496_solr_collection1
> [junit4:junit4]   2> 131735 T1294 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 131735 T1223 oaz.ZooKeeper.close Session: 
> 0x138b509b89c0006 closed
> [junit4:junit4]   2> 131737 T1223 oejsh.ContextHandler.doStop stopped 
> o.e.j.s.ServletContextHandler{/solr,null}
> [junit4:junit4]   2> 131787 T1292 C110 P40496 REQ /update 
> {wt=javabin&version=2} status=503 QTime=32760 
> [junit4:junit4]   2> 131789 T1292 C110 P40496 oasc.SolrException.log SEVERE 
> null:org.eclipse.jetty.io.EofException
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:154)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:101)
> [junit4:junit4]   2>          at 
> org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:185)
> [junit4:junit4]   2>          at 
> org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:94)
> [junit4:junit4]   2>          at 
> org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:49)
> [junit4:junit4]   2>          at 
> org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:403)
> [junit4:junit4]   2>          at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:288)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:267)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1067)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.Server.handle(Server.java:351)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:952)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> [junit4:junit4]   2>          at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2>  
> [junit4:junit4]   2> 131791 T1292 C110 P40496 oasc.SolrException.log SEVERE 
> null:org.eclipse.jetty.io.EofException
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:154)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:101)
> [junit4:junit4]   2>          at 
> org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:185)
> [junit4:junit4]   2>          at 
> org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:94)
> [junit4:junit4]   2>          at 
> org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:49)
> [junit4:junit4]   2>          at 
> org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:403)
> [junit4:junit4]   2>          at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:288)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:267)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1067)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.Server.handle(Server.java:351)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:952)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> [junit4:junit4]   2>          at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> [junit4:junit4]   2>          at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2>  
> [junit4:junit4]   2> 131799 T1292 C110 P40496 oasc.SolrCore.close 
> [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6d09edc3
> [junit4:junit4]   2> 131807 T1292 C110 P40496 oasc.SolrCore.closeSearcher 
> [collection1] Closing main searcher on request.
> [junit4:junit4]   2> 131810 T1292 C110 P40496 oasu.DirectUpdateHandler2.close 
> closing DirectUpdateHandler2{commits=6,autocommits=0,soft 
> autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=36,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4]   2> 131811 T1292 C110 P40496 oasc.RecoveryStrategy.close 
> WARNING Stopping recovery for core collection1 
> zkNodeName=127.0.0.1:40496_solr_collection1
> [junit4:junit4]   2> 131887 T1223 oas.SolrTestCaseJ4.tearDown ###Ending 
> testDistribSearch
> [junit4:junit4]   1>      
> /solr/collections/control_collection/leader_elect/control_shard (1)
> [junit4:junit4]   1>       
> /solr/collections/control_collection/leader_elect/control_shard/election (1)
> [junit4:junit4]   1>        
> /solr/collections/control_collection/leader_elect/control_shard/election/88019246090289154-127.0.0.1:52276_solr_collection1-n_0000000000
>  (0)
> [junit4:junit4]   1>     /solr/collections/control_collection/leaders (1)
> [junit4:junit4]   1>      
> /solr/collections/control_collection/leaders/control_shard (0)
> [junit4:junit4]   1>      DATA:
> [junit4:junit4]   1>          {
> [junit4:junit4]   1>            "core":"collection1",
> [junit4:junit4]   1>            "node_name":"127.0.0.1:52276_solr",
> [junit4:junit4]   1>            "base_url":"http://127.0.0.1:52276/solr"}
> [junit4:junit4]   1>   /solr/clusterstate.json (0)
> [junit4:junit4]   1>   DATA:
> [junit4:junit4]   1>       {
> [junit4:junit4]   1>         "collection1":{
> [junit4:junit4]   1>           "shard1":{
> [junit4:junit4]   1>             "127.0.0.1:47557_solr_collection1":{
> [junit4:junit4]   1>               "shard":"shard1",
> [junit4:junit4]   1>               "leader":"true",
> [junit4:junit4]   1>               "roles":null,
> [junit4:junit4]   1>               "state":"active",
> [junit4:junit4]   1>               "core":"collection1",
> [junit4:junit4]   1>               "collection":"collection1",
> [junit4:junit4]   1>               "node_name":"127.0.0.1:47557_solr",
> [junit4:junit4]   1>               "base_url":"http://127.0.0.1:47557/solr"},
> [junit4:junit4]   1>             "127.0.0.1:14987_solr_collection1":{
> [junit4:junit4]   1>               "shard":"shard1",
> [junit4:junit4]   1>               "roles":null,
> [junit4:junit4]   1>               "state":"active",
> [junit4:junit4]   1>               "core":"collection1",
> [junit4:junit4]   1>               "collection":"collection1",
> [junit4:junit4]   1>               "node_name":"127.0.0.1:14987_solr",
> [junit4:junit4]   1>               "base_url":"http://127.0.0.1:14987/solr"}},
> [junit4:junit4]   1>           "shard2":{
> [junit4:junit4]   1>             "127.0.0.1:18607_solr_collection1":{
> [junit4:junit4]   1>               "shard":"shard2",
> [junit4:junit4]   1>               "roles":null,
> [junit4:junit4]   1>               "state":"active",
> [junit4:junit4]   1>               "core":"collection1",
> [junit4:junit4]   1>               "collection":"collection1",
> [junit4:junit4]   1>               "node_name":"127.0.0.1:18607_solr",
> [junit4:junit4]   1>               "base_url":"http://127.0.0.1:18607/solr"},
> [junit4:junit4]   1>             "127.0.0.1:40496_solr_collection1":{
> [junit4:junit4]   1>               "shard":"shard2",
> [junit4:junit4]   1>               "leader":"true",
> [junit4:junit4]   1>               "roles":null,
> [junit4:junit4]   1>               "state":"active",
> [junit4:junit4]   1>               "core":"collection1",
> [junit4:junit4]   1>               "collection":"collection1",
> [junit4:junit4]   1>               "node_name":"127.0.0.1:40496_solr",
> [junit4:junit4]   1>               
> "base_url":"http://127.0.0.1:40496/solr"}}},
> [junit4:junit4]   1>         
> "control_collection":{"control_shard":{"127.0.0.1:52276_solr_collection1":{
> [junit4:junit4]   1>               "shard":"control_shard",
> [junit4:junit4]   1>               "leader":"true",
> [junit4:junit4]   1>               "roles":null,
> [junit4:junit4]   1>               "state":"active",
> [junit4:junit4]   1>               "core":"collection1",
> [junit4:junit4]   1>               "collection":"control_collection",
> [junit4:junit4]   1>               "node_name":"127.0.0.1:52276_solr",
> [junit4:junit4]   1>               
> "base_url":"http://127.0.0.1:52276/solr"}}}}
> [junit4:junit4]   1>  /zookeeper (1)
> [junit4:junit4]   1>  DATA:
> [junit4:junit4]   1>      
> [junit4:junit4]   1> 
> [junit4:junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=FullSolrCloudTest -Dtests.method=testDistribSearch 
> -Dtests.seed=60281BBAEEA74A2D -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=es_CR -Dtests.timezone=America/Indianapolis 
> -Dtests.file.encoding=US-ASCII
> [junit4:junit4]   1> 
> [junit4:junit4]   2>
> [junit4:junit4]    > (@AfterClass output)
> [junit4:junit4]   2> 131996 T1223 oas.SolrTestCaseJ4.deleteCore ###deleteCore
> [junit4:junit4]   2> NOTE: test params are: codec=SimpleText, 
> sim=RandomSimilarityProvider(queryNorm=true,coord=true): {}, locale=es_CR, 
> timezone=America/Indianapolis
> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 
> 1.7.0_04 (64-bit)/cpus=16,threads=3,free=232254176,total=354418688
> [junit4:junit4]   2> NOTE: All tests run in this JVM: [ScriptEngineTest, 
> ReturnFieldsTest, UUIDFieldTest, OverseerTest, 
> TestNorwegianMinimalStemFilterFactory, TestNumberUtils, TestSystemIdResolver, 
> TestArbitraryIndexDir, TestCJKBigramFilterFactory, TestReplicationHandler, 
> TestCollationKeyRangeQueries, TestIndexSearcher, 
> TestIndonesianStemFilterFactory, TestOmitPositions, 
> WordBreakSolrSpellCheckerTest, TestHindiFilters, TimeZoneUtilsTest, 
> XsltUpdateRequestHandlerTest, QueryParsingTest, 
> TestPersianNormalizationFilterFactory, TestMultiCoreConfBootstrap, 
> FileBasedSpellCheckerTest, TestRussianLightStemFilterFactory, 
> TestJapaneseTokenizerFactory, TestSuggestSpellingConverter, 
> TestSolrDeletionPolicy1, TestShingleFilterFactory, PrimUtilsTest, 
> LengthFilterTest, TestNGramFilters, DistributedTermsComponentTest, 
> TestBeiderMorseFilterFactory, ResourceLoaderTest, TestReload, 
> URLClassifyProcessorTest, TermVectorComponentTest, 
> TestDefaultSimilarityFactory, TestGroupingSearch, TestPseudoReturnFields, 
> TestCoreContainer, DistanceFunctionTest, TestSolrDeletionPolicy2, 
> TestJmxMonitoredMap, CloudStateTest, TestPortugueseLightStemFilterFactory, 
> TestSort, TestCharFilters, TestElisionFilterFactory, HighlighterConfigTest, 
> TestRecovery, CommonGramsFilterFactoryTest, TestStressReorder, 
> QueryEqualityTest, TestPatternReplaceFilterFactory, TestUtils, MultiTermTest, 
> TestChineseTokenizerFactory, IndexReaderFactoryTest, UpdateParamsTest, 
> TestQueryTypes, TestWriterPerf, TestMultiWordSynonyms, 
> FieldAnalysisRequestHandlerTest, CloudStateUpdateTest, 
> TestQuerySenderNoQuery, TestHungarianLightStemFilterFactory, 
> TestPropInjectDefaults, TestEnglishMinimalStemFilterFactory, 
> TestJapaneseReadingFormFilterFactory, TestFastLRUCache, 
> TestKeepFilterFactory, TestDocSet, ZkSolrClientTest, TestXIncludeConfig, 
> TestRussianFilters, FullSolrCloudTest]
> [junit4:junit4]   2> 
> [junit4:junit4] Completed on J1 in 132.06s, 1 test, 1 error <<< FAILURES!
> [...truncated 525 lines...]
> [junit4:junit4]   2> 
> [junit4:junit4]   2> 69095 T6047 oasc.LeaderElector$1.process WARNING  
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /overseer_elect/election
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1249)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:266)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:263)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:67)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:263)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:91)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:57)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:125)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
> [junit4:junit4]   2> 
> [junit4:junit4]   2> 69095 T6047 oascc.ZkStateReader$3.process Updating live 
> nodes
> [junit4:junit4]   2> 69095 T6047 oascc.ZkStateReader$3.process WARNING 
> ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4]   2> 69096 T6047 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 94701 T6034 oasc.SyncStrategy.syncToMe Could not tell a 
> replica to recover org.apache.solr.client.solrj.SolrServerException: 
> IOException occured when talking to server at: 
> http://127.0.0.1:16057/solr/collection1
> [junit4:junit4]   2>  at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:414)
> [junit4:junit4]   2>  at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:182)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.SyncStrategy.syncToMe(SyncStrategy.java:237)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.SyncStrategy.syncReplicas(SyncStrategy.java:119)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.SyncStrategy.sync(SyncStrategy.java:82)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:158)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:151)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:96)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:57)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:125)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
> [junit4:junit4]   2> Caused by: org.apache.http.conn.ConnectTimeoutException: 
> Connect to 127.0.0.1:16057 timed out
> [junit4:junit4]   2>  at 
> org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:125)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
> [junit4:junit4]   2>  at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
> [junit4:junit4]   2>  at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
> [junit4:junit4]   2>  ... 11 more
> [junit4:junit4]   2> 
> [junit4:junit4]   2> 94702 T6034 oasc.RecoveryStrategy.close WARNING Stopping 
> recovery for core collection1 zkNodeName=127.0.0.1:16064_solr_collection1
> [junit4:junit4]   2> 94703 T6034 oasc.LeaderElector$1.process WARNING  
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /overseer/queue/qn-
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:643)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:244)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:241)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:67)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:241)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:236)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.ZkController.publish(ZkController.java:734)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:168)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:151)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:96)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:57)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:125)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
> [junit4:junit4]   2> 
> [junit4:junit4]   2> 94703 T6034 oascc.ZkStateReader$3.process Updating live 
> nodes
> [junit4:junit4]   2> 94703 T6034 oascc.ZkStateReader$3.process WARNING 
> ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4]   2> 94704 T6034 oasc.LeaderElector$1.process WARNING  
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /overseer_elect/election
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1249)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:266)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:263)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:67)
> [junit4:junit4]   2>  at 
> org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:263)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:91)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:57)
> [junit4:junit4]   2>  at 
> org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:125)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
> [junit4:junit4]   2>  at 
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
> [junit4:junit4]   2> 
> [junit4:junit4]   2> 94704 T6034 oaz.ClientCnxn$EventThread.run EventThread 
> shut down
> [junit4:junit4]   2> 94958 T6028 C182 P16064 oasc.SolrException.log SEVERE 
> shard update error StdNode: 
> http://127.0.0.1:16057/solr/collection1/:org.apache.solr.client.solrj.SolrServerException:
>  Server refused connection at: http://127.0.0.1:16057/solr/collection1
> [junit4:junit4]   2>          at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:407)
> [junit4:junit4]   2>          at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:182)
> [junit4:junit4]   2>          at 
> org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:336)
> [junit4:junit4]   2>          at 
> org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:317)
> [junit4:junit4]   2>          at 
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> [junit4:junit4]   2>          at 
> java.util.concurrent.FutureTask.run(FutureTask.java:166)
> [junit4:junit4]   2>          at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [junit4:junit4]   2>          at 
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> [junit4:junit4]   2>          at 
> java.util.concurrent.FutureTask.run(FutureTask.java:166)
> [junit4:junit4]   2>          at 
> org.apache.solr.update.SolrCmdDistributor$BoundedExecutor$1.run(SolrCmdDistributor.java:538)
> [junit4:junit4]   2>          at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> [junit4:junit4]   2>          at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]   2>          at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2>  Caused by: 
> org.apache.http.conn.HttpHostConnectException: Connection to 
> http://127.0.0.1:16057 refused
> [junit4:junit4]   2>          at 
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
> [junit4:junit4]   2>          at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
> [junit4:junit4]   2>          ... 12 more
> [junit4:junit4]   2>  Caused by: java.net.ConnectException: Operation timed 
> out
> [junit4:junit4]   2>          at 
> java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit4:junit4]   2>          at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> [junit4:junit4]   2>          at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> [junit4:junit4]   2>          at 
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> [junit4:junit4]   2>          at 
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> [junit4:junit4]   2>          at java.net.Socket.connect(Socket.java:579)
> [junit4:junit4]   2>          at 
> org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
> [junit4:junit4]   2>          at 
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
> [junit4:junit4]   2>          ... 20 more
> [junit4:junit4]   2>  
> [junit4:junit4]   2> 160627 T5988 oas.SolrTestCaseJ4.endTrackingSearchers 
> SEVERE ERROR: SolrIndexSearcher opens=24 closes=23
> [junit4:junit4]   2> NOTE: test params are: codec=Lucene40: 
> {range_facet_l=PostingsFormat(name=SimpleText), text=Pulsing40(freqCutoff=19 
> minBlockSize=16 maxBlockSize=45), _version_=PostingsFormat(name=SimpleText), 
> rnd_b=Pulsing40(freqCutoff=20 minBlockSize=16 maxBlockSize=45), 
> intDefault=PostingsFormat(name=SimpleText), id=Pulsing40(freqCutoff=20 
> minBlockSize=16 maxBlockSize=45), timestamp=PostingsFormat(name=SimpleText), 
> a_t=PostingsFormat(name=SimpleText), range_facet_sl=Pulsing40(freqCutoff=19 
> minBlockSize=16 maxBlockSize=45), 
> range_facet_si=MockVariableIntBlock(baseBlockSize=25), 
> other_tl1=PostingsFormat(name=SimpleText), 
> multiDefault=MockVariableIntBlock(baseBlockSize=25), 
> a_si=MockVariableIntBlock(baseBlockSize=25)}, 
> sim=RandomSimilarityProvider(queryNorm=true,coord=false): {}, locale=en, 
> timezone=Etc/GMT-12
> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 
> 1.7.0_04 (64-bit)/cpus=16,threads=6,free=88472488,total=354418688
> [junit4:junit4]   2> NOTE: All tests run in this JVM: [ScriptEngineTest, 
> ReturnFieldsTest, UUIDFieldTest, OverseerTest, 
> TestNorwegianMinimalStemFilterFactory, TestNumberUtils, TestSystemIdResolver, 
> TestArbitraryIndexDir, TestCJKBigramFilterFactory, TestReplicationHandler, 
> TestCollationKeyRangeQueries, TestIndexSearcher, 
> TestIndonesianStemFilterFactory, TestOmitPositions, 
> WordBreakSolrSpellCheckerTest, TestHindiFilters, TimeZoneUtilsTest, 
> XsltUpdateRequestHandlerTest, QueryParsingTest, 
> TestPersianNormalizationFilterFactory, TestMultiCoreConfBootstrap, 
> FileBasedSpellCheckerTest, TestRussianLightStemFilterFactory, 
> TestJapaneseTokenizerFactory, TestSuggestSpellingConverter, 
> TestSolrDeletionPolicy1, TestShingleFilterFactory, PrimUtilsTest, 
> LengthFilterTest, TestNGramFilters, DistributedTermsComponentTest, 
> TestBeiderMorseFilterFactory, ResourceLoaderTest, TestReload, 
> URLClassifyProcessorTest, TermVectorComponentTest, 
> TestDefaultSimilarityFactory, TestGroupingSearch, TestPseudoReturnFields, 
> TestCoreContainer, DistanceFunctionTest, TestSolrDeletionPolicy2, 
> TestJmxMonitoredMap, CloudStateTest, TestPortugueseLightStemFilterFactory, 
> TestSort, TestCharFilters, TestElisionFilterFactory, HighlighterConfigTest, 
> TestRecovery, CommonGramsFilterFactoryTest, TestStressReorder, 
> QueryEqualityTest, TestPatternReplaceFilterFactory, TestUtils, MultiTermTest, 
> TestChineseTokenizerFactory, IndexReaderFactoryTest, UpdateParamsTest, 
> TestQueryTypes, TestWriterPerf, TestMultiWordSynonyms, 
> FieldAnalysisRequestHandlerTest, CloudStateUpdateTest, 
> TestQuerySenderNoQuery, TestHungarianLightStemFilterFactory, 
> TestPropInjectDefaults, TestEnglishMinimalStemFilterFactory, 
> TestJapaneseReadingFormFilterFactory, TestFastLRUCache, 
> TestKeepFilterFactory, TestDocSet, ZkSolrClientTest, TestXIncludeConfig, 
> TestRussianFilters, FullSolrCloudTest, RecoveryZkTest, 
> SpellCheckComponentTest, LeaderElectionTest, 
> DistributedSpellCheckComponentTest, ZkControllerTest, TestRandomFaceting, 
> SpellCheckCollatorTest, TestExtendedDismaxParser, SoftAutoCommitTest, 
> AutoCommitTest, BasicZkTest, TestPhoneticFilterFactory, TestRangeQuery, 
> PeerSyncTest, SortByFunctionTest, SignatureUpdateProcessorFactoryTest, 
> TestBinaryResponseWriter, SampleTest, CoreAdminHandlerTest, TestTrie, 
> DirectSolrConnectionTest, TestRemoteStreaming, CacheHeaderTest, 
> PolyFieldTest, SpatialFilterTest, TestCSVResponseWriter, DocumentBuilderTest, 
> MBeansHandlerTest, CurrencyFieldTest, IndexSchemaTest, 
> BinaryUpdateRequestHandlerTest, TermsComponentTest, SolrInfoMBeanTest, 
> StandardRequestHandlerTest, TestPHPSerializedResponseWriter, 
> PingRequestHandlerTest, SuggesterWFSTTest, JsonLoaderTest, 
> DocumentAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, 
> SolrPluginUtilsTest, TestValueSourceCache, RequestHandlersTest, 
> SuggesterFSTTest, TestQueryUtils, UniqFieldsUpdateProcessorFactoryTest, 
> CSVRequestHandlerTest, TestSearchPerf, BadIndexSchemaTest, TestPropInject, 
> PrimitiveFieldTypeTest, FieldMutatingUpdateProcessorTest, TestLFUCache, 
> TestCollationField, OutputWriterTest, TestJapaneseKatakanaStemFilterFactory, 
> TestConfig, SpellPossibilityIteratorTest, TestPluginEnable, 
> TestHyphenationCompoundWordTokenFilterFactory, TestIBSimilarityFactory, 
> OpenExchangeRatesOrgProviderTest, TestCzechStemFilterFactory, 
> TestDelimitedPayloadTokenFilterFactory, TestStandardFactories, TestBadConfig, 
> TestKeywordMarkerFilterFactory, CommonGramsQueryFilterFactoryTest, 
> TestGreekStemFilterFactory, RAMDirectoryFactoryTest, 
> SnowballPorterFilterFactoryTest, TestGermanNormalizationFilterFactory, 
> TestItalianLightStemFilterFactory, TestPorterStemFilterFactory, 
> TestGermanMinimalStemFilterFactory, TestBulgarianStemFilterFactory, 
> TestTurkishLowerCaseFilterFactory, TestFinnishLightStemFilterFactory, 
> TestLatvianStemFilterFactory, TestDocumentBuilder, TestChineseFilterFactory, 
> TestCollationKeyFilterFactory, TestSynonymMap, SyncSliceTest]
> [junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest 
> -Dtests.seed=60281BBAEEA74A2D -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=en -Dtests.timezone=Etc/GMT-12 -Dtests.file.encoding=US-ASCII
> [junit4:junit4]   2> 
> [junit4:junit4] ERROR   0.00s J1 | SyncSliceTest (suite)
> [junit4:junit4]    > Throwable #1: java.lang.AssertionError: ERROR: 
> SolrIndexSearcher opens=24 closes=23
> [junit4:junit4]    >  at 
> __randomizedtesting.SeedInfo.seed([60281BBAEEA74A2D]:0)
> [junit4:junit4]    >  at org.junit.Assert.fail(Assert.java:93)
> [junit4:junit4]    >  at 
> org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:216)
> [junit4:junit4]    >  at 
> org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:82)
> [junit4:junit4]    >  at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown 
> Source)
> [junit4:junit4]    >  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit4:junit4]    >  at java.lang.reflect.Method.invoke(Method.java:601)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:754)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
> [junit4:junit4]    >
> [junit4:junit4] Completed on J1 in 160.65s, 1 test, 1 failure <<< FAILURES!
> [...truncated 19 lines...]
> 
> [...truncated 13464 lines...]
> 
> [...truncated 13464 lines...]
> 
> [...truncated 13464 lines...]
> 
> [...truncated 13464 lines...]
> 
> [...truncated 13410 lines...]
> [junit4:junit4]   2> 160627 T5988 oas.SolrTestCaseJ4.endTrackingSearchers 
> SEVERE ERROR: SolrIndexSearcher opens=24 closes=23
> [junit4:junit4]   2> NOTE: test params are: codec=Lucene40: 
> {range_facet_l=PostingsFormat(name=SimpleText), text=Pulsing40(freqCutoff=19 
> minBlockSize=16 maxBlockSize=45), _version_=PostingsFormat(name=SimpleText), 
> rnd_b=Pulsing40(freqCutoff=20 minBlockSize=16 maxBlockSize=45), 
> intDefault=PostingsFormat(name=SimpleText), id=Pulsing40(freqCutoff=20 
> minBlockSize=16 maxBlockSize=45), timestamp=PostingsFormat(name=SimpleText), 
> a_t=PostingsFormat(name=SimpleText), range_facet_sl=Pulsing40(freqCutoff=19 
> minBlockSize=16 maxBlockSize=45), 
> range_facet_si=MockVariableIntBlock(baseBlockSize=25), 
> other_tl1=PostingsFormat(name=SimpleText), 
> multiDefault=MockVariableIntBlock(baseBlockSize=25), 
> a_si=MockVariableIntBlock(baseBlockSize=25)}, 
> sim=RandomSimilarityProvider(queryNorm=true,coord=false): {}, locale=en, 
> timezone=Etc/GMT-12
> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 
> 1.7.0_04 (64-bit)/cpus=16,threads=6,free=88472488,total=354418688
> [junit4:junit4]   2> NOTE: All tests run in this JVM: [ScriptEngineTest, 
> ReturnFieldsTest, UUIDFieldTest, OverseerTest, 
> TestNorwegianMinimalStemFilterFactory, TestNumberUtils, TestSystemIdResolver, 
> TestArbitraryIndexDir, TestCJKBigramFilterFactory, TestReplicationHandler, 
> TestCollationKeyRangeQueries, TestIndexSearcher, 
> TestIndonesianStemFilterFactory, TestOmitPositions, 
> WordBreakSolrSpellCheckerTest, TestHindiFilters, TimeZoneUtilsTest, 
> XsltUpdateRequestHandlerTest, QueryParsingTest, 
> TestPersianNormalizationFilterFactory, TestMultiCoreConfBootstrap, 
> FileBasedSpellCheckerTest, TestRussianLightStemFilterFactory, 
> TestJapaneseTokenizerFactory, TestSuggestSpellingConverter, 
> TestSolrDeletionPolicy1, TestShingleFilterFactory, PrimUtilsTest, 
> LengthFilterTest, TestNGramFilters, DistributedTermsComponentTest, 
> TestBeiderMorseFilterFactory, ResourceLoaderTest, TestReload, 
> URLClassifyProcessorTest, TermVectorComponentTest, 
> TestDefaultSimilarityFactory, TestGroupingSearch, TestPseudoReturnFields, 
> TestCoreContainer, DistanceFunctionTest, TestSolrDeletionPolicy2, 
> TestJmxMonitoredMap, CloudStateTest, TestPortugueseLightStemFilterFactory, 
> TestSort, TestCharFilters, TestElisionFilterFactory, HighlighterConfigTest, 
> TestRecovery, CommonGramsFilterFactoryTest, TestStressReorder, 
> QueryEqualityTest, TestPatternReplaceFilterFactory, TestUtils, MultiTermTest, 
> TestChineseTokenizerFactory, IndexReaderFactoryTest, UpdateParamsTest, 
> TestQueryTypes, TestWriterPerf, TestMultiWordSynonyms, 
> FieldAnalysisRequestHandlerTest, CloudStateUpdateTest, 
> TestQuerySenderNoQuery, TestHungarianLightStemFilterFactory, 
> TestPropInjectDefaults, TestEnglishMinimalStemFilterFactory, 
> TestJapaneseReadingFormFilterFactory, TestFastLRUCache, 
> TestKeepFilterFactory, TestDocSet, ZkSolrClientTest, TestXIncludeConfig, 
> TestRussianFilters, FullSolrCloudTest, RecoveryZkTest, 
> SpellCheckComponentTest, LeaderElectionTest, 
> DistributedSpellCheckComponentTest, ZkControllerTest, TestRandomFaceting, 
> SpellCheckCollatorTest, TestExtendedDismaxParser, SoftAutoCommitTest, 
> AutoCommitTest, BasicZkTest, TestPhoneticFilterFactory, TestRangeQuery, 
> PeerSyncTest, SortByFunctionTest, SignatureUpdateProcessorFactoryTest, 
> TestBinaryResponseWriter, SampleTest, CoreAdminHandlerTest, TestTrie, 
> DirectSolrConnectionTest, TestRemoteStreaming, CacheHeaderTest, 
> PolyFieldTest, SpatialFilterTest, TestCSVResponseWriter, DocumentBuilderTest, 
> MBeansHandlerTest, CurrencyFieldTest, IndexSchemaTest, 
> BinaryUpdateRequestHandlerTest, TermsComponentTest, SolrInfoMBeanTest, 
> StandardRequestHandlerTest, TestPHPSerializedResponseWriter, 
> PingRequestHandlerTest, SuggesterWFSTTest, JsonLoaderTest, 
> DocumentAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, 
> SolrPluginUtilsTest, TestValueSourceCache, RequestHandlersTest, 
> SuggesterFSTTest, TestQueryUtils, UniqFieldsUpdateProcessorFactoryTest, 
> CSVRequestHandlerTest, TestSearchPerf, BadIndexSchemaTest, TestPropInject, 
> PrimitiveFieldTypeTest, FieldMutatingUpdateProcessorTest, TestLFUCache, 
> TestCollationField, OutputWriterTest, TestJapaneseKatakanaStemFilterFactory, 
> TestConfig, SpellPossibilityIteratorTest, TestPluginEnable, 
> TestHyphenationCompoundWordTokenFilterFactory, TestIBSimilarityFactory, 
> OpenExchangeRatesOrgProviderTest, TestCzechStemFilterFactory, 
> TestDelimitedPayloadTokenFilterFactory, TestStandardFactories, TestBadConfig, 
> TestKeywordMarkerFilterFactory, CommonGramsQueryFilterFactoryTest, 
> TestGreekStemFilterFactory, RAMDirectoryFactoryTest, 
> SnowballPorterFilterFactoryTest, TestGermanNormalizationFilterFactory, 
> TestItalianLightStemFilterFactory, TestPorterStemFilterFactory, 
> TestGermanMinimalStemFilterFactory, TestBulgarianStemFilterFactory, 
> TestTurkishLowerCaseFilterFactory, TestFinnishLightStemFilterFactory, 
> TestLatvianStemFilterFactory, TestDocumentBuilder, TestChineseFilterFactory, 
> TestCollationKeyFilterFactory, TestSynonymMap, SyncSliceTest]
> [junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest 
> -Dtests.seed=60281BBAEEA74A2D -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=en -Dtests.timezone=Etc/GMT-12 -Dtests.file.encoding=US-ASCII
> [junit4:junit4]   2> 
> [junit4:junit4] ERROR   0.00s J1 | SyncSliceTest (suite)
> [junit4:junit4]    > Throwable #1: java.lang.AssertionError: ERROR: 
> SolrIndexSearcher opens=24 closes=23
> [junit4:junit4]    >  at 
> __randomizedtesting.SeedInfo.seed([60281BBAEEA74A2D]:0)
> [junit4:junit4]    >  at org.junit.Assert.fail(Assert.java:93)
> [junit4:junit4]    >  at 
> org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:216)
> [junit4:junit4]    >  at 
> org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:82)
> [junit4:junit4]    >  at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown 
> Source)
> [junit4:junit4]    >  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit4:junit4]    >  at java.lang.reflect.Method.invoke(Method.java:601)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:754)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4]    >  at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
> [junit4:junit4]    >  at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
> [junit4:junit4]    >
> [junit4:junit4] Completed on J1 in 160.65s, 1 test, 1 failure <<< FAILURES!
> [junit4:junit4]  
> [junit4:junit4] JVM J0:     1.03 ..   870.40 =   869.37s
> [junit4:junit4] JVM J1:     1.19 ..   972.46 =   971.27s
> [junit4:junit4] Execution time total: 16 minutes 12 seconds
> [junit4:junit4] Tests summary: 305 suites, 1085 tests, 1 suite-level error, 1 
> error, 7 ignored (2 assumptions)
> 
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/build.xml:29:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/solr/build.xml:144:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/module-build.xml:62:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/common-build.xml:1094:
>  The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/common-build.xml:789:
>  There were test failures: 305 suites, 1085 tests, 1 suite-level error, 1 
> error, 7 ignored (2 assumptions)
> 
> Total time: 29 minutes 5 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]

- Mark Miller
lucidimagination.com












---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to