Gus Heck created SOLR-13036:
-------------------------------
Summary: JettySolrRunner's port retry is broken
Key: SOLR-13036
URL: https://issues.apache.org/jira/browse/SOLR-13036
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Tests
Affects Versions: master (8.0)
Reporter: Gus Heck
I was running tests and hit the following stack trace which shows the
BindException getting wrapped by an IOException, which then evades our port
already bound retry logic in JettySolrRunner#retryOnPortBindFailure() which is
catching BindException directly. The result is that we never retry as intended.
This observation from reading the code is supported by the fact that the
logging statement starting with "Port in use..." did not appear in the log.
{code:java}
build.out4.txt- [beaster] [18:06:13.618] ERROR 25.9s J15 |
TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader <<<
build.out4.txt: [beaster] > Throwable #1: java.io.IOException: Failed to bind
to /127.0.0.1:36497
build.out4.txt- [beaster] > at
__randomizedtesting.SeedInfo.seed([FD0124C33BA78623:83EA04D3F8C08919]:0)
build.out4.txt- [beaster] > at
org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
build.out4.txt- [beaster] > at
org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
build.out4.txt- [beaster] > at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
build.out4.txt- [beaster] > at
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
build.out4.txt- [beaster] > at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
build.out4.txt- [beaster] > at
org.eclipse.jetty.server.Server.doStart(Server.java:394)
build.out4.txt- [beaster] > at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
build.out4.txt- [beaster] > at
org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:510)
build.out4.txt- [beaster] > at
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:449)
build.out4.txt- [beaster] > at
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:417)
build.out4.txt- [beaster] > at
org.apache.solr.cloud.TestCloudConsistency.addDocWhenOtherReplicasAreNetworkPartitioned(TestCloudConsistency.java:225)
build.out4.txt- [beaster] > at
org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:136)
build.out4.txt- [beaster] > at
org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:98)
build.out4.txt- [beaster] > at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
build.out4.txt- [beaster] > at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
build.out4.txt- [beaster] > at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
build.out4.txt- [beaster] > at java.lang.reflect.Method.invoke(Method.java:498)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
build.out4.txt- [beaster] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
build.out4.txt- [beaster] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
build.out4.txt- [beaster] > at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
build.out4.txt- [beaster] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
build.out4.txt- [beaster] > at java.lang.Thread.run(Thread.java:748)
build.out4.txt- [beaster] > Caused by: java.net.BindException: Address already
in use
build.out4.txt- [beaster] > at sun.nio.ch.Net.bind0(Native Method)
build.out4.txt- [beaster] > at sun.nio.ch.Net.bind(Net.java:433)
build.out4.txt- [beaster] > at sun.nio.ch.Net.bind(Net.java:425)
build.out4.txt- [beaster] > at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
build.out4.txt- [beaster] > at
sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
build.out4.txt- [beaster] > at
org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
build.out4.txt- [beaster] > ... 51 more
{code}
SOLR-7339 first noticed this but seems to have not tracked it down because
other bigger problems were extant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
