[
https://issues.apache.org/jira/browse/HDFS-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268218#comment-14268218
]
Chris Nauroth commented on HDFS-7553:
-------------------------------------
Hi [~xieliang007]. If I understand correctly, you were trying to reset the
port to 0, which would then retrigger the logic for dynamically selecting a new
bind port. However, I don't think this is right, because all of this is
happening in a {{Before}} method before each test runs. The failure report
shows the bind failure part-way through one test after a NameNode restart, so
it doesn't look like an interaction spanning multiple tests that run into a
port conflict with each other.
Another possibility here might be that NameNode shutdown waits for completion
of the RPC server shutdown, but not the HTTP server shutdown. This happens in
{{NameNode#join}}:
{code}
public void join() {
try {
rpcServer.join();
} catch (InterruptedException ie) {
LOG.info("Caught interrupted exception ", ie);
}
}
{code}
However, there is no call to {{HttpServer2#join}}. This would ultimately block
waiting for Jetty's thread pool to shut down. Perhaps we have a race condition
where Jetty remains running bound to that port for a brief window after
NameNode shutdown. Then, when the restart happens, we have a conflict on that
port.
> fix the TestDFSUpgradeWithHA due to BindException
> -------------------------------------------------
>
> Key: HDFS-7553
> URL: https://issues.apache.org/jira/browse/HDFS-7553
> Project: Hadoop HDFS
> Issue Type: Test
> Components: test
> Affects Versions: 2.7.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HDFS-7553-001.txt
>
>
> see
> https://builds.apache.org/job/PreCommit-HDFS-Build/9092//testReport/org.apache.hadoop.hdfs.server.namenode.ha/TestDFSUpgradeWithHA/testNfsUpgrade/
> :
> Error Message
> Port in use: localhost:57896
> Stacktrace
> java.net.BindException: Port in use: localhost:57896
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at
> org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
> at
> org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:868)
> at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:809)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:142)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:704)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:591)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:763)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:747)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1443)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1815)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.restartNameNode(MiniDFSCluster.java:1796)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA.testNfsUpgrade(TestDFSUpgradeWithHA.java:285)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
