[
https://issues.apache.org/jira/browse/HDFS-17357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17811182#comment-17811182
]
ASF GitHub Bot commented on HDFS-17357:
---------------------------------------
hfutatzhanghb commented on code in PR #6502:
URL: https://github.com/apache/hadoop/pull/6502#discussion_r1467398721
##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/net/NioInetPeer.java:
##########
@@ -91,7 +91,11 @@ public void close() throws IOException {
try {
in.close();
} finally {
- out.close();
+ try {
+ out.close();
+ } finally {
+ socket.close();
+ }
Review Comment:
@LiuGuH Hi, Leave one comment. Please check DFSUtilClient#peerFromSocket.
https://github.com/apache/hadoop/blob/897f446d54315b70754eb8023cf195c902bca98e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java#L799
Should we better set keep-alive time here or not ?
> EC: NioInetPeer.close() should close socket connection.
> -------------------------------------------------------
>
> Key: HDFS-17357
> URL: https://issues.apache.org/jira/browse/HDFS-17357
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: liuguanghua
> Assignee: liuguanghua
> Priority: Major
> Labels: pull-request-available
>
> NioInetPeer.close() now do not close socket connection.
>
> In my environment，all data were stored with EC.
> And I found 3w+ connections leakage in datanode . And I found many warn
> message as blew.
> 2024-01-22 15:27:57,500 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:
> hostname:50010:DataXceiverServer
>
> When any Exception is found in DataXceiverServer, it will execute clostStream.
> IOUtils.closeStream(peer) -> Peer.close() -> NioInetPeer.close()
> But NioInetPeer.close() is not invoked with close socket connection. And
> this will lead to connection leakage.
> Other subClass of Peer's close() is implemented with socket.close(). See
> EncryptedPeer, DomainPeer, BasicInetPeer
>
>
> This solution can be reporduced as following:
> (1) Client write data to HDFS
> (2) datanode Xceiver count max to DFS_DATANODE_MAX_RECEIVER_THREADS_KEY ， the
> new Xceiver will fail and throw IOException . And the socket will not release.
> (3) Client crash for that no new data will be added or client.close is
> executed.
> (4) There will be socket connection leakage between datanodes.
>
>
> The connection leakage like this
> dn1
> dn1:57042 dn2:50010 ESTABLISHED
> dn2
> dn2:50010 dn1:57042 ESTABLISHED
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
