[
https://issues.apache.org/jira/browse/HADOOP-18346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576953#comment-17576953
]
Shawn edited comment on HADOOP-18346 at 8/8/22 7:19 PM:
--------------------------------------------------------
Seems we already have a JIRA for this issue
https://issues.apache.org/jira/browse/HADOOP-18365
was (Author: sokui):
Seems we already have a JIRA for this issue
> Suggest to update the IP address of ConnectionId when server of Connection
> got updated
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-18346
> URL: https://issues.apache.org/jira/browse/HADOOP-18346
> Project: Hadoop Common
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.3.1
> Reporter: Shawn
> Priority: Critical
>
> When use ProtobufRpcEngine to get RPC proxy, we apply the InetSocketAddress
> addr, and this address is then kept in the ConnectionId (see
> [here|https://github.com/apache/hadoop/blob/rel/release-3.3.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java#L139]
> and
> [here|https://github.com/apache/hadoop/blob/rel/release-3.3.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java#L1681]).
> When the destination IP of the connection changes, we have code to update the
> server variable in the connection (see
> [here|https://github.com/apache/hadoop/blob/rel/release-3.3.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java#L641]).
> This makes the retry of the connection to succeed.
> However, the above logic only temporary fix the connection. After the RPC
> call of the client, the connection will be closed. Next time when we use the
> same proxy to do the RPC call, we will again to fail the first connection and
> succeed the second one after the server IP got updates.
> My suggestion is to not only update the server variable in the connection,
> but also update the address of the ConnectionId in the Invoker. This way, we
> can reuse the proxy with the latest IP address.
> The use case for this is: in Apache ozone project, we initialize the RPC
> proxy with a fixed InetSocketAddress (for example, datanode needs to set up
> the RPC proxy to scm). But if lately the server IP changes (e.g. in k8s, the
> IP of a pod is very likely to change), every time the proxy needs two calls
> (1st fails and the 2nd succeeds with the updated IP). By using the above
> solution, we can eliminate this issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
