[
https://issues.apache.org/jira/browse/HDFS-17286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-17286:
---------------------------------
Target Version/s: 3.5.1
> Add UDP as a transfer protocol for HDFS
> ---------------------------------------
>
> Key: HDFS-17286
> URL: https://issues.apache.org/jira/browse/HDFS-17286
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
> Reporter: Xing Lin
> Priority: Major
> Attachments: active.png, observer.png
>
>
> Right now, every connection in HDFS is based on RPC/IPC which is based on
> TCP. Connection is re-used based on ConnectionID, which includes RpcTimeout
> as part of the key to identify a connection. The consequence is if we want to
> use a different rpc timeout between two hosts, this would create different
> TCP connections.
> A use case which motivated us to consider UDP is getHAServiceState() in
> ObserverReadProxyProvider. We'd like getHAServiceState() to time out with a
> much smaller timeout threshold and move to probe next Namenode. To support
> this, we used an executorService and set a timeout for the task in
> HDFS-17030. This implementation can be improved by using UDP to query
> HAServiceState. getHAServiceState() does not have to be very reliable, as we
> can always fall back to the active.
> Another motivation is it seems 5~10% of RPC calls hitting our
> active/observers are GetHAServiceState(). If we can move them off to the UDP
> server, that can hopefully improve RPC latency.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
