[
https://issues.apache.org/jira/browse/HDFS-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845563#comment-16845563
]
Vinayakumar B commented on HDFS-14440:
--------------------------------------
{quote}My only concern was the replacement of invokeSequential with
invokeConcurrent.
The functionality will be the same but the now we have a trade-off between
latency and number of rpc calls.
{quote}
Yes, its a trade-off. But we should be more lean towards overall latency of
Client's operation than number of RPC calls from routers(which occurs in
parallel and results in no/negligible overhead).
As explained by [~ayushtkn] earlier, in case of File-not-Exist-already, total
number of RPC remains same, but latency improves a lot. This is usually the
most frequent case.
In case of file-already-exist, total number of RPCs, will be same as number of
remote locations. Since executed in parallel, overall latency will be
negligible. This case is less frequent as compared to former case.
I guess, this should be fair enough trade-off for client's RPC performance
improvement.
> RBF: Optimize the file write process in case of multiple destinations.
> ----------------------------------------------------------------------
>
> Key: HDFS-14440
> URL: https://issues.apache.org/jira/browse/HDFS-14440
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Attachments: HDFS-14440-HDFS-13891-01.patch,
> HDFS-14440-HDFS-13891-02.patch, HDFS-14440-HDFS-13891-03.patch,
> HDFS-14440-HDFS-13891-04.patch, HDFS-14440-HDFS-13891-05.patch,
> HDFS-14440-HDFS-13891-06.patch
>
>
> In case of multiple destinations, We need to check if the file already exists
> in one of the subclusters for which we use the existing getBlockLocation()
> API which is by default a sequential Call,
> In an ideal scenario where the file needs to be created each subcluster shall
> be checked sequentially, this can be done concurrently to save time.
> In another case where the file is found and if the last block is null, we
> need to do getFileInfo to all the locations to get the location where the
> file exists. This also can be prevented by use of ConcurrentCall since we
> shall be having the remoteLocation to where the getBlockLocation returned a
> non null entry.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
