[
https://issues.apache.org/jira/browse/HDFS-13248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16812173#comment-16812173
]
He Xiaoqiao commented on HDFS-13248:
------------------------------------
Thanks [~elgoiri],
{quote}My main concern with modifying ClientProtocol is that it requires the
client itself to change.
The change is backwards compatible but for it to work you need the client to be
up to date.
>From our experience, this is pretty challenging.{quote}
The documents does not depict scope of changing detailed. Actually, we only
need `modify the Namenode and the Router` rather than require change client if
we push to use modifying {{ClientProtocol}}
(1) All client keep to use current interface of {{ClientProtocol}};
(2) When router receive RPC request, it get client hostname firstly, then
switch to invoke additional method which include parameter {{ClientMachine}}
to Namenode;
(3) When RPC request to Namenode, it determine to use {{clientMachine}} if not
null or get client hostname by {{Server.getRemoteAddress}} if {{ClientMachine}}
is null.
In one word, We need to modify Namenode and Router but Client.
{quote}BTW, we could do right away the one that
RouterRpcServer#getBlockLocations() reorders the destinations.{quote}
I agree to reorder at Router again, but I think it's not necessary if we can
handle this case under this ticket. Since reorder operation may reduce QPS of
router. Please correct me if there are something wrong. FYI.
> RBF: Namenode need to choose block location for the client
> ----------------------------------------------------------
>
> Key: HDFS-13248
> URL: https://issues.apache.org/jira/browse/HDFS-13248
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Weiwei Wu
> Assignee: Íñigo Goiri
> Priority: Major
> Attachments: HDFS-13248.000.patch, HDFS-13248.001.patch,
> HDFS-13248.002.patch, HDFS-13248.003.patch, HDFS-13248.004.patch,
> HDFS-13248.005.patch, HDFS-Router-Data-Locality.odt, RBF Data Locality
> Design.pdf, clientMachine-call-path.jpeg, debug-info-1.jpeg, debug-info-2.jpeg
>
>
> When execute a put operation via router, the NameNode will choose block
> location for the router, not for the real client. This will affect the file's
> locality.
> I think on both NameNode and Router, we should add a new addBlock method, or
> add a parameter for the current addBlock method, to pass the real client
> information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
