[
https://issues.apache.org/jira/browse/HDFS-13248?focusedWorklogId=744310&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744310
]
ASF GitHub Bot logged work on HDFS-13248:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Mar/22 18:31
Start Date: 18/Mar/22 18:31
Worklog Time Spent: 10m
Work Description: omalley commented on a change in pull request #4081:
URL: https://github.com/apache/hadoop/pull/4081#discussion_r830256594
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
##########
@@ -1899,7 +1907,27 @@ private void verifySoftwareVersion(DatanodeRegistration
dnReg)
}
}
- private static String getClientMachine() {
+ private String getClientMachine() {
+ if (ipProxyUsers != null) {
+ // Get the real user (or effective if it isn't a proxy user)
+ UserGroupInformation user = Server.getRemoteUser().getRealUserOrSelf();
+ if (ArrayUtils.contains(ipProxyUsers, user.getShortUserName())) {
+ CallerContext context = CallerContext.getCurrent();
+ if (context != null && context.isContextValid()) {
+ String cc = context.getContext();
+ // if the rpc has a caller context of "clientIp:1.2.3.4,CLI",
+ // return "1.2.3.4" as the client machine.
+ String key = CallerContext.CLIENT_IP_STR +
+ CallerContext.Builder.KEY_VALUE_SEPARATOR;
+ int posn = cc.indexOf(key);
+ if (posn != -1) {
+ posn += key.length();
+ int end = cc.indexOf(",", posn);
+ return end == -1 ? cc.substring(posn) : cc.substring(posn, end);
Review comment:
The Routers always puts the "clientIp:" at the front of the caller
context. This code doesn't assume that, but it means that it is much less
sensitive that that kind of problem.
We can't pre-compute using the caller context, because it is sent along with
each rpc request as part of the RpcRequestHeader. One cost of that is that we
have to compute it as part of the call handling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 744310)
Time Spent: 2.5h (was: 2h 20m)
> RBF: Namenode need to choose block location for the client
> ----------------------------------------------------------
>
> Key: HDFS-13248
> URL: https://issues.apache.org/jira/browse/HDFS-13248
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Wu Weiwei
> Assignee: Íñigo Goiri
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13248.000.patch, HDFS-13248.001.patch,
> HDFS-13248.002.patch, HDFS-13248.003.patch, HDFS-13248.004.patch,
> HDFS-13248.005.patch, HDFS-Router-Data-Locality.odt, RBF Data Locality
> Design.pdf, clientMachine-call-path.jpeg, debug-info-1.jpeg, debug-info-2.jpeg
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When execute a put operation via router, the NameNode will choose block
> location for the router, not for the real client. This will affect the file's
> locality.
> I think on both NameNode and Router, we should add a new addBlock method, or
> add a parameter for the current addBlock method, to pass the real client
> information.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
