[
https://issues.apache.org/jira/browse/HDFS-13248?focusedWorklogId=744607&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744607
]
ASF GitHub Bot logged work on HDFS-13248:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Mar/22 08:41
Start Date: 19/Mar/22 08:41
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #4081:
URL: https://github.com/apache/hadoop/pull/4081#discussion_r830458682
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeRpcServer.java
##########
@@ -59,5 +70,87 @@ public void testNamenodeRpcBindAny() throws IOException {
conf.unset(DFS_NAMENODE_RPC_BIND_HOST_KEY);
}
}
+
+ /**
+ * Get the preferred DataNode location for the first block of the
+ * given file.
+ * @param fs The file system to use
+ * @param p The path to use
+ * @return the preferred host to get the data
+ */
+ private static String getPreferredLocation(DistributedFileSystem fs,
+ Path p) throws IOException{
+ // Use getLocatedBlocks because it is the basis for HDFS open,
+ // but provides visibility into which host will be used.
+ LocatedBlocks blocks = fs.getClient()
+ .getLocatedBlocks(p.toUri().getPath(), 0);
+ return blocks.get(0).getLocations()[0].getHostName();
+ }
+
+ /**
+ * A test to make sure that if an authorized user adds "clientIp:" to their
+ * caller context, it will be used to make locality decisions on the NN.
+ */
+ @Test
+ public void testNamenodeRpcClientIpProxy() throws IOException {
+ // Because of the randomness of the NN assigning DN, we run multiple
+ // trials. 1/3^20=3e-10, so that should be good enough.
+ final int ITERATIONS_TO_USE = 20;
Review comment:
variable name fetches checkstyle warning
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4081/5/artifact/out/results-checkstyle-root.txt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 744607)
Time Spent: 4.5h (was: 4h 20m)
> RBF: Namenode need to choose block location for the client
> ----------------------------------------------------------
>
> Key: HDFS-13248
> URL: https://issues.apache.org/jira/browse/HDFS-13248
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Wu Weiwei
> Assignee: Íñigo Goiri
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13248.000.patch, HDFS-13248.001.patch,
> HDFS-13248.002.patch, HDFS-13248.003.patch, HDFS-13248.004.patch,
> HDFS-13248.005.patch, HDFS-Router-Data-Locality.odt, RBF Data Locality
> Design.pdf, clientMachine-call-path.jpeg, debug-info-1.jpeg, debug-info-2.jpeg
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> When execute a put operation via router, the NameNode will choose block
> location for the router, not for the real client. This will affect the file's
> locality.
> I think on both NameNode and Router, we should add a new addBlock method, or
> add a parameter for the current addBlock method, to pass the real client
> information.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
