[
https://issues.apache.org/jira/browse/HDFS-12459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244343#comment-16244343
]
Rushabh S Shah commented on HDFS-12459:
---------------------------------------
bq. I don't think so. WebHDFS.md is the doc for webhdfs, not for
WebHdfsFileSystem.
I see your point. I think its okay to go with the approach in patch.
I have couple of comments regarding v5 of the patch.
1. +NamenodeWebHdfsMethods.java+
{noformat}
final String js = JsonUtil.toJsonString("BlockLocations",
JsonUtil.toJsonMap(locations));
{noformat}
{{JsonUtil}} should have method {{toJsonString#toJsonString(BlockLocations[])}}
just to be consistent with other methods. Refer to
{{JsonUtil.toJsonString(BlockStoragePolicy[] storagePolicies)}}
Some of the recently added methods passes the key along with map.
Apologies for not pointing out in the previous review.
2. +TestWebHDFS.java+
{noformat}
public void testWebHdfsGetBlockLocationsWithStorageType() throws Exception{
MiniDFSCluster cluster = null;
final Configuration conf = WebHdfsTestUtil.createConf();
+ final int offset = 42;
+ final int length = 512;
+ final Path path = new Path("/foo");
+ byte[] contents = new byte[1024];
+ RANDOM.nextBytes(contents);
+ try {
+ cluster = new MiniDFSCluster.Builder(conf).numDataNodes(1).build();
+ final WebHdfsFileSystem fs = WebHdfsTestUtil.getWebHdfsFileSystem(conf,
+ WebHdfsConstants.WEBHDFS_SCHEME);
+ try (OutputStream os = fs.create(path)) {
+ os.write(contents);
+ }
+ BlockLocation[] locations = fs.getFileBlockLocations(path, offset,
+ length);
+ for (BlockLocation location: locations) {
+ StorageType[] storageTypes = location.getStorageTypes();
+ Assert.assertTrue(storageTypes != null && storageTypes.length > 0 &&
+ storageTypes[0] == StorageType.DISK);
+ }
+ } finally {
+ if (cluster != null) {
+ cluster.shutdown();
+ }
+ }
+ }
{noformat}
>From the diff, it looks like you changed
>{{testWebHdfsGetBlockLocationsWithStorageType}} method which is not correct.
When I applied the change and did git blame, it shows the same behavior.
Can you please fix that.
> Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API
> ----------------------------------------------------------------
>
> Key: HDFS-12459
> URL: https://issues.apache.org/jira/browse/HDFS-12459
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-12459.001.patch, HDFS-12459.002.patch,
> HDFS-12459.003.patch, HDFS-12459.004.patch, HDFS-12459.005.patch
>
>
> HDFS-11156 was reverted because the implementation was non optimal, based on
> the suggestion from [~shahrs87], we should avoid creating a dfs client to get
> block locations because that create extra RPC call. Instead we should use
> {{NamenodeProtocols#getBlockLocations}} then covert {{LocatedBlocks}} to
> {{BlockLocation[]}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
