[
https://issues.apache.org/jira/browse/HDFS-12459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243251#comment-16243251
]
Weiwei Yang commented on HDFS-12459:
------------------------------------
Hi [~shahrs87]
bq. IMO, it should be GET_BLOCK_LOCATIONS and since it confirms to
fileSystem#getFileBlockLocations, the end user should not care about the
implementation. That way it will be consistent with the actual implementation
also.
I don't think so. WebHDFS.md is the doc for webhdfs, not for WebHdfsFileSystem.
In webhdfs, if user queries with GET_BLOCK_LOCATIONS parameter, this will be
handled by {{NamenodeWebHdfsMethods}}
{code}
case GET_BLOCK_LOCATIONS:
{
final long offsetValue = offset.getValue();
final Long lengthValue = length.getValue();
final LocatedBlocks locatedblocks = np.getBlockLocations(fullpath,
offsetValue, lengthValue != null? lengthValue: Long.MAX_VALUE);
final String js = JsonUtil.toJsonString(locatedblocks);
return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
}
{code}
the response will be {{LocatedBlocks}} instead of {{BlockLocation[]}}, this is
not complaint with file system API. That is also the issue this patch fixed. So
the doc in current patch seems correct to me. What you are concerning seems to
be {{WebHdfsFileSystem}}, the FS implementation over webhdfs, it internally
calls GET_BLOCK_LOCATIONS to query webhdfs and parses the output from
{{LocatedBlocks}} to {{BlockLocation[]}}, so from API level it is still
consistent.
||Component||API||API Scope||Response||
|WebHDFS|http://localhost:1234/webhdfs/v1/tmp/file?op=GETFILEBLOCKLOCATIONS|public|BlockLocation[]|
|WebHDFS|http://localhost:1234/webhdfs/v1/tmp/file?op=GET_BLOCK_LOCATIONS|private|LocatedBlocks|
|WebHdfsFileSystem|getFileBlockLocations(final FileStatus status, final long
offset, final long length)|public|BlockLocation[]|
Please let me know if this makes sense.
> Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API
> ----------------------------------------------------------------
>
> Key: HDFS-12459
> URL: https://issues.apache.org/jira/browse/HDFS-12459
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-12459.001.patch, HDFS-12459.002.patch,
> HDFS-12459.003.patch, HDFS-12459.004.patch, HDFS-12459.005.patch
>
>
> HDFS-11156 was reverted because the implementation was non optimal, based on
> the suggestion from [~shahrs87], we should avoid creating a dfs client to get
> block locations because that create extra RPC call. Instead we should use
> {{NamenodeProtocols#getBlockLocations}} then covert {{LocatedBlocks}} to
> {{BlockLocation[]}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
