[
https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15725889#comment-15725889
]
Weiwei Yang commented on HDFS-11156:
------------------------------------
Uploaded v7 patch, it includes
# V6 patch in this ticket
# The doc patch in HDFS-11166
# Added a fall back mechanism to handle the backwards compatibility.
I set up a 2.7.2 cluster, and tested to use trunk client library to access
webhdfs on the 2.7.2 cluster, without the #3 fix, it failed with following
exception
{noformat}
[yangww@host1 hadoop-3.0.0-alpha2-SNAPSHOT]$ ./bin/hadoop jar
/tmp/WebhdfsClient.jar WebhdfsClient
16/12/06 06:43:24 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
org.apache.hadoop.ipc.RemoteException(java.lang.IllegalArgumentException):
Invalid value for webhdfs parameter "op": No enum constant
org.apache.hadoop.hdfs.web.resources.GetOpParam.Op.GETFILEBLOCKLOCATIONS
at
org.apache.hadoop.hdfs.web.JsonUtilClient.toRemoteException(JsonUtilClient.java:73)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:474)
{noformat}
with the patch, it falls back to use GET_BLOCK_LOCATIONS, output looks like
{noformat}
[yangww@host1 hadoop-3.0.0-alpha2-SNAPSHOT]$ ./bin/hadoop jar
/tmp/WebhdfsClient.jar WebhdfsClient
16/12/06 07:36:24 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
16/12/06 07:36:25 INFO web.WebHdfsFileSystem: Invalid webhdfs operation
parameter GETFILEBLOCKLOCATIONS.Fallback to use GET_BLOCK_LOCATIONS
... block locations ...
{noformat}
> Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API
> ----------------------------------------------------
>
> Key: HDFS-11156
> URL: https://issues.apache.org/jira/browse/HDFS-11156
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
> Affects Versions: 2.7.3
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch,
> HDFS-11156.03.patch, HDFS-11156.04.patch, HDFS-11156.05.patch,
> HDFS-11156.06.patch, HDFS-11156.07.patch
>
>
> Following webhdfs REST API
> {code}
> http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GET_BLOCK_LOCATIONS&offset=0&length=1
> {code}
> will get a response like
> {code}
> {
> "LocatedBlocks" : {
> "fileLength" : 1073741824,
> "isLastBlockComplete" : true,
> "isUnderConstruction" : false,
> "lastLocatedBlock" : { ... },
> "locatedBlocks" : [ {...} ]
> }
> }
> {code}
> This represents for *o.a.h.h.p.LocatedBlocks*. However according to
> *FileSystem* API,
> {code}
> public BlockLocation[] getFileBlockLocations(Path p, long start, long len)
> {code}
> clients would expect an array of BlockLocation. This mismatch should be
> fixed. Marked as Incompatible change as this will change the output of the
> GET_BLOCK_LOCATIONS API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
