[
https://issues.apache.org/jira/browse/HDFS-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744506#comment-14744506
]
Colin Patrick McCabe commented on HDFS-9055:
--------------------------------------------
A big part of the goal of webhdfs is to support distCp between clusters on
different major versions of Hadoop. We never guaranteed that with our RPC
protocol (no, not even RPCv9), so we need a protocol that does guarantee it.
Bumping the major version of webhdfs does nothing to advance that goal, and in
fact it makes it harder by creating a different, incompatible "flavor" of
webhdfs. We retired the trusty old HFTP protocol because webhdfs was the
do-all, be-all solution for cross-version compatibility. I think it's not
unfair to ask webhdfs to actually provide that compatibility!
Some of the JIRAs here seem to be moving processing from the client to the
NameNode. For example, HDFS-9058 proposes that we implement "find" on the
NameNode and send the results back to the client. HDFS-9051 proposes that the
NameNode support recursive listing of directories (probably similar to
getContent etc.). I am concerned that this will lead to extra complexity on
the NameNode, and-- at least if implemented in the obvious way-- longer
latencies since the FSN lock will be held for longer for the "do it all"
operation. I am not convinced that the benefits outweigh the extra complexity
and potential scalability issues.
As [~cnauroth] commented, all of the JIRAs up here besides HDFS-7822 are just
new features (like adding snapshot or truncate support) that can easily be done
without bumping the major version of libwebhdfs. And even for that JIRA,
[~kihwal]'s original proposal for HDFS-7822 was a compatible one.
Unless there is something I am missing, we should simply add the features that
make sense to WebHDFS v1 rather than creating an incompatible version.
> WebHDFS REST v2
> ---------------
>
> Key: HDFS-9055
> URL: https://issues.apache.org/jira/browse/HDFS-9055
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: webhdfs
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
>
> There's starting to be enough changes to fix and add missing functionality to
> webhdfs that we should probably update to REST v2. This also gives us an
> opportunity to deal with some incompatible issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
