[
https://issues.apache.org/jira/browse/HADOOP-10741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14155967#comment-14155967
]
Sanjay Radia commented on HADOOP-10741:
---------------------------------------
I see part of the counter argument being that folks using rest are doing it
for one of two reasons.
1) Protocol compatibility - this was the orignal motivation in the past when
HDFS protocols were not compatible across some versions, This has been fixed.
2) Want a lightweight client that is independent of any version of HDFS.
However as Mohammad has pointed out in his description, ustomer using web hdfs
rest protocol find that managing failure, auth, etc is painful, Hence a library
would help.
I can see Andrew's argument of putting it outside Hadoop common to better
satisfy (2). We can decide the exact mechanism to distribute this library
later.
Note the goal of this library is *not* another FS API but a client side library
that wraps hdfs's rest protocol. It is valid question to see if this API should
mimic that actual Hadoop FS API?
Mohammad please post the patch. We will figure out the mechanism of
distributing that library separately. Thanks.
> A lightweight WebHDFS client library
> ------------------------------------
>
> Key: HADOOP-10741
> URL: https://issues.apache.org/jira/browse/HADOOP-10741
> Project: Hadoop Common
> Issue Type: New Feature
> Components: tools
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Mohammad Kamrul Islam
>
> One of the motivations for creating WebHDFS is for applications connecting to
> HDFS from outside the cluster. In order to do so, users have to either
> # install Hadoop and use WebHdfsFileSsytem, or
> # develop their own client using the WebHDFS REST API.
> For #1, it is very difficult to manage and unnecessarily complicated for
> other applications since Hadoop is not a lightweight library. For #2, it is
> not easy to deal with security and handle transient errors.
> Therefore, we propose adding a lightweight WebHDFS client as a separated
> library which does not depend on Common and HDFS. The client can be packaged
> as a standalone jar. Other applications simply add the jar to their
> classpath for using it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
