[
https://issues.apache.org/jira/browse/HDFS-8154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497038#comment-14497038
]
Jakob Homan commented on HDFS-8154:
-----------------------------------
Currently WebHDFS exists as a [human-readable
specification|https://hadoop.apache.org/docs/r2.5.1/hadoop-project-dist/hadoop-hdfs/WebHDFS.html]
and a single implementation, that provided by the NameNode and its DataNodes.
We should extract WebHDFS out as a specification, which would allow easier
implementation of both WebHDFS-backed servers and clients. Additionally, this
would make it easier to generate documentation and verify correctness.
The current human-readable spec makes it possible to implement WebHDFS clients
(for example,
[Perl|http://search.cpan.org/~afaris/Apache-Hadoop-WebHDFS-0.04/lib/Apache/Hadoop/WebHDFS.pm],
[Python|https://pypi.python.org/pypi/pywebhdfs],
[.NET|https://hadoopsdk.codeplex.com/wikipage?title=WebHDFS%20Client] ,
[non-Hadoop backed JVM|https://issues.apache.org/jira/browse/HADOOP-10741],
etc.). However, each client must be built by someone parsing out that spec and
writing up their own client implementation from scratch.
There are frameworks, such as [Swagger|http://swagger.io/] and
[RAML|http://raml.org/] that allow one to define a REST interface and then
create documentation, generate client stubs and built tests against that
framework.
In addition to clients, an more programmatic WebHDFS specification would allow
other backend systems to more easily implement the WebHDFS interface. Any
Hadoop application would then be able to access that back end through the
oah.WebHDFSFileSystem or through one of the non-JVM clients described above.
This JIRA will cover specifying the WebHDFS spec in a framework like Swaggger
or RAML, switching the WebHDFS documentation to be built from this so as to be
authoritative, and verifying that the current implementation provided by the
namenode and datanodes comports with this specification.
> Extract WebHDFS protocol out as a specification to allow easier clients and
> servers
> -----------------------------------------------------------------------------------
>
> Key: HDFS-8154
> URL: https://issues.apache.org/jira/browse/HDFS-8154
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: webhdfs
> Reporter: Jakob Homan
> Assignee: Jakob Homan
>
> WebHDFS would be more useful if there were a programmatic description of its
> interface, which would allow one to more easily create servers and clients.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
