[
https://issues.apache.org/jira/browse/HDFS-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13790652#comment-13790652
]
Sanjay Radia commented on HDFS-5324:
------------------------------------
I am copying my comment from hdfs-dev:
HDFS pluggability (and relation to pluggability added as part of Federation)
* Pluggabilty and federation are orthogonal, although we did improved the
pluggabily of HDFS as part of federation implementation. The *block layer* was
separated out as part of the federation work and hence makes the general
development of new of HDFS namespace implementations easier. Federation's
pluggablity was targeted towards someone writing a new NN and reusing the
block storage layer via a library and *optionally* living side-by-side with
different implementations of the NN *within the same cluster*. Hence we added
notion of block pools and separated out the block management layer.
* So your proposed work is clearly not in conflict with Federation or even with
the pluggability that Federation added, but philosophically, your proposal is
complementary.
Considerations: A Public API?
The FileSystem/AbstractFileSystem APIs and the newly proposed
AbstractFSNamesystem are targeting very different kinds of plugability into
Hadoop. The former takes a thin application API (FileSystem and FileContext)
and makes it easy for users to plug in different filesytems (S3, LocalFS, etc)
as Hadoop compatible filesystems. In contrast the later (the proposed
AbstractFSNamesystem) is a fatter interface inside the depths of HDFS
implementation and makes parts of the impl pluggable.
I would not make your proposed AbstractFSNamesystem a public stable Hadoop API
but instead direct it towards to HDFS developers who want to extend the
implementation of HDFS more easily. Were you envisioning the
AbstractFSNamesystem to be a stable public Hadoop API? If someone has their own
private implementation for this new abstract class, would the HDFS community
have the freedom to modify the abstract class in incompatible ways?
> Make Namespace implementation pluggable in the namenode
> -------------------------------------------------------
>
> Key: HDFS-5324
> URL: https://issues.apache.org/jira/browse/HDFS-5324
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.1.1-beta
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Milind Bhandarkar
> Fix For: 3.0.0
>
>
> For the last couple of months, we have been working on making Namespace
> implementation in the namenode pluggable. We have demonstrated that it can
> be done without major surgery on the namenode, and does not have noticeable
> performance impact. We would like to contribute it back to Apache HDFS.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
