[
https://issues.apache.org/jira/browse/HDFS-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13794517#comment-13794517
]
Colin Patrick McCabe commented on HDFS-5324:
--------------------------------------------
As you know, we currently have a reader/writer lock which protects almost
everything in {{FSNamesystem}}. This lock is used in a lot of places, such as
in the {{BlockManager}}, not just inside the {{FSNamesystem}} class itself.
Having a "big lock" such as this is fundamentally incompatible with having a
paged namespace. The problem is, if you take the lock, and then do something
that requires you to go out to disk, other threads could hang for seconds or
minutes at a time. Disk I/O is unpredictable and often takes a while.
Currently, we never have to worry about this, since everything in the
{{FSNamesystem}} is held in memory, and memory can be accessed relatively
quickly.
If you can come up with patches to implement fine-grained locking, then we
could start thinking about code sharing. But until that point, I don't see how
it's possible to share much code here.
bq. As I have replied to Sanjay, I am not proposing a stable API within HDFS
for implementing Namespaces
If that's true, then the API should be private to HDFS, and your alternate FSN
implementation should be in the hadoop-hdfs project, right?
> Make Namespace implementation pluggable in the namenode
> -------------------------------------------------------
>
> Key: HDFS-5324
> URL: https://issues.apache.org/jira/browse/HDFS-5324
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.1.1-beta
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Milind Bhandarkar
> Fix For: 3.0.0
>
> Attachments: AbstractNamesystem.java
>
>
> For the last couple of months, we have been working on making Namespace
> implementation in the namenode pluggable. We have demonstrated that it can
> be done without major surgery on the namenode, and does not have noticeable
> performance impact. We would like to contribute it back to Apache HDFS.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
