[
https://issues.apache.org/jira/browse/HDFS-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236420#comment-14236420
]
Ming Ma commented on HDFS-7396:
-------------------------------
It might be nice if we can enforce the correctness via unit tests. For example,
https://issues.apache.org/jira/browse/HDFS-7182 tries to assert any JMX metrics
shouldn't require FSNamesystem read lock with the modified unit test.
Federated NN is sort of orthogonal to this discussion given it is still useful
to improve the overall synchronization in NN even for federated NN. However it
does make synchronization improvement less critical; e.g., you can get the NN
JVM heap size under 32G and use compressed pointer, etc. There is still
improvement we need to make to federated NN, such as dynamic reallocation
amount namespaces, have each NN host several volumes for efficient move, etc.
It will be interesting to understand if there is any plan to scale HDFS in that
direction.
> Revisit synchronization in Namenode
> -----------------------------------
>
> Key: HDFS-7396
> URL: https://issues.apache.org/jira/browse/HDFS-7396
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
>
> HDFS-2106 separated block management to a new package from namenode. As part
> of it, some code was refactored to new classes such as DatanodeManager,
> HeartbeatManager, etc. There are opportunities for improve locking in
> namenode while currently the synchronization in namenode is mainly done by a
> single global FSNamesystem lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
