[
https://issues.apache.org/jira/browse/HDFS-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth reassigned HDFS-4702:
-----------------------------------
Assignee: Chris Nauroth
After some further research, my current theory is that the namesystem lock
dates all the way back to HADOOP-1762. Before that patch, the namenode would
persist allocated storage IDs from registered datanodes. After that patch, we
switched to a technique of using random storage IDs that no longer required
persistence at the namenode. Later, {{OP_DATANODE_ADD}} and
{{OP_DATANODE_REMOVE}} were marked deprecated in HDFS-3137.
When datanode registrations were persisted at the namenode, it would have been
necessary for {{DatanodeManager#fetchDatanodes}} to hold a namesystem read lock
to ensure a consistent read while datanode adds and removals were happening
concurrently. Now that the namenode doesn't persist datanode registrations, I
believe the namesystem read lock is unnecessary.
{{DatanodeManager#fetchDatanodes}} calls
{{DatanodeManager#getDatanodeListForReport}}, which is already synchronized on
the internal map of datanodes. I suspect this is sufficient locking, and we
can remove the namesystem lock.
Assigning to myself for patch and test work.
> remove namesystem lock from DatanodeManager#fetchDatanodes
> ----------------------------------------------------------
>
> Key: HDFS-4702
> URL: https://issues.apache.org/jira/browse/HDFS-4702
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
>
> {{DatanodeManager#fetchDatanodes}} currently holds the namesystem read lock
> while iterating through data nodes. This method is called from the namenode
> web UI. HDFS-3990 reported a performance problem in this code path. This is
> a follow-up jira to investigate whether or not we can remove the lock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
