[
https://issues.apache.org/jira/browse/HDFS-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13146301#comment-13146301
]
Hudson commented on HDFS-2495:
------------------------------
Integrated in Hadoop-Mapreduce-trunk #891 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/891/])
HDFS-2495. Increase granularity of write operations in ReplicationMonitor
thus reducing contention for write lock. Contributed by Tomasz Nykiel.
hairong :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1199024
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
> Increase granularity of write operations in ReplicationMonitor thus reducing
> contention for write lock
> ------------------------------------------------------------------------------------------------------
>
> Key: HDFS-2495
> URL: https://issues.apache.org/jira/browse/HDFS-2495
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Reporter: Tomasz Nykiel
> Assignee: Tomasz Nykiel
> Fix For: 0.24.0
>
> Attachments: replicationMon.patch, replicationMon.patch-1
>
>
> For processing blocks in ReplicationMonitor
> (BlockManager.computeReplicationWork), we first obtain a list of blocks to be
> replicated by calling chooseUnderReplicatedBlocks, and then for each block
> which was found, we call computeReplicationWorkForBlock. The latter processes
> a block in three stages, acquiring the writelock twice per call:
> 1. obtaining block related info (livenodes, srcnode, etc.) under lock
> 2. choosing target for replication
> 3. scheduling replication (under lock)
> We would like to change this behaviour and decrease contention for the write
> lock, by batching blocks and executing 1,2,3, for sets of blocks, rather than
> for each one separately. This would decrease the number of writeLock to 2,
> from 2*numberofblocks.
> Also, the info level logging can be pushed outside the writelock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
