[
https://issues.apache.org/jira/browse/HDFS-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13595589#comment-13595589
]
Amareshwari Sriramadasu commented on HDFS-4562:
-----------------------------------------------
Here is the log for one of the block :
{noformat}
2013-03-06 00:02:21,787 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.allocateBlock: /user/mapred/system/job_201303040902_85451/jobToken.
blk_6341538266279390032_231558736
2013-03-06 00:02:21,794 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: x.y.z.63:50010 is added to
blk_6341538266279390032_231558736 size 226
2013-03-06 00:02:21,794 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: x.y.z.73:50010 is added to
blk_6341538266279390032_231558736 size 226
2013-03-06 00:02:21,803 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* ask
x.y.z.63:50010 to replicate blk_6341538266279390032_231558736 to datanode(s)
x.y.z.74:50010
2013-03-06 00:02:21,803 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: x.y.z.67:50010 is added to
blk_6341538266279390032_231558736 size 226
2013-03-06 00:02:23,784 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: x.y.z.74:50010 is added to
blk_6341538266279390032_231558736 size 226
2013-03-06 00:02:23,784 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.chooseExcessReplicates: (192.168.176.74:50010,
blk_6341538266279390032_231558736) is added to recentInvalidateSets
2013-03-06 00:02:30,805 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* ask
x.y.z.73:50010 to delete blk_2922851987421619867_231558569
blk_6341538266279390032_231558736 blk_7283921456384156061_231558575
{noformat}
For almost every that is created in HDFS, we are seeing excess replica being
created. Here is some analysis that I have done :
Looking at the following code in FSNameSystem.computeReplicationWorkForBlock :
{noformat}
numEffectiveReplicas = numReplicas.liveReplicas() +
pendingReplications.getNumReplicas(block);
if(numEffectiveReplicas >= requiredReplication) {
neededReplications.remove(block, priority); // remove from
neededReplications
replIndex--;
NameNode.stateChangeLog.info("BLOCK* "
+ "Removing block " + block
+ " from neededReplications as it has enough replicas.");
return false;
}
{noformat}
There should not be an excess replica getting created. From the above log, if
the replica corresponding to x.y.z.67 has been in pending replicas.
I see that pendingReplications does not get updated during the creation of the
file. Correct me if I'm wrong.
I'm not sure if the above analysis is valid. Please correct me if I'm wrong.
> Many excess replicas getting created
> ------------------------------------
>
> Key: HDFS-4562
> URL: https://issues.apache.org/jira/browse/HDFS-4562
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 1.1.1
> Reporter: Amareshwari Sriramadasu
> Fix For: 1.2.0
>
>
> We are seeing too many excess replicas getting created in our cluster. The
> number excess replicas in day coming out to be more than 1 lakh.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
