[
https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870278#action_12870278
]
Todd Lipcon commented on HDFS-1172:
-----------------------------------
Particular sequence of events:
# client finishes writing to block with 3 replicas
# first DN happens to heartbeat, so addStoredBlock is called in the NN
# client calls completeFile, which calls checkReplicationFactor(newFile) when
finalizing the INode
# NN adds block to pending replication
# Replication monitor runs and schedules two replications
# second and third pipeline DNs send their addStoredBlock notifications with
their heartbeats
# Replications finish, and the new replicas report the new blocks as well
# NN notices the excess replicas and schedules deletion
This doesn't cause major issues, but we do end up wasting a fair amount of disk
and network resources.
The question is why sometimes the immediate heartbeat on blockReceived doesn't
trigger as it's supposed to.
> Blocks in newly completed files are considered under-replicated too quickly
> ---------------------------------------------------------------------------
>
> Key: HDFS-1172
> URL: https://issues.apache.org/jira/browse/HDFS-1172
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.21.0
> Reporter: Todd Lipcon
>
> I've seen this for a long time, and imagine it's a known issue, but couldn't
> find an existing JIRA. It often happens that we see the NN schedule
> replication on the last block of files very quickly after they're completed,
> before the other DNs in the pipeline have a chance to report the new block.
> This results in a lot of extra replication work on the cluster, as we
> replicate the block and then end up with multiple excess replicas which are
> very quickly deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
