[
https://issues.apache.org/jira/browse/HDFS-7787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14321144#comment-14321144
]
Ravi Prakash commented on HDFS-7787:
------------------------------------
Thanks Frode!
Could you please verify that the block was indeed replicated *from* the
decommissioning node; i.e. did you see such a message in the logs? Or are you
inferring that only from the "number of underreplicated blocks" and "blocks
with no live replicas" . e.g. I can think another of another explanation.
Lets say there are nodeX, nodeY and nodeZ and you have decommissioned nodeZ.
Let's say blockA is on nodeX and nodeZ. This would count it under "number of
underreplicated blocks". Since the replication work is calculated per-datanode,
maybe blockA was replicated *from* nodeX to (say) nodeY. Thus the count for
"number of underreplicated blocks" would go down before "blocks with no live
replicas" . In the meantime blockB, blockC etc. which were present only on
nodeZ were being replicated (but since they are only on this 1 node, their
counts will decrease slower). This is obviously assuming that nodeX doesn't
have any blocks which fall under "blocks with no live replicas" which could
have been the case.
> Wrong priorty of replication
> ----------------------------
>
> Key: HDFS-7787
> URL: https://issues.apache.org/jira/browse/HDFS-7787
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.6.0
> Environment: 2 namenodes HA, 6 datanodes in two racks
> Reporter: Frode Halvorsen
> Labels: balance, hdfs, replication-performance
>
> Each file has a setting of 3 replicas. split on different racks.
> After a simulated crash of one rack (shutdown of all nodes, deleted
> data-directory an started nodes) and decommssion of one of the nodes in the
> orther rack the replication does not follow 'normal' rules...
> My cluster has appx 25 mill files, and the one node I now try to decommision
> has 9 millions underreplicated blocks, and 3,5 million blocks with 'no live
> replicas'. After a restart of the node, it starts to replicate both types of
> blocks, but after a while, it only repliates under-replicated blocks with
> other live copies. I would think that the 'normal' way to do this would be to
> make sure that all blocks this node keeps the only copy of, should be the
> first to be replicated/balanced ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
