[
https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156865#comment-13156865
]
Uma Maheswara Rao G commented on HDFS-1765:
-------------------------------------------
One scenario here:
Accidentally user configured single rack so, all the block goes to under
replicated.
after some time, high priority blocks got added to queues.
While processing it is trying to process only previos underreplicated blocks,
even though there are high priority blocks added.
Looking at this part, single replecatioIndex maintained for all the priority
queues. For suppose if it needs to process 100blocks in this iteration, it may
not be able to find the targets because there is only single rack available.
so, the replecationIndex will not be decremented. In Next iteration it will try
to skip the number of replecationIndex blocks. Lets say 50 blocks added to high
priority queue now. ReplIndex was already 100. So, it will skip first 100
blcoks and start picking the remaining. Due to this, high peiority blocks are
not getting chance to replicate quickly.
> Block Replication should respect under-replication block priority
> -----------------------------------------------------------------
>
> Key: HDFS-1765
> URL: https://issues.apache.org/jira/browse/HDFS-1765
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Affects Versions: 0.23.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.24.0
>
> Attachments: underReplicatedQueue.pdf
>
>
> Currently under-replicated blocks are assigned different priorities depending
> on how many replicas a block has. However the replication monitor works on
> blocks in a round-robin fashion. So the newly added high priority blocks
> won't get replicated until all low-priority blocks are done. One example is
> that on decommissioning datanode WebUI we often observe that "blocks with
> only decommissioning replicas" do not get scheduled to replicate before other
> blocks, so risking data availability if the node is shutdown for repair
> before decommission completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
