[
https://issues.apache.org/jira/browse/HDFS-6074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923744#comment-13923744
]
Steve Loughran commented on HDFS-6074:
--------------------------------------
I think we do that already:
[[https://github.com/apache/hadoop-common/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/UnderReplicatedBlocks.java#L141]]
{code}
private int getPriority(Block block,
int curReplicas,
int decommissionedReplicas,
int expectedReplicas) {
assert curReplicas >= 0 : "Negative replicas!";
if (curReplicas >= expectedReplicas) {
// Block has enough copies, but not enough racks
return QUEUE_REPLICAS_BADLY_DISTRIBUTED;
} else if (curReplicas == 0) {
// If there are zero non-decommissioned replicas but there are
// some decommissioned replicas, then assign them highest priority
if (decommissionedReplicas > 0) {
return QUEUE_HIGHEST_PRIORITY;
}
//all we have are corrupt blocks
return QUEUE_WITH_CORRUPT_BLOCKS;
} else if (curReplicas == 1) {
//only on replica -risk of loss
// highest priority
return QUEUE_HIGHEST_PRIORITY;
} else if ((curReplicas * 3) < expectedReplicas) {
//there is less than a third as many blocks as requested;
//this is considered very under-replicated
return QUEUE_VERY_UNDER_REPLICATED;
} else {
//add to the normal queue for under replicated blocks
return QUEUE_UNDER_REPLICATED;
}
}
{code}
though HDFS-4366 (in trunk) fixed a prioritisation issue. see also HDFS-2472
for a proposal to prioritise blocks all on a single rack by differentiating
under-replicated-one-rack from under-replicated-multiple-racks.
> Under replicated blocks with ONE replica should get replication priority over
> blocks with more than one replica.
> ----------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-6074
> URL: https://issues.apache.org/jira/browse/HDFS-6074
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.2.0
> Reporter: Mike George
> Priority: Minor
>
> We had two nodes fail at the same time. There were over 2000 blocks at
> higher risk of causing corrupt files since they were single replica. There
> were over 40000 under replicated blocks with most having two replicas and the
> namenode priority to recreate missing replicas clearly placed no priority of
> single replica blocks.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
