Ming Ma created HDFS-9315:
-----------------------------
Summary: Update excess storage type list properly when del hint is
picked
Key: HDFS-9315
URL: https://issues.apache.org/jira/browse/HDFS-9315
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Ming Ma
HDFS-8647 makes it easier to reason about various block placement scenarios.
Here is one potential issue where {{excessTypes}} isn't updated when
{{delNodeHint}} is picked. When {{delNodeHint}} isn't picked, the excess
storage identified will be removed from excessTypes.
{noformat}
if (useDelHint(firstOne, delNodeHintStorage, addedNodeStorage,
moreThanOne, tmpExcessTypes)) {
cur = delNodeHintStorage;
} else { // regular excessive replica removal
cur = chooseReplicaToDelete((short) expectedNumOfReplicas, moreThanOne,
exactlyOne, tmpExcessTypes);
}
chooseReplicaToDelete(...) {
...
excessTypes.remove(storage.getStorageType());
}
{noformat}
It isn't clear how this can happen in real world; maybe HDFS-9314. Usually when
del hint is used, the delta between expected num replica and the actual replica
is one and thus this shouldn't cause any issue.
Still it is better to make it consistent. Each time an excess replica is
picked, excessTypes should be updated regardless whether it comes from del hint
or not.
Note this issue was there prior to HDFS-8647.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
