[
https://issues.apache.org/jira/browse/HDFS-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated HDFS-9315:
--------------------------
Attachment: HDFS-9315.patch
Here is the patch that explains the scenario and the fix. Update
{{excessTypes}} regardless of whether del hint is picked.
> Update excess storage type list properly when del hint is picked
> ----------------------------------------------------------------
>
> Key: HDFS-9315
> URL: https://issues.apache.org/jira/browse/HDFS-9315
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Ming Ma
> Attachments: HDFS-9315.patch
>
>
> HDFS-8647 makes it easier to reason about various block placement scenarios.
> Here is one potential issue where {{excessTypes}} isn't updated when
> {{delNodeHint}} is picked. When {{delNodeHint}} isn't picked, the excess
> storage identified will be removed from excessTypes.
> {noformat}
> if (useDelHint(firstOne, delNodeHintStorage, addedNodeStorage,
> moreThanOne, tmpExcessTypes)) {
> cur = delNodeHintStorage;
> } else { // regular excessive replica removal
> cur = chooseReplicaToDelete((short) expectedNumOfReplicas,
> moreThanOne,
> exactlyOne, tmpExcessTypes);
> }
> chooseReplicaToDelete(...) {
> ...
> excessTypes.remove(storage.getStorageType());
> }
> {noformat}
> It isn't clear how this can happen in real world; maybe HDFS-9314. Usually
> when del hint is used, the delta between expected num replica and the actual
> replica is one and thus this shouldn't cause any issue.
> Still it is better to make it consistent. Each time an excess replica is
> picked, excessTypes should be updated regardless whether it comes from del
> hint or not.
> Note this issue was there prior to HDFS-8647.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
