[
https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012152#comment-15012152
]
Ming Ma commented on HDFS-9314:
-------------------------------
Thanks @Xiao Chen for working on this. I agree with [~zhz] and [~walter.k.su],
this could end up violating block placement policy, as the fallback
{{chooseReplicaToDelete}} call uses the remaining replicas which aren't honored
by the block placement policy.
* Another way to address this, maybe we can modify the semantics of
{{BlockPlacementPolicyDefault#pickupReplicaSet}} to include "all replicas that
deletion of the replica won't violate the policy". So using the example listed
in the description, it will return all 4 replicas instead given deletion of any
one of them is ok given the number of racks >=2 afterwards. This does change
the existing semantics of "making sure the number of racks remains unchanged
after deletion." But it should be ok given {{BlockPlacementPolicyDefault}} only
guarantees "the number of racks >= 2" policy. With this approach, there is no
need for the fallback call.
* We still have the case like what Water mentioned above where it just can't
pick any replica to satisfy the block placement policy. IMHO, we can follow up
in a separate jira as that might require more changes to schedule replication
etc. This jira can just cover the case single pass can solve and make sure it
doesn't suggest replica that will violate block placement policy.
> Improve BlockPlacementPolicyDefault's picking of excess replicas
> ----------------------------------------------------------------
>
> Key: HDFS-9314
> URL: https://issues.apache.org/jira/browse/HDFS-9314
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Ming Ma
> Assignee: Xiao Chen
> Attachments: HDFS-9314.001.patch, HDFS-9314.002.patch
>
>
> The test case used in HDFS-9313 identified NullPointerException as well as
> the limitation of excess replica picking. If the current replicas are on
> {SSD(rack r1), DISK(rack 2), DISK(rack 3), DISK(rack 3)} and the storage
> policy changes to HOT_STORAGE_POLICY_ID, BlockPlacementPolicyDefault's won't
> be able to delete SSD replica.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
