[
https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874873#comment-15874873
]
Elek, Marton commented on HDFS-11295:
-------------------------------------
According to the suggestion of [~arpiagariu] I created a visual explanation
(see the attached image) about this modification.
This is the exact situation what the patched unit test uses:
Without the patch the node2 would be used as the _summary_ of the free spaces
is the lowest there (2 GB). But there is a storage (storage5) where the free
space is only 0.5 gb. So after the patch the storage 5 will be used: even if
the overall free space on node4 is 100.5 Gb the storage5 has only 0.5 gb so it
should be preferred to delete unnecessary blocks from there.
> Check storage remaining instead of node remaining in
> BlockPlacementPolicyDefault.chooseReplicaToDelete()
> --------------------------------------------------------------------------------------------------------
>
> Key: HDFS-11295
> URL: https://issues.apache.org/jira/browse/HDFS-11295
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 2.7.1
> Reporter: Xiao Liang
> Assignee: Elek, Marton
> Attachments: HDFS-11295.001.patch, HDFS-11295.002.patch,
> HDFS-11295.003.patch, HDFS-11295.jpg
>
>
> Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic
> for choosing replica to delete is to pick the node with the least free
> space(node.getRemaining()), if all hearbeats are within the tolerable
> heartbeat interval.
> However, a node may have multiple storages and node.getRemaining() is a sum
> of the remainings of them, if free space of the storage with the block to be
> delete is low, free space of the node could still be high due to other
> storages of the node, finally the storage chosen may not be the storage with
> least free space.
> So using storage.getRemaining() to choose a storage with least free space for
> choosing replica to delete may be a better way to balance storage usage.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
