[jira] [Updated] (HDFS-11295) Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete()
[ https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-11295: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 2.9.0 Target Version/s: (was: 3.0.0-alpha3) Status: Resolved (was: Patch Available) Committed to trunk and branch-2. Thanks for the contribution [~elek]. > Check storage remaining instead of node remaining in > BlockPlacementPolicyDefault.chooseReplicaToDelete() > > > Key: HDFS-11295 > URL: https://issues.apache.org/jira/browse/HDFS-11295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Xiao Liang >Assignee: Elek, Marton > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HDFS-11295.001.patch, HDFS-11295.002.patch, > HDFS-11295.003.patch, HDFS-11295.004.patch, HDFS-11295.jpg > > > Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic > for choosing replica to delete is to pick the node with the least free > space(node.getRemaining()), if all hearbeats are within the tolerable > heartbeat interval. > However, a node may have multiple storages and node.getRemaining() is a sum > of the remainings of them, if free space of the storage with the block to be > delete is low, free space of the node could still be high due to other > storages of the node, finally the storage chosen may not be the storage with > least free space. > So using storage.getRemaining() to choose a storage with least free space for > choosing replica to delete may be a better way to balance storage usage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11295) Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete()
[ https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-11295: Attachment: HDFS-11295.004.patch Thanks, the review. Improved according to the comments. > Check storage remaining instead of node remaining in > BlockPlacementPolicyDefault.chooseReplicaToDelete() > > > Key: HDFS-11295 > URL: https://issues.apache.org/jira/browse/HDFS-11295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Xiao Liang >Assignee: Elek, Marton > Attachments: HDFS-11295.001.patch, HDFS-11295.002.patch, > HDFS-11295.003.patch, HDFS-11295.004.patch, HDFS-11295.jpg > > > Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic > for choosing replica to delete is to pick the node with the least free > space(node.getRemaining()), if all hearbeats are within the tolerable > heartbeat interval. > However, a node may have multiple storages and node.getRemaining() is a sum > of the remainings of them, if free space of the storage with the block to be > delete is low, free space of the node could still be high due to other > storages of the node, finally the storage chosen may not be the storage with > least free space. > So using storage.getRemaining() to choose a storage with least free space for > choosing replica to delete may be a better way to balance storage usage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11295) Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete()
[ https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-11295: Attachment: HDFS-11295.jpg > Check storage remaining instead of node remaining in > BlockPlacementPolicyDefault.chooseReplicaToDelete() > > > Key: HDFS-11295 > URL: https://issues.apache.org/jira/browse/HDFS-11295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Xiao Liang >Assignee: Elek, Marton > Attachments: HDFS-11295.001.patch, HDFS-11295.002.patch, > HDFS-11295.003.patch, HDFS-11295.jpg > > > Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic > for choosing replica to delete is to pick the node with the least free > space(node.getRemaining()), if all hearbeats are within the tolerable > heartbeat interval. > However, a node may have multiple storages and node.getRemaining() is a sum > of the remainings of them, if free space of the storage with the block to be > delete is low, free space of the node could still be high due to other > storages of the node, finally the storage chosen may not be the storage with > least free space. > So using storage.getRemaining() to choose a storage with least free space for > choosing replica to delete may be a better way to balance storage usage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11295) Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete()
[ https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-11295: Attachment: HDFS-11295.002.patch > Check storage remaining instead of node remaining in > BlockPlacementPolicyDefault.chooseReplicaToDelete() > > > Key: HDFS-11295 > URL: https://issues.apache.org/jira/browse/HDFS-11295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Xiao Liang >Assignee: Elek, Marton > Attachments: HDFS-11295.001.patch, HDFS-11295.002.patch > > > Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic > for choosing replica to delete is to pick the node with the least free > space(node.getRemaining()), if all hearbeats are within the tolerable > heartbeat interval. > However, a node may have multiple storages and node.getRemaining() is a sum > of the remainings of them, if free space of the storage with the block to be > delete is low, free space of the node could still be high due to other > storages of the node, finally the storage chosen may not be the storage with > least free space. > So using storage.getRemaining() to choose a storage with least free space for > choosing replica to delete may be a better way to balance storage usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11295) Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete()
[ https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-11295: Attachment: HDFS-11295.001.patch > Check storage remaining instead of node remaining in > BlockPlacementPolicyDefault.chooseReplicaToDelete() > > > Key: HDFS-11295 > URL: https://issues.apache.org/jira/browse/HDFS-11295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Xiao Liang >Assignee: Elek, Marton > Attachments: HDFS-11295.001.patch > > > Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic > for choosing replica to delete is to pick the node with the least free > space(node.getRemaining()), if all hearbeats are within the tolerable > heartbeat interval. > However, a node may have multiple storages and node.getRemaining() is a sum > of the remainings of them, if free space of the storage with the block to be > delete is low, free space of the node could still be high due to other > storages of the node, finally the storage chosen may not be the storage with > least free space. > So using storage.getRemaining() to choose a storage with least free space for > choosing replica to delete may be a better way to balance storage usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11295) Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete()
[ https://issues.apache.org/jira/browse/HDFS-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-11295: Target Version/s: 3.0.0-alpha3 (was: 2.7.1) Status: Patch Available (was: Open) > Check storage remaining instead of node remaining in > BlockPlacementPolicyDefault.chooseReplicaToDelete() > > > Key: HDFS-11295 > URL: https://issues.apache.org/jira/browse/HDFS-11295 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 2.7.1 >Reporter: Xiao Liang >Assignee: Elek, Marton > > Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic > for choosing replica to delete is to pick the node with the least free > space(node.getRemaining()), if all hearbeats are within the tolerable > heartbeat interval. > However, a node may have multiple storages and node.getRemaining() is a sum > of the remainings of them, if free space of the storage with the block to be > delete is low, free space of the node could still be high due to other > storages of the node, finally the storage chosen may not be the storage with > least free space. > So using storage.getRemaining() to choose a storage with least free space for > choosing replica to delete may be a better way to balance storage usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
