[
https://issues.apache.org/jira/browse/HDFS-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564741#comment-15564741
]
Zhe Zhang edited comment on HDFS-10967 at 10/11/16 9:10 AM:
------------------------------------------------------------
Just realized HDFS-8131 and HDFS-8041 address this issue with different
approaches.
Since HDFS-8131 is only in 2.8.0 (not released) and 3.0.0.alpha1, I think we
can still revisit the issue (which involves configurable class and another
config knob). Pinging [~andrew.wang] to make sure.
[~liushaohui] Nice work under HDFS-8131! I think we can consider merging the
{{AvailableSpaceBlockPlacementPolicy}} algorithm back to
{{BlockPlacementPolicyDefault}}? As [~mingma] mentioned above, this will allow
other BPPs to benefit from the logic. BTW if you have used
{{AvailableSpaceBlockPlacementPolicy}} in production, any insights will be much
appreciated.
It's interesting that the 3 JIRAs use related but different config knobs.
# HDFS-8131: control how *aggressively* we should prefer high-storage nodes
# HDFS-8041: set a *threshold* of fullness; nodes above the threshold are not
considered
# HDFS-10967 v3 patch: set a *threshold* of fullness; nodes above the threshold
are given lower chance. With 3 random retries, this is more aggressive than the
HDFS-8131 algorithm. If number of retries is set to 1, it is roughly as
aggressive as HDFS-8131
I think even when the cluster is severely imbalanced, by configuring
{{dfs.namenode.available-space-block-placement-policy.balanced-space-preference-fraction}}
to be {{1.0}}, we can pretty effectively place data out of the near-full
nodes. Assuming half of DNs have small storage space and the other half have
big storage space, a {{1.0}} config would lead to {{25%}} chance for a 2nd-3rd
replica to land on small-storage node. Thoughts?
was (Author: zhz):
Just realized HDFS-8131 and HDFS-8041 address this issue with different
approaches.
> Add configuration for BlockPlacementPolicy to avoid near-full DataNodes
> -----------------------------------------------------------------------
>
> Key: HDFS-10967
> URL: https://issues.apache.org/jira/browse/HDFS-10967
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Labels: balancer
> Attachments: HDFS-10967.00.patch, HDFS-10967.01.patch,
> HDFS-10967.02.patch, HDFS-10967.03.patch
>
>
> Large production clusters are likely to have heterogeneous nodes in terms of
> storage capacity, memory, and CPU cores. It is not always possible to
> proportionally ingest data into DataNodes based on their remaining storage
> capacity. Therefore it's possible for a subset of DataNodes to be much closer
> to full capacity than the rest.
> This heterogeneity is most likely rack-by-rack -- i.e. _m_ whole racks of
> low-storage nodes and _n_ whole racks of high-storage nodes. So It'd be very
> useful if we can lower the chance for those near-full DataNodes to become
> destinations for the 2nd and 3rd replicas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
