[
https://issues.apache.org/jira/browse/HDFS-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870344#comment-16870344
]
Ayush Saxena commented on HDFS-14578:
-------------------------------------
Thanx [~jojochuang] for putting this up.
I guess that is the default behaviour of {{AvailableBlockPlacementPolicy}} that
it intends to maintain data Locality, and optimize the selection thereafter.
If the client isn't bothered with data locality, he can use use avoidLocal and
get its intended result.
>From HDFS-8131, Yongjun mentionioned :
bq. 3. Wonder if we could have another placement policy that could potentially
have a choice to choose a different DN than local DN for the first node, so we
don't always choose the local DN as the first node.
He I guess was proposing to have a new BPP which does this, not that it is a
problem with the current one.
Well Out of curiosity, Can you help what exactly is the problem customer
facing, What is the use case.
I checked the code, I think, if we require we may implement our own
{{chooseLocalStorage(..)}} and get the intended behavior for the first node
too, Like choosing b/w local node and local rack node or something like that,
which fits the use case.
> AvailableSpaceBlockPlacementPolicy always prefers local node
> ------------------------------------------------------------
>
> Key: HDFS-14578
> URL: https://issues.apache.org/jira/browse/HDFS-14578
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: block placement
> Affects Versions: 2.8.0, 2.7.4, 3.0.0-alpha1
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> It looks like AvailableSpaceBlockPlacementPolicy prefers local disk just like
> in the BlockPlacementPolicyDefault
>
> As Yongjun mentioned in
> [HDFS-8131|https://issues.apache.org/jira/browse/HDFS-8131?focusedCommentId=16558739&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16558739],
>
> {quote}Class AvailableSpaceBlockPlacementPolicy extends
> BlockPlacementPolicyDefault. But it doesn't change the behavior of choosing
> the first node in BlockPlacementPolicyDefault, so even with this new feature,
> the local DN is always chosen as the first DN (of course when it is not
> excluded), and the new feature only changes the selection of the rest of the
> two DNs.
> {quote}
> I'm file this Jira as I groom Cloudera's internal Jira and found this
> unreported issue. We do have a customer hitting this problem. I don't have a
> fix, but thought it would be beneficial to report it to Apache Jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
