[
https://issues.apache.org/jira/browse/YARN-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511503#comment-16511503
]
Yufei Gu commented on YARN-8394:
--------------------------------
Hi [~cheersyang],
Let me clarify a little bit. The code logic should be:
{code:java}
if "yarn.scheduler.capacity.node-locality-delay" is -1:
disable "yarn.scheduler.capacity.rack-locality-additional-delay"
{code}
So that, a user doesn't need to set it manually, which is suggested by the doc
you added. Moreover, if the code logic had been there, we would just say that
if you disable yarn.scheduler.capacity.node-locality-delay, you disable
yarn.scheduler.capacity.rack-locality-additional-delay as well.
{quote}
Note, this feature should be disabled if YARN is deployed separately with the
file system, as locality is meaningless. This can be done by setting
`yarn.scheduler.capacity.node-locality-delay` to `-1`, in this case, request's
locality constraint is ignored.
{quote}
> Improve data locality documentation for Capacity Scheduler
> ----------------------------------------------------------
>
> Key: YARN-8394
> URL: https://issues.apache.org/jira/browse/YARN-8394
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
> Fix For: 3.2.0, 3.1.1, 3.0.4
>
> Attachments: YARN-8394.001.patch, YARN-8394.002.patch
>
>
> YARN-6344 introduces a new parameter
> {{yarn.scheduler.capacity.rack-locality-additional-delay}} in
> capacity-scheduler.xml, we need to add some documentation in
> {{CapacityScheduler.md}} accordingly.
> Moreover, we are seeing more and more clusters are separating storage and
> computation where file system is always remote, in such cases we need to
> introduce how to compromise data locality in CS otherwise MR jobs are
> suffering.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
