[
https://issues.apache.org/jira/browse/YARN-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403185#comment-16403185
]
Konstantinos Karanasos edited comment on YARN-8034 at 3/17/18 1:22 AM:
-----------------------------------------------------------------------
Hi [~jagadish1...@gmail.com],
As [~jlowe] mentioned, this is very related to YARN-6344 for the capacity
scheduler. What you should look at is the
"yarn.scheduler.capacity.rack-locality-additional-delay" parameter.
Since you have only one (or very few) container requests, the current logic (if
you let the above parameter to its default value) will lead to relaxing
locality almost immediately. If you set that parameter to a positive value, you
should achieve your desired behavior.
was (Author: kkaranasos):
Hi [~jagadish1...@gmail.com],
As [~jlowe] mentioned, this is very related to YARN-6344 for the capacity
scheduler. What you should look at is the
"yarn.scheduler.capacity.rack-locality-additional-delay" parameter.
Since you have only one (or very few) container requests, the current logic (if
you let the above parameter to its default value) value will lead to relaxing
locality almost immediately. If you set that parameter to a positive value, you
should achieve your desired behavior.
> Clarification on preferredHost request with relaxedLocality
> -----------------------------------------------------------
>
> Key: YARN-8034
> URL: https://issues.apache.org/jira/browse/YARN-8034
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Jagadish
> Priority: Major
>
> I work on Apache Samza, a stateful stream-processing framework that leverages
> Yarn for resource management. The Samza AM requests resources on specific
> hosts to schedule stateful jobs. We set relaxLocality = true in these
> requests we make to Yarn. Often we have observed that we don't get containers
> on the hosts that we requested them on and the Yarn RM returns containers on
> arbitrary hosts.
> Do you know what the behavior of the FairScheduler/CapacityScheduler is when
> setting "relaxLocality = true".I did play around by setting a high value for
> yarn.scheduler.capacity.node-locality-delay but it did not seem to matter.
> However, when setting relaxLocality = false, we get resources on the exact
> hosts we requested on.
> The behavior I want from Yarn is "Honor locality to the best possible extent
> and only return a container on an arbitrary host if the requested host is
> down". Is there a way to accomplish this?
> If you can point me to the Scheduler code, I'm happy to look at it as well.
> For context, we have continuous scheduling enabled in our clusters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
