[
https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622544#comment-13622544
]
Bikas Saha commented on YARN-392:
---------------------------------
I dont think its possible for location specific and non-location specific
requests to live at the same priority. This is mainly because of the way
current schedulers are implemented in the RM (grouped together and keyed by
location and priority). Such requests have to be separated by priority and that
may not be a bad thing IMO.
I discussed this offline with [~vinodkv] and I would like to suggest and
extension to the approach. Instead of a flag, how about specifying a time
interval that tells the RM how long to wait before dropping locality. A time
interval of infinite would be the same as a boolean flag and so this approach
covers the other one. Additionally, it lets a large map to be more conservative
about dropping locality over latency and a short job more aggressive. Currently
the value of this interval comes from config and maps to the number of
scheduling attempts missed by this request. This is done by keeping a count of
node heartbeats. Given the number of nodes and heartbeat interval, the user
specified time interval can easily be mapped to a count that matches the
current implementation. So this will not be a perf hit nor a change in logic
compared to existing code.
Another thing to consider is allowing users to say I want to be scheduled only
on these racks. Again, I dont think we can mix node-specfic and rack-specific
scheduling at the same priority.
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>
> Key: YARN-392
> URL: https://issues.apache.org/jira/browse/YARN-392
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Sandy Ryza
> Attachments: YARN-392-1.patch, YARN-392.patch
>
>
> Currently its not possible to specify scheduling requests for specific nodes
> and nowhere else. The RM automatically relaxes locality to rack and * and
> assigns non-specified machines to the app.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
