[
https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584703#comment-13584703
]
Alejandro Abdelnur commented on YARN-392:
-----------------------------------------
I'd like to restate he problem.
Making things a bit more high level, the end goal is for an AM to give certain
hints to the RM scheduler on how it plans to use requested resources.
Hints are just that, 'hints'. They may not be taken into consideration, AMs
must not rely on hints to be able to work properly. It is fine if the RM
scheduler ignores hints completely (because it is too busy or because it does
not understand them). An RM scheduler that understands a hint may use it to
make more optimal allocation decisions and may give AMs a speed boost.
Another thing to keep in mind is that hints won't complicate the RM logic as
the RM only involvement is passing them to the scheduler.
Examples of hints are: gang scheduling, desired locality, desired
multi-locality, resources fulfillment timeout, future resource allocation.
I can understand the worries about going task centric, but I think the hints
approach is a bit different. Being able to specify hints will enable scheduling
features experimentation without requiring protocol changes. Eventually, if we
find that a hint is a good feature to support at scheduler API level we may
eventually add it to the protocol/API.
The changes in the protocol/API would be as simple as having and extra String
field in resource requests and resources allocations to indicate hints (on
requests) and receive the hints taken into consideration (on allocations).
Thoughts?
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>
> Key: YARN-392
> URL: https://issues.apache.org/jira/browse/YARN-392
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Sandy Ryza
> Attachments: YARN-392.patch
>
>
> Currently its not possible to specify scheduling requests for specific nodes
> and nowhere else. The RM automatically relaxes locality to rack and * and
> assigns non-specified machines to the app.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
