[
https://issues.apache.org/jira/browse/YARN-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956325#comment-15956325
]
Wangda Tan commented on YARN-6443:
----------------------------------
[[email protected]],
I was thinking about similar things while working on YARN-5139. One issue I can
think of is, it will break ordering of applications/queues/schedulerKeys.
The ideal solution in my mind is pass multiple nodes while doing scheduling and
scoring/sorting nodes based on placement policy (for example, based on
locality).
There're some optimizations we can make to reduce the overhead of sorting. I
put some of the stuffs in early patch of YARN-5139:
https://issues.apache.org/jira/secure/attachment/12825346/wip-4.YARN-5139.patch#file-36
please check LocalityNodesScorer.java.
I think we can use the similar approach to optimize affinity/anti-affinity
which I mentioned offline.
Please let me know your thoughts.
> Allow for Priority order relaxing in favor of improved node/rack locality
> --------------------------------------------------------------------------
>
> Key: YARN-6443
> URL: https://issues.apache.org/jira/browse/YARN-6443
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler, fairscheduler
> Reporter: Arun Suresh
> Assignee: Arun Suresh
>
> Currently the Schedulers examine an applications pending Requests in Priority
> order. This JIRA proposes to introduce a flag (either via the
> ApplicationMasterService::registerApplication() or via some Scheduler
> configuration) to favor an ordering that is baised to the node that is
> currently heartbeating by relaxing the priority constraint.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
