[
https://issues.apache.org/jira/browse/YARN-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216674#comment-14216674
]
Wangda Tan commented on YARN-1963:
----------------------------------
Thanks [~sunilg], [~vinodkv] for your great effort on this!
I've just read through the design doc, some comments:
1) yarn.app.priority
How this get to be implemented? Does this means, any YARN application doesn't
need change a line of their code, can specify priority when submit the app
using yarn CLI?
I think if this can be done, we should extend to other YARN parameters like
queue, node-label-expression, etc.
2) Specify only highest priority for queue and user
I found there are property like
{{yarn.scheduler.root.<queue_name>.priority_label=high,low}} and
{{yarn.scheduler.capacity.root.<queue_name>.<priority_label>.acl=user1,user2}}.
I would perfer just specify only highest priority for queue and user. For
example, it doesn't make sense to me if priority = \{high,mid,low\}, and a
queue can access \{high,low\} only. Is there any benefit to specify individual
priorities instead of highest priority?
3) User limit and priority
I think we shouldn't consider user limit within priority level, because the
priority is not a specific kind of resource. Comparing to node label, you
cannot say, user-X of queue-A used 8G highest priority resource, but you can
say, user-X of queue-A used 8G resource in node with label=GPU. There's no
difference for a 2G resource allocated to highest/lowest priority.
If we want to implement this,
bq. it will not be fair to schedule resources in a uniform manner for all
application in a queue with respect to user limits.
I suggest to add preemption within queue considering priority. Upon YARN-2069,
we can considering user-limit and priority together -- while enforcing
user-limit, we always preempt from lower priority applications.
Any thoughts?
Thanks,
Wangda
> Support priorities across applications within the same queue
> -------------------------------------------------------------
>
> Key: YARN-1963
> URL: https://issues.apache.org/jira/browse/YARN-1963
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: api, resourcemanager
> Reporter: Arun C Murthy
> Assignee: Sunil G
> Attachments: YARN Application Priorities Design.pdf
>
>
> It will be very useful to support priorities among applications within the
> same queue, particularly in production scenarios. It allows for finer-grained
> controls without having to force admins to create a multitude of queues, plus
> allows existing applications to continue using existing queues which are
> usually part of institutional memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
