[
https://issues.apache.org/jira/browse/YARN-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363645#comment-14363645
]
Wangda Tan commented on YARN-1963:
----------------------------------
I think label-based and integer-based priorities are just two different ways to
configure as well as API. No matter we choose to use label-based or
integer-based priority, we should use integer only to implement internal logic
(like in CapacityScheduler).
In addition, we can make "label-based" priority to be just an alias of
"integer-based" priority. For example, if we define queue's usable label to be
\[0-5\], we need to add one "label" alias for *all* priorities if we want to
support label-based priority (such as VERY_LOW, LOW, NORMAL, HIGH, VERY_HIGH).
When alias is setting, YARN can accept label for priority and can show them on
web UI.
With this, configuration/API and our internal logic could be consistent.
For example, a label-based configuration:
{code}
<priority-of-queue-root.engineering>
<alias>
0:VERY_LOW, 1:LOW, 2:NORMAL, 3:HIGH, 4:VERY_HIGH
</alias>
<queue-defaults>
<default>NORMAL</default>
<max>HIGH</max>
<queue-defaults>
<user-setting users="u1, u2, g1">
<default>HIGH</default>
<max>VERY_HIGH</max>
</user-setting>
<user-setting users="u3, u4">
...
</user-setting
</prority>
{code}
Which means:
- Each integer has an alias
- For users from "u1", "u2", "g1", their default application priority is HIGH,
and they can manually set its priority up to VERY_HIGH
- For other users, their default priority is NORMAL, and they can set its
priority up to HIGH
This configuration is equal to number-based config.
{code}
<priority-of-queue-root.engineering>
<queue-defaults>
<default>2</default>
<max>3</max>
<queue-defaults>
<user-setting users="u1, u2, g1">
<default>3</default>
<max>4</max>
</user-setting>
<user-setting users="u3, u4">
...
</user-setting
</prority>
{code}
bq. ...For example, take an app framework that runs multiple utility jobs
simultaneously and needs to set the relative priorities between them. If label
names themselves are in the API then that app framework doesn't work on some
clusters that don't have the expected labels configured...
I think this may not be a very big issue, no matter application uses label or
integer, additional configuration should be made. We should only edit
configuration instead of making changes of code.
> Support priorities across applications within the same queue
> -------------------------------------------------------------
>
> Key: YARN-1963
> URL: https://issues.apache.org/jira/browse/YARN-1963
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: api, resourcemanager
> Reporter: Arun C Murthy
> Assignee: Sunil G
> Attachments: 0001-YARN-1963-prototype.patch, YARN Application
> Priorities Design.pdf, YARN Application Priorities Design_01.pdf
>
>
> It will be very useful to support priorities among applications within the
> same queue, particularly in production scenarios. It allows for finer-grained
> controls without having to force admins to create a multitude of queues, plus
> allows existing applications to continue using existing queues which are
> usually part of institutional memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
