Jason Lowe commented on YARN-1963:

bq. Do we have a range? Otherwise, nothing stops users from setting their 
priority be INTEGER_MAX and everybody scratching their heads.

With any range, users could just set to the range max.  The problem of having 
all the users set their priority to the highest is an orthogonal problem to how 
priorities are represented.  As similarly suggested for labels, I think there 
should be a portion of the numerical range for highest priority reserved for 

bq. If we have a range, which side is up? is -20 >20 like unix (isn't intuitive 
at all to me) or -20 < 20 (intuitive)?

I'd prefer higher priority values lead to higher priority in scheduling, but I 
don't care too much either way.

bq. What does a negative priority means anything anyways?

A priority in isolation is meaningless since it only derives semantic meaning 
when compared to another priority.  This applies to labels as well.  A job 
running at VERY_HIGH priority is not what you originally thought if everything 
else in the cluster is running at VERY_VERY_HIGH priority.  So negative 
priorities are just a part of the numerical range, and it's straightforward to 
compare negative numbers as well as positive numbers and know their ordering.  
If it's still thought to be too confusing we can always limit priorities to >=0 
without losing much.

bq. Admin comes and says "I need a new super-high priority", now your ranges 
need to be dynamically size-able.

Like UNIX, it would be easy to add limitations to the high priority range so 
users can't just arbitrarily set their priorities to the highest level.  Then 
admins will always have the ability to make something higher priority than what 
users were able to set.

I don't mind if we want to have label mappings to numerical priorities, but my 
biggest concern with labels is putting them in the API itself.  For example, 
take an app framework that runs multiple utility jobs simultaneously and needs 
to set the relative priorities between them.  If label names themselves are in 
the API then that app framework doesn't work on some clusters that don't have 
the expected labels configured.  Or another concern: if one has to use labels 
to specify priority, what if they really need 40 different priorities?  Can 
they come up with that many descriptive label names that users can reason their 
relative ordering just based on the name?  Most likely they will end up using 
label names like PRIORITY_1, PRIORITY_2, ... PRIORITY_50, and the pain of 
having to configure all of that.

> Support priorities across applications within the same queue 
> -------------------------------------------------------------
>                 Key: YARN-1963
>                 URL: https://issues.apache.org/jira/browse/YARN-1963
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: api, resourcemanager
>            Reporter: Arun C Murthy
>            Assignee: Sunil G
>         Attachments: 0001-YARN-1963-prototype.patch, YARN Application 
> Priorities Design.pdf, YARN Application Priorities Design_01.pdf
> It will be very useful to support priorities among applications within the 
> same queue, particularly in production scenarios. It allows for finer-grained 
> controls without having to force admins to create a multitude of queues, plus 
> allows existing applications to continue using existing queues which are 
> usually part of institutional memory.

This message was sent by Atlassian JIRA

Reply via email to