[
https://issues.apache.org/jira/browse/YARN-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472262#comment-15472262
]
Wangda Tan commented on YARN-4945:
----------------------------------
[~eepayne],
bq. I actually would like to have the priority policy and the
minumum-user-limit-percent policy be turned on separately. I'm not sure of the
best way to do that, but our users don't use application priority very much.
Now application priority is not very popular because it's not included by any
release yet, but I bet it will be a popular feature in the future. :)
I would prefer to have a unified preemption policy to handle all intra-queue
preemption, because:
- We have different combination of intra preemption criteria, like priority,
fairness, user-limit, fifo. We cannot have a separate policy for each different
combination. For example, fairness + user-limit and priority + user-limit.
- Lots of common part of intra-queue preemption policies, especially after
ideal-allocation resource is calculated for each apps, we have the common logic
to select containers.
- We may need different implementation for ideal-allocation resource
calculator, one for fairness and one for fifo, both consider user-limit and
priority.
bq. Perhaps CapacitySchedulerConfiguration could have something like:
As I mentioned above, we should enable intra-queue preemption by default for
all limits. If we really need some parameters for better control, we can add
them in the future. Otherwise it cause troubles, for example: consider priority
without consider user-limit, excessive preemption could happen.
> [Umbrella] Capacity Scheduler Preemption Within a queue
> -------------------------------------------------------
>
> Key: YARN-4945
> URL: https://issues.apache.org/jira/browse/YARN-4945
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wangda Tan
> Attachments: Intra-Queue Preemption Use Cases.pdf,
> IntraQueuepreemption-CapacityScheduler (Design).pdf, YARN-2009-wip.2.patch,
> YARN-2009-wip.patch, YARN-2009-wip.v3.patch, YARN-2009.v0.patch
>
>
> This is umbrella ticket to track efforts of preemption within a queue to
> support features like:
> YARN-2009. YARN-2113. YARN-4781.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
