[
https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609848#comment-15609848
]
Eric Payne commented on YARN-2009:
----------------------------------
bq. there are cases where unnecessary preemption still happens.
I have a multi-tenant use case where 1 or 2 intra-queue preemptions occur
unnecessarily _every cycle_.
bq. many of potential computation mistakes should be able to be corrected
automatically by several rounds
This one will not. It happens every round.
I can provide the details of my manual test if you would like. I haven't found
a way yet to change the configs to make this stop happening. If you have
suggestions, please let me know.
Many of our queues are used by multiple users, and if a high priority app can
cause apps from other users to be unnecessarily preempted every preemption
cycle, that will not be acceptable by our users.
bq. Let's focus all the user-limit related issues in the next patch.
The patch works well if users don't share queues. However, my concern is that
if users share queues AND they want to use this feature to manage priorities
between apps of the same user, this will cause problems with other users in the
queue. If we put this in as it is, we are stating that this feature must not be
turned on if any preemptable queue in the cluster is multi-tenant. Is that what
we want?
> Priority support for preemption in ProportionalCapacityPreemptionPolicy
> -----------------------------------------------------------------------
>
> Key: YARN-2009
> URL: https://issues.apache.org/jira/browse/YARN-2009
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Devaraj K
> Assignee: Sunil G
> Attachments: YARN-2009.0001.patch, YARN-2009.0002.patch,
> YARN-2009.0003.patch, YARN-2009.0004.patch, YARN-2009.0005.patch,
> YARN-2009.0006.patch, YARN-2009.0007.patch, YARN-2009.0008.patch,
> YARN-2009.0009.patch, YARN-2009.0010.patch, YARN-2009.0011.patch,
> YARN-2009.0012.patch, YARN-2009.0013.patch, YARN-2009.0014.patch,
> YARN-2009.0015.patch
>
>
> While preempting containers based on the queue ideal assignment, we may need
> to consider preempting the low priority application containers first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
