[ https://issues.apache.org/jira/browse/YARN-2009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609597#comment-15609597 ]
Wangda Tan commented on YARN-2009: ---------------------------------- [~eepayne], [~sunilg]. I'm not sure if we need to consider free resource for a queue. There're two scenarios we need to cover 1) One queue's used < configured, and cluster is full 2) One queue's used < configured, and cluster has enough free resource For 1), we should do something for the queue to make sure it can balance, if inter-queue preemption happens at the same time, less intra-queue preemption happens For 2), it is possible that we mark some resource to preemption candidate, but before these resource preempted, the queue will get more resource and previously added candidates will be cancelled. And also, the whole point of intra-queue preemption is to make sure queue can balance itself. We have configs for intra-queue preemption minimum-threshold/max-allowable-limit to make sure we will 1) Preempt queue's resource when it is not beyond a threshold 2) Do not preempt too much resource In addition, inter-queue preemption happens first before intra-queue preemption. With all the above safeguards, existing logics looks safe since we won't kill containers immediately after they get selected. I'm a little concern that considering the (configured - used) resource of queue could cause sometimes intra-queue preemption cannot be triggered. > Priority support for preemption in ProportionalCapacityPreemptionPolicy > ----------------------------------------------------------------------- > > Key: YARN-2009 > URL: https://issues.apache.org/jira/browse/YARN-2009 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler > Reporter: Devaraj K > Assignee: Sunil G > Attachments: YARN-2009.0001.patch, YARN-2009.0002.patch, > YARN-2009.0003.patch, YARN-2009.0004.patch, YARN-2009.0005.patch, > YARN-2009.0006.patch, YARN-2009.0007.patch, YARN-2009.0008.patch, > YARN-2009.0009.patch, YARN-2009.0010.patch, YARN-2009.0011.patch, > YARN-2009.0012.patch, YARN-2009.0013.patch, YARN-2009.0014.patch, > YARN-2009.0015.patch > > > While preempting containers based on the queue ideal assignment, we may need > to consider preempting the low priority application containers first. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org