[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930339#comment-15930339
]
Eric Payne commented on YARN-2113:
----------------------------------
Hi [~sunilg]. Thanks for this patch. I'm excited to look at it.
There does seem to be a problem with preempting when it shouldn't. If 2 apps
are in a queue, both are asking for resources, and one is over and one is under
user limit, the app that is under its user limit will get preempted.
Adding the he following unit test to
{{TestProportionalCapacityPreemptionPolicyIntraQueueUserLimit}} should
demonstrate this. I have also observed this behavior in my manual testing.
{code}
public void testIntraQueuePreemptionWithTwoRequstingUsers()
throws IOException {
// Set max preemption limit as 50%.
conf.setFloat(CapacitySchedulerConfiguration.
INTRAQUEUE_PREEMPTION_MAX_ALLOWABLE_LIMIT,
(float) 0.5);
String labelsConfig = "=100,true;";
String nodesConfig = // n1 has no label
"n1= res=100";
String queuesConfig =
// guaranteed,max,used,pending,reserved
"root(=[100 100 100 20 0]);" + // root
"-a(=[100 100 100 20 0])"; // b
String appsConfig =
// queueName\t(priority,resource,host,expression,#repeat,reserved,pending)
"a\t" // app1 in a
+ "(1,1,n1,,100,false,10,user1);" + // app1 a
"a\t" // app2 in a
+ "(1,1,n1,,40,false,10,user2)";
buildEnv(labelsConfig, nodesConfig, queuesConfig, appsConfig);
policy.editSchedule();
// app2 needs more resource and its well under its user-limit. Hence preempt
// resources from app1.
verify(mDisp, times(0)).handle(argThat(
new TestProportionalCapacityPreemptionPolicy.IsPreemptionRequestFor(
getAppAttemptId(2))));
}
{code}
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---------------------------------------------------------------
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Sunil G
> Attachments: YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
