[jira] [Commented] (YARN-3627) Preemption not triggered in Fair scheduler when maxResources is set on parent queue
[
https://issues.apache.org/jira/browse/YARN-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560636#comment-14560636
]
Bibin A Chundatt commented on YARN-3627:
[~sunilg] . Thnk you for looking into the issue.
Preemption not triggered in Fair scheduler when maxResources is set on parent
queue
---
Key: YARN-3627
URL: https://issues.apache.org/jira/browse/YARN-3627
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler, scheduler
Environment: Suse 11 SP3, 2 NM
Reporter: Bibin A Chundatt
Consider the below scenario of fair configuration
Root (10Gb cluster resource)
--Q1 (maxResources 4gb)
Q1.1 (maxResources 4gb)
Q1.2 (maxResources 4gb)
--Q2 (maxResources 6GB)
No applications are running in Q2
Submit one application with to Q1.1 with 50 maps 4Gb gets allocated to Q1.1
Now submit application to Q1.2 the same will be starving for memory always.
Preemption will never get triggered since
yarn.scheduler.fair.preemption.cluster-utilization-threshold =.8 and the
cluster utilization is below .8.
*Fairscheduler.java*
{code}
private boolean shouldAttemptPreemption() {
if (preemptionEnabled) {
return (preemptionUtilizationThreshold Math.max(
(float) rootMetrics.getAllocatedMB() / clusterResource.getMemory(),
(float) rootMetrics.getAllocatedVirtualCores() /
clusterResource.getVirtualCores()));
}
return false;
}
{code}
Are we supposed to configure in running cluster maxResources 0mb and 0
cores so that all queues can take full cluster resources always if
available??
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3627) Preemption not triggered in Fair scheduler when maxResources is set on parent queue
[
https://issues.apache.org/jira/browse/YARN-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560558#comment-14560558
]
Sunil G commented on YARN-3627:
---
Hi [~bibinchundatt]
preemptionUtilizationThreshold default value is 0.8.
As per your scenario, allocated MB is 4Gb and the Total cluster is 10 GB, hence
the factor will come as 0.4.
{code}
private boolean shouldAttemptPreemption() {
if (preemptionEnabled) {
return (preemptionUtilizationThreshold Math.max(
(float) rootMetrics.getAllocatedMB() / clusterResource.getMemory(),
(float) rootMetrics.getAllocatedVirtualCores() /
clusterResource.getVirtualCores()));
}
return false;
}
{code}
So this check will not pass. And this behavior is expected, You could configure
a lesser value for preemptionUtilizationThreshold, to kick the preemption for
the level2 queues. Preferably we would like to kick preemption when the cluster
usage is above some good amount of percentage (80% as per default).
Preemption not triggered in Fair scheduler when maxResources is set on parent
queue
---
Key: YARN-3627
URL: https://issues.apache.org/jira/browse/YARN-3627
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler, scheduler
Environment: Suse 11 SP3, 2 NM
Reporter: Bibin A Chundatt
Consider the below scenario of fair configuration
Root (10Gb cluster resource)
--Q1 (maxResources 4gb)
Q1.1 (maxResources 4gb)
Q1.2 (maxResources 4gb)
--Q2 (maxResources 6GB)
No applications are running in Q2
Submit one application with to Q1.1 with 50 maps 4Gb gets allocated to Q1.1
Now submit application to Q1.2 the same will be starving for memory always.
Preemption will never get triggered since
yarn.scheduler.fair.preemption.cluster-utilization-threshold =.8 and the
cluster utilization is below .8.
*Fairscheduler.java*
{code}
private boolean shouldAttemptPreemption() {
if (preemptionEnabled) {
return (preemptionUtilizationThreshold Math.max(
(float) rootMetrics.getAllocatedMB() / clusterResource.getMemory(),
(float) rootMetrics.getAllocatedVirtualCores() /
clusterResource.getVirtualCores()));
}
return false;
}
{code}
Are we supposed to configure in running cluster maxResources 0mb and 0
cores so that all queues can take full cluster resources always if
available??
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3627) Preemption not triggered in Fair scheduler when maxResources is set on parent queue
[
https://issues.apache.org/jira/browse/YARN-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541782#comment-14541782
]
Bibin A Chundatt commented on YARN-3627:
Hi [~kasha] Thank you for looking into the same . But YARN-3405 also doesn't
change *shouldAttemptPreemption()* So the primary check of threshold will
happen . And subqueue Q1-1 is not preempted since its below threshold and Q1-2
will starve for resource.
Preemption not triggered in Fair scheduler when maxResources is set on parent
queue
---
Key: YARN-3627
URL: https://issues.apache.org/jira/browse/YARN-3627
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler, scheduler
Environment: Suse 11 SP3, 2 NM
Reporter: Bibin A Chundatt
Consider the below scenario of fair configuration
Root (10Gb cluster resource)
--Q1 (maxResources 4gb)
Q1.1 (maxResources 4gb)
Q1.2 (maxResources 4gb)
--Q2 (maxResources 6GB)
No applications are running in Q2
Submit one application with to Q1.1 with 50 maps 4Gb gets allocated to Q1.1
Now submit application to Q1.2 the same will be starving for memory always.
Preemption will never get triggered since
yarn.scheduler.fair.preemption.cluster-utilization-threshold =.8 and the
cluster utilization is below .8.
*Fairscheduler.java*
{code}
private boolean shouldAttemptPreemption() {
if (preemptionEnabled) {
return (preemptionUtilizationThreshold Math.max(
(float) rootMetrics.getAllocatedMB() / clusterResource.getMemory(),
(float) rootMetrics.getAllocatedVirtualCores() /
clusterResource.getVirtualCores()));
}
return false;
}
{code}
Are we supposed to configure in running cluster maxResources 0mb and 0
cores so that all queues can take full cluster resources always if
available??
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3627) Preemption not triggered in Fair scheduler when maxResources is set on parent queue
[
https://issues.apache.org/jira/browse/YARN-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540111#comment-14540111
]
Karthik Kambatla commented on YARN-3627:
I see this as a duplicate or closely related to YARN-3405. [~bibinchundatt] -
are you able to try out the patch there and see if it solves the issue for
here. I ll try and get to YARN-3405 later this week.
Preemption not triggered in Fair scheduler when maxResources is set on parent
queue
---
Key: YARN-3627
URL: https://issues.apache.org/jira/browse/YARN-3627
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler, scheduler
Environment: Suse 11 SP3, 2 NM
Reporter: Bibin A Chundatt
Consider the below scenario of fair configuration
Root (10Gb cluster resource)
--Q1 (maxResources 4gb)
Q1.1 (maxResources 4gb)
Q1.2 (maxResources 4gb)
--Q2 (maxResources 6GB)
No applications are running in Q2
Submit one application with to Q1.1 with 50 maps 4Gb gets allocated to Q1.1
Now submit application to Q1.2 the same will be starving for memory always.
Preemption will never get triggered since
yarn.scheduler.fair.preemption.cluster-utilization-threshold =.8 and the
cluster utilization is below .8.
*Fairscheduler.java*
{code}
private boolean shouldAttemptPreemption() {
if (preemptionEnabled) {
return (preemptionUtilizationThreshold Math.max(
(float) rootMetrics.getAllocatedMB() / clusterResource.getMemory(),
(float) rootMetrics.getAllocatedVirtualCores() /
clusterResource.getVirtualCores()));
}
return false;
}
{code}
Are we supposed to configure in running cluster maxResources 0mb and 0
cores so that all queues can take full cluster resources always if
available??
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3627) Preemption not triggered in Fair scheduler when maxResources is set on parent queue
[
https://issues.apache.org/jira/browse/YARN-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14541201#comment-14541201
]
Bibin A Chundatt commented on YARN-3627:
[~kasha] seems related to YARN-3405 .Will try the patch soon. Would be great if
YARN-3405 gets resolved.
Preemption not triggered in Fair scheduler when maxResources is set on parent
queue
---
Key: YARN-3627
URL: https://issues.apache.org/jira/browse/YARN-3627
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler, scheduler
Environment: Suse 11 SP3, 2 NM
Reporter: Bibin A Chundatt
Consider the below scenario of fair configuration
Root (10Gb cluster resource)
--Q1 (maxResources 4gb)
Q1.1 (maxResources 4gb)
Q1.2 (maxResources 4gb)
--Q2 (maxResources 6GB)
No applications are running in Q2
Submit one application with to Q1.1 with 50 maps 4Gb gets allocated to Q1.1
Now submit application to Q1.2 the same will be starving for memory always.
Preemption will never get triggered since
yarn.scheduler.fair.preemption.cluster-utilization-threshold =.8 and the
cluster utilization is below .8.
*Fairscheduler.java*
{code}
private boolean shouldAttemptPreemption() {
if (preemptionEnabled) {
return (preemptionUtilizationThreshold Math.max(
(float) rootMetrics.getAllocatedMB() / clusterResource.getMemory(),
(float) rootMetrics.getAllocatedVirtualCores() /
clusterResource.getVirtualCores()));
}
return false;
}
{code}
Are we supposed to configure in running cluster maxResources 0mb and 0
cores so that all queues can take full cluster resources always if
available??
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3627) Preemption not triggered in Fair scheduler when maxResources is set on parent queue
[
https://issues.apache.org/jira/browse/YARN-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14540074#comment-14540074
]
Bibin A Chundatt commented on YARN-3627:
Hi [~kasha] any comments in the same.
Seems in this case the pre-emption logic will not work at all. Only when
preemption threshold to really low value the same will work .
Preemption not triggered in Fair scheduler when maxResources is set on parent
queue
---
Key: YARN-3627
URL: https://issues.apache.org/jira/browse/YARN-3627
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler, scheduler
Environment: Suse 11 SP3, 2 NM
Reporter: Bibin A Chundatt
Consider the below scenario of fair configuration
Root (10Gb cluster resource)
--Q1 (maxResources 4gb)
Q1.1 (maxResources 4gb)
Q1.2 (maxResources 4gb)
--Q2 (maxResources 6GB)
No applications are running in Q2
Submit one application with to Q1.1 with 50 maps 4Gb gets allocated to Q1.1
Now submit application to Q1.2 the same will be starving for memory always.
Preemption will never get triggered since
yarn.scheduler.fair.preemption.cluster-utilization-threshold =.8 and the
cluster utilization is below .8.
*Fairscheduler.java*
{code}
private boolean shouldAttemptPreemption() {
if (preemptionEnabled) {
return (preemptionUtilizationThreshold Math.max(
(float) rootMetrics.getAllocatedMB() / clusterResource.getMemory(),
(float) rootMetrics.getAllocatedVirtualCores() /
clusterResource.getVirtualCores()));
}
return false;
}
{code}
Are we supposed to configure in running cluster maxResources 0mb and 0
cores so that all queues can take full cluster resources always if
available??
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
