[
https://issues.apache.org/jira/browse/YARN-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353084#comment-16353084
]
Miklos Szegedi commented on YARN-7859:
--------------------------------------
[~wangwj], thank you for the debugging, patch and the suggestion. While your
patch probably works in the above mentioned scenario, it just works around the
problem. I think the issue is that this code below states that it compares the
demands but it instead says, if {{s1>0 ? "s1 is bigger": (s2>0 ? "s2 is bigger"
: "=")}}.
{code:java}
private int compareDemand(Schedulable s1, Schedulable s2) {
int res = 0;
Resource demand1 = s1.getDemand();
Resource demand2 = s2.getDemand();
if (demand1.equals(Resources.none()) && Resources.greaterThan(
RESOURCE_CALCULATOR, null, demand2, Resources.none())) {
res = 1;
} else if (demand2.equals(Resources.none()) && Resources.greaterThan(
RESOURCE_CALCULATOR, null, demand1, Resources.none())) {
res = -1;
}
return res;
}
{code}
This is not what it should do. This is not comparison and I think this is
causing the starvation of your second queue picking the first one if it has any
demand. Could you try removing your workaround and fixing the {{compare}}
function above? I think that will solve the issue with much less lines of code.
> New feature: add queue scheduling deadLine in fairScheduler.
> ------------------------------------------------------------
>
> Key: YARN-7859
> URL: https://issues.apache.org/jira/browse/YARN-7859
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: fairscheduler
> Affects Versions: 3.0.0
> Reporter: wangwj
> Assignee: wangwj
> Priority: Major
> Labels: fairscheduler, features, patch
> Fix For: 3.0.0
>
> Attachments: YARN-7859-v1.patch, log, screenshot-1.png,
> screenshot-3.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> As everyone knows.In FairScheduler the phenomenon of queue scheduling
> starvation often occurs when the number of cluster jobs is large.The App in
> one or more queue are pending.So I have thought a way to solve this
> problem.Add queue scheduling deadLine in fairScheduler.When a queue is not
> scheduled for FairScheduler within a specified time.We mandatory scheduler it!
> On the basis of the above, I propose this issue...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
