[
https://issues.apache.org/jira/browse/YARN-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354266#comment-16354266
]
Miklos Szegedi commented on YARN-7859:
--------------------------------------
[~wangwj], thank you for the patch. Sorry, I think there is a misunderstanding.
You changed your code in the latest patch. I did not say that there is
something wrong with your code, what I was saying is that the reason for the
bug is that this code:
{code:java}
private int compareDemand(Schedulable s1, Schedulable s2) {
int res = 0;
Resource demand1 = s1.getDemand();
Resource demand2 = s2.getDemand();
if (demand1.equals(Resources.none()) && Resources.greaterThan(
RESOURCE_CALCULATOR, null, demand2, Resources.none())) {
res = 1;
} else if (demand2.equals(Resources.none()) && Resources.greaterThan(
RESOURCE_CALCULATOR, null, demand1, Resources.none())) {
res = -1;
}
return res;
}{code}
should be written something like this:
{code:java}
private int compareDemand(Schedulable s1, Schedulable s2) {
int res = 0;
Resource demand1 = s1.getDemand();
Resource demand2 = s2.getDemand();
if (demand1.equals(Resources.none()) && Resources.greaterThan(
RESOURCE_CALCULATOR, null, demand2, Resources.none())) {
res = 1;
} else if (demand2.equals(Resources.none()) && Resources.greaterThan(
RESOURCE_CALCULATOR, null, demand1, Resources.none())) {
res = -1;
}
return RESOURCE_CALCULATOR.compare(null, demand1, demand2);
}{code}
The demand is not compared right now, that is causing the starvation of your
second queue because regardless of demand they are considered equal. It is an
existing issue that I wanted to point out, not an issue with your patch.
> New feature: add queue scheduling deadLine in fairScheduler.
> ------------------------------------------------------------
>
> Key: YARN-7859
> URL: https://issues.apache.org/jira/browse/YARN-7859
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: fairscheduler
> Affects Versions: 3.0.0
> Reporter: wangwj
> Assignee: wangwj
> Priority: Major
> Labels: fairscheduler, features, patch
> Fix For: 3.0.0
>
> Attachments: YARN-7859-v1.patch, YARN-7859-v2.patch, log,
> screenshot-1.png, screenshot-3.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> As everyone knows.In FairScheduler the phenomenon of queue scheduling
> starvation often occurs when the number of cluster jobs is large.The App in
> one or more queue are pending.So I have thought a way to solve this
> problem.Add queue scheduling deadLine in fairScheduler.When a queue is not
> scheduled for FairScheduler within a specified time.We mandatory scheduler it!
> On the basis of the above, I propose this issue...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
