[
https://issues.apache.org/jira/browse/YARN-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381544#comment-16381544
]
wangwj commented on YARN-7859:
------------------------------
Hi,[~yufeigu],thank you for your reply.
Sorry for some reason I did not deal with this issue for a week. Now pick
it up..
1、Global scheduling has done two years.I think It's hard to be used in the next
few years. But My method can solve this problem immediately.
2、If we take your suggestion to setting minShare to a smaller value for
long-running job queue.Others queue will must be affected and one or more
queues probably can't be shceduled for a long time..This is a phenomenon that
we all don't want to see. I've done this experiment in our cluster
In my cluster,I apply my patch, After do this.. According to the cluster
phenomenon, my method is a good solution to this problem..And we don't think
the method break fairness.Because We only mandatory scheduler a queue once when
it is not scheduled for a long time. And it has't potential risks if we have
introduce a queue property.Because only the code in this patch has used this
property.
I will continue to look at the situation of my cluster.
Anyway I will continue the Global scheduling...Before global scheduling
is available, I rely on my patch to solve the problem in my cluster..
Thx...
> New feature: add queue scheduling deadLine in fairScheduler.
> ------------------------------------------------------------
>
> Key: YARN-7859
> URL: https://issues.apache.org/jira/browse/YARN-7859
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: fairscheduler
> Affects Versions: 3.0.0
> Reporter: wangwj
> Assignee: wangwj
> Priority: Major
> Labels: fairscheduler, features, patch
> Fix For: 3.0.0
>
> Attachments: YARN-7859-v1.patch, YARN-7859-v2.patch, log,
> screenshot-1.png, screenshot-3.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> As everyone knows.In FairScheduler the phenomenon of queue scheduling
> starvation often occurs when the number of cluster jobs is large.The App in
> one or more queue are pending.So I have thought a way to solve this
> problem.Add queue scheduling deadLine in fairScheduler.When a queue is not
> scheduled for FairScheduler within a specified time.We mandatory scheduler it!
> On the basis of the above, I propose this issue...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
