[
https://issues.apache.org/jira/browse/YARN-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372338#comment-16372338
]
Yufei Gu commented on YARN-7859:
--------------------------------
Hi [~wangwj], let me reiterate my stands. This is a real issue needs to be
solve. Thank you for filing this. I am open to any proposal. Unfortunately the
solution you propose here breaks the fairness, which leads to potential issues.
A new queue property could also bring lots of corner cases. Other than that,
global scheduling seems a right way to fix this issue. You can either wait for
it or work on it. Before global scheduling is available, there are some
workarounds(decreasing minShare of long-running job) to mitigate. Global
scheduling may not be the only right solution. I would also recommend you
rethink the solution, so that it doesn't break fairness, and validate potential
risks thoroughly if you want to introduce any new queue property. Anyway, thank
you for filing this. Great job so far.
> New feature: add queue scheduling deadLine in fairScheduler.
> ------------------------------------------------------------
>
> Key: YARN-7859
> URL: https://issues.apache.org/jira/browse/YARN-7859
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: fairscheduler
> Affects Versions: 3.0.0
> Reporter: wangwj
> Assignee: wangwj
> Priority: Major
> Labels: fairscheduler, features, patch
> Fix For: 3.0.0
>
> Attachments: YARN-7859-v1.patch, YARN-7859-v2.patch, log,
> screenshot-1.png, screenshot-3.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> As everyone knows.In FairScheduler the phenomenon of queue scheduling
> starvation often occurs when the number of cluster jobs is large.The App in
> one or more queue are pending.So I have thought a way to solve this
> problem.Add queue scheduling deadLine in fairScheduler.When a queue is not
> scheduled for FairScheduler within a specified time.We mandatory scheduler it!
> On the basis of the above, I propose this issue...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
