[
https://issues.apache.org/jira/browse/YARN-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346762#comment-16346762
]
wangwj commented on YARN-7859:
------------------------------
In my cluster,I did an experiment.
There are two queues in my cluster:
!screenshot-2.png!
And the configuration associated with this issue are :
!screenshot-3.png!
And I run two jobs in each queue.
Of course,before the experiment, I add some log In the code.
After two jobs completed,I intercepted some of the logs...
>From the log we can see one queue has scheduled mandatory if the queue was not
>scheduled within 3S.
> New feature: add queue scheduling deadLine in fairScheduler.
> ------------------------------------------------------------
>
> Key: YARN-7859
> URL: https://issues.apache.org/jira/browse/YARN-7859
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: fairscheduler
> Affects Versions: 3.0.0
> Reporter: wangwj
> Priority: Major
> Labels: fairscheduler, features, patch
> Fix For: 3.0.0
>
> Attachments: YARN-7859-v1.patch, screenshot-1.png, screenshot-2.png,
> screenshot-3.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> As everyone knows.In FairScheduler the phenomenon of queue scheduling
> starvation often occurs when the number of cluster jobs is large.The App in
> one or more queue are pending.So I have thought a way to solve this
> problem.Add queue scheduling deadLine in fairScheduler.When a queue is not
> scheduled for FairScheduler within a specified time.We mandatory scheduler it!
> Now the way of community solves queue scheduling to starvation is preempt
> container.But this way may increases the failure rate of the job.
> On the basis of the above, I propose this issue...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
