[
https://issues.apache.org/jira/browse/YARN-9277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766694#comment-16766694
]
Zhaohui Xin edited comment on YARN-9277 at 2/13/19 3:09 AM:
------------------------------------------------------------
Hi, [~yufeigu]. Thanks for your reply.
{quote}Correct me if I am wrong, there are no priority between Yarn jobs.
Priority has been applied to tasks inside one job, which was there before the
FS preemption overhaul. We need only priorities between mappers and reducers or
other customized priorities since AM containers are always the first priority
and have been taken care.
{quote}
You are right. Yarn jobs have the same priority in FairScheduler currently. So
this restriction is invalid in community version. This will be valid after
[YARN-2098|https://issues.apache.org/jira/browse/YARN-2098].
{code:java}
public Priority getPriority() {
// Right now per-app priorities are not passed to scheduler,
// so everyone has the same priority.
return appPriority;
}{code}
was (Author: uranus):
Hi, [~yufeigu]. Thanks for your reply.
{quote}Correct me if I am wrong, there are no priority between Yarn jobs.
Priority has been applied to tasks inside one job, which was there before the
FS preemption overhaul. We need only priorities between mappers and reducers or
other customized priorities since AM containers are always the first priority
and have been taken care.
{quote}
You are right. Yarn jobs have the same priority currently. So this restriction
is invalid in community version. BTW, we honored app's priority from
_ApplicationSubmissionContext_ in our cluster. I think the community should
also change like this, but this is another problem.
{code:java}
public Priority getPriority() {
// Right now per-app priorities are not passed to scheduler,
// so everyone has the same priority.
return appPriority;
}{code}
> Add more restrictions In FairScheduler Preemption
> --------------------------------------------------
>
> Key: YARN-9277
> URL: https://issues.apache.org/jira/browse/YARN-9277
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Reporter: Zhaohui Xin
> Assignee: Zhaohui Xin
> Priority: Major
> Attachments: YARN-9277.001.patch, YARN-9277.002.patch
>
>
>
> I think we should add more restrictions in fair scheduler preemption.
> * We should not preempt self
> * We should not preempt high priority job
> * We should not preempt container which has been running for a long time.
> * ...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
