[
https://issues.apache.org/jira/browse/YARN-4462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059444#comment-15059444
]
Sunil G commented on YARN-4462:
-------------------------------
Hi [~Tao Jie]
Trying to understand the problem here. SO you mentioned that few applications
were preempted and became slow, correct?. This can happen only if the resource
usage went across the desired configured limit for that Queue. And there were
demands (some applications waiting to get containers) to an under-serving
queue. This would have resulted in preemption.
In preemption, we will be considering the last submitted applications as first
candidate to preempt (within each application, will consider the lowest
priority containers) within queue. Few tuning could be done to avoid some
sudden trigger of preemption such as "dead-zone" etc. Pls check the preemption
related configurations for same.
Now coming to the suggestion, I fee its kind of a hack over preemption.
Preemption policy now chooses the apps/containers based on above explained
criteria. And if we have more apps labelled with "do-not-preempt", I feel it
may not give a fair preemption across applications in a queue. I am not very
sure whether we need to give some extra power to a queue/user to save some
v.imp apps. But needs to see how far it will help in reality.
> Scheduler should prevent certain application from being preempted
> -----------------------------------------------------------------
>
> Key: YARN-4462
> URL: https://issues.apache.org/jira/browse/YARN-4462
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
> Affects Versions: 2.6.0
> Reporter: Tao Jie
>
> When scheduler preemption is enabled, applications could be preempted if they
> obtain resource over they should take.
> When a mapreduce application is preempted some resource, it just runs slower.
> However, when the preempted application is a long-run service, such as tomcat
> running in slider, the service would fail.
> So we should have a flag for application to indicate the scheduler that those
> application should not be preempted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
