Sunil G commented on YARN-4462:

Hi [~Tao Jie]
Trying to understand the problem here. SO you mentioned that few applications 
were preempted and became slow, correct?. This can happen only if the resource 
usage went across the desired configured limit for that Queue. And there were 
demands (some applications waiting to get containers) to an under-serving 
queue. This would have resulted in preemption.

In preemption, we will be considering the last submitted applications as first 
candidate to preempt (within each application, will consider the lowest 
priority containers) within queue. Few tuning could be done to avoid some 
sudden trigger of preemption such as "dead-zone" etc. Pls check the preemption 
related configurations for same.

Now coming to the suggestion, I fee its kind of a hack over preemption. 
Preemption policy now chooses the apps/containers based on above explained 
criteria. And if we have more apps labelled with "do-not-preempt", I feel it 
may not give a fair preemption across applications in a queue. I am not very 
sure whether we need to give some extra power to a queue/user to save some 
v.imp apps. But needs to see how far it will help in reality. 

> Scheduler should prevent certain application from being preempted
> -----------------------------------------------------------------
>                 Key: YARN-4462
>                 URL: https://issues.apache.org/jira/browse/YARN-4462
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: scheduler
>    Affects Versions: 2.6.0
>            Reporter: Tao Jie
> When scheduler preemption is enabled, applications could be preempted if they 
> obtain resource over they should take. 
> When a mapreduce application is preempted some resource, it just runs slower. 
> However, when the preempted application is a long-run service, such as tomcat 
> running in slider, the service would fail.
> So we should have a flag for application to indicate the scheduler that those 
> application should not be preempted. 

This message was sent by Atlassian JIRA

Reply via email to