Sunil G commented on YARN-4390:

Hi [~eepayne]
Thank you for raising this one. We also ran in to many of this use cases while 
testing preemption. And such cases are annoying.

Adding to the use case, these 8 selected container can possibly run in multiple 
nodes too. This will result reservation for the requesting app and further set 
of preemption in next round.

As I see, YARN-4108 is trying to do a lazy preemption approach. If I am not 
wrong, scheduler will be able to detect whether a certain set of preemption to 
satisfy one huge request will be acceptable or not. If not, another set of 
preemption unit to be considered. preemption unit means collection of 
potentially to-be-preempted containers. 
May be I feel, YARN-4108 is a common solution for all such cases. Could you 
also please check that.
 cc/[~leftnoteasy] for clarifying.

> Consider container request size during CS preemption
> ----------------------------------------------------
>                 Key: YARN-4390
>                 URL: https://issues.apache.org/jira/browse/YARN-4390
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>    Affects Versions: 3.0.0, 2.8.0, 2.7.3
>            Reporter: Eric Payne
>            Assignee: Eric Payne
> There are multiple reasons why preemption could unnecessarily preempt 
> containers. One is that an app could be requesting a large container (say 
> 8-GB), and the preemption monitor could conceivably preempt multiple 
> containers (say 8, 1-GB containers) in order to fill the large container 
> request. These smaller containers would then be rejected by the requesting AM 
> and potentially given right back to the preempted app.

This message was sent by Atlassian JIRA

Reply via email to