Eric Payne commented on YARN-2592:

Closing this, since it is expected that as long as there are available 
resources, queue usage should grow evenly based on percentage of absolute 
capacity, even when preemption can happen to fill this growth as long as the 
absolute max capacity is not reached and the queues are growing evenly.

> Preemption can kill containers to fulfil need of already over-capacity queue.
> -----------------------------------------------------------------------------
>                 Key: YARN-2592
>                 URL: https://issues.apache.org/jira/browse/YARN-2592
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 2.5.1
>            Reporter: Eric Payne
> There are scenarios in which one over-capacity queue can cause preemption of 
> another over-capacity queue. However, since killing containers may lose work, 
> it doesn't make sense to me to kill containers to feed an already 
> over-capacity queue.
> Consider the following:
> {code}
> root has A,B,C, total capacity = 90
> A.guaranteed = 30, A.pending = 5, A.current = 40
> B.guaranteed = 30, B.pending = 0, B.current = 50
> C.guaranteed = 30, C.pending = 0, C.current = 0
> {code}
> In this case, the queue preemption monitor will kill 5 resources from queue B 
> so that queue A can pick them up, even though queue A is already over its 
> capacity. This could lose any work that those containers in B had already 
> done.
> Is there a use case for this behavior? It seems to me that if a queue is 
> already over its capacity, it shouldn't destroy the work of other queues. If 
> the over-capacity queue needs more resources, that seems to be a problem that 
> should be solved by increasing its guarantee.

This message was sent by Atlassian JIRA

Reply via email to