Wangda Tan commented on YARN-3937:

Thanks for working on this, I just took a quick look at the patch.

I think the check of {{analyzeForPreemptionCancellation}} seems not correct to 
me. In a big cluster, it will be (likely) there're always some containers will 
be preempted in every execution of preemption check.

Instead, you can take a look at following code to see if you can leverage it.
    // Keep the preempted list clean
    for (Iterator<RMContainer> i = preempted.keySet().iterator(); i.hasNext();){
      RMContainer id = i.next();
      // garbage collect containers that are irrelevant for preemption
      if (preempted.get(id) + 2 * maxWaitTime < clock.getTime()) {

And one minor suggestion is:
- containersToPreempt can be concurrent map to avoid synchronized lock of 


> Introducing REMOVE_CONTAINER_FROM_PREEMPTION event to notify Scheduler and AM 
> when a container is no longer to be preempted
> ---------------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-3937
>                 URL: https://issues.apache.org/jira/browse/YARN-3937
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>    Affects Versions: 2.7.1
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-3937.patch, 0002-YARN-3937.patch
> As discussed in YARN-3784, there are scenarios like few other applications 
> released containers or same application has revoked its resource requests. In 
> these cases, we may not have to preempt a container which would have been 
> marked for preemption earlier. 
> Introduce a new event to remove such containers if present in the 
> to-be-preempted list of scheduler or inform AM about such a scenario.

This message was sent by Atlassian JIRA

Reply via email to