Sunil G commented on YARN-3937:

Thank you [~leftnoteasy] for the comments.
bq.there're always some containers will be preempted in every execution 
Yes. I agree with your point. So along with 
{{analyzeForPreemptionCancellation}}, we can do in below part of code as you 
mentioned. We can remove containers from {{preempted}} list as it might not be 
needed for preemption anymore and this loop will be done every time policy is 
ran. We can raise CANCEL_PREEMPTION events for such containers also. As 
suggested earlier, we can still have {{analyzeForPreemptionCancellation}} for 
sudden cancellation. pls share your thoughts on same.  I updated patch as per 
this idea, also addressed other comments. 

one doubt:
bq.containersToPreempt can be concurrent map
Now {{containersToPreempt}} is a Set. Hence we used the lock from 
FicaSchedulerApp itself. We need to store only ContainerID for now. With 
YARN-3784, I try to add a timeout and there I used a map. So do we need a map 
here now. pls suggest your thoughts.

> Introducing REMOVE_CONTAINER_FROM_PREEMPTION event to notify Scheduler and AM 
> when a container is no longer to be preempted
> ---------------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-3937
>                 URL: https://issues.apache.org/jira/browse/YARN-3937
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>    Affects Versions: 2.7.1
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-3937.patch, 0002-YARN-3937.patch
> As discussed in YARN-3784, there are scenarios like few other applications 
> released containers or same application has revoked its resource requests. In 
> these cases, we may not have to preempt a container which would have been 
> marked for preemption earlier. 
> Introduce a new event to remove such containers if present in the 
> to-be-preempted list of scheduler or inform AM about such a scenario.

This message was sent by Atlassian JIRA

Reply via email to