[
https://issues.apache.org/jira/browse/YARN-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680289#comment-14680289
]
Sunil G commented on YARN-3937:
-------------------------------
Thank you [~leftnoteasy] for the comments.
bq.there're always some containers will be preempted in every execution
Yes. I agree with your point. So along with
{{analyzeForPreemptionCancellation}}, we can do in below part of code as you
mentioned. We can remove containers from {{preempted}} list as it might not be
needed for preemption anymore and this loop will be done every time policy is
ran. We can raise CANCEL_PREEMPTION events for such containers also. As
suggested earlier, we can still have {{analyzeForPreemptionCancellation}} for
sudden cancellation. pls share your thoughts on same. I updated patch as per
this idea, also addressed other comments.
one doubt:
bq.containersToPreempt can be concurrent map
Now {{containersToPreempt}} is a Set. Hence we used the lock from
FicaSchedulerApp itself. We need to store only ContainerID for now. With
YARN-3784, I try to add a timeout and there I used a map. So do we need a map
here now. pls suggest your thoughts.
> Introducing REMOVE_CONTAINER_FROM_PREEMPTION event to notify Scheduler and AM
> when a container is no longer to be preempted
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-3937
> URL: https://issues.apache.org/jira/browse/YARN-3937
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Affects Versions: 2.7.1
> Reporter: Sunil G
> Assignee: Sunil G
> Attachments: 0001-YARN-3937.patch, 0002-YARN-3937.patch
>
>
> As discussed in YARN-3784, there are scenarios like few other applications
> released containers or same application has revoked its resource requests. In
> these cases, we may not have to preempt a container which would have been
> marked for preemption earlier.
> Introduce a new event to remove such containers if present in the
> to-be-preempted list of scheduler or inform AM about such a scenario.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
