[
https://issues.apache.org/jira/browse/YARN-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214377#comment-15214377
]
Sunil G commented on YARN-4822:
-------------------------------
Hi [~leftnoteasy]
Thank you very much for doing this work item. Its looking fine and useful.
Few comments initially.
1.
{code}
Map<ApplicationAttemptId, Set<RMContainer>> toPreempt = null;
314 for (PreemptionCandidatesSelector selectionPolicy :
315 candidatesSelectionPolicies) {
316 toPreempt = selectionPolicy.selectCandidates(toPreempt,
317 clusterResources, totalPreemptionAllowed);
318 }
{code}
I am not much getting much idea abt the use of having multiple policies. Bcz if
one selected candidate from policy1 can be vetoed by another policy (now we
have only one), will it be tough to explain. Here multi layer filtering is what
happening, will it be fine we can use one policy or a group of selected policy
rather than looping all. Since we do not have a usecase now, i think its ok.
But it will be great if you could share some info on same.
2. {{PreemptableResourceCalculator}} does not have much stateful information to
keep or storing some stateful information
from premption round to round. So do we need to keep this object per
PreemptionCandidatesSelector object. May be a generic one (static util class)
is enough for whole PCPP?
> Refactor existing Preemption Policy of CS for easier adding new approach to
> select preemption candidates
> --------------------------------------------------------------------------------------------------------
>
> Key: YARN-4822
> URL: https://issues.apache.org/jira/browse/YARN-4822
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-4822.1.patch, YARN-4822.2.patch, YARN-4822.3.patch,
> YARN-4822.4.patch, YARN-4822.5.patch
>
>
> Currently, ProportionalCapacityPreemptionPolicy has hard coded logic to
> select candidates to be preempted (based on FIFO order of
> applications/containers). It's not a simple to add new candidate-selection
> logics, such as preemption for large container, intra-queeu fairness/policy,
> etc.
> In this JIRA, I propose to do following changes:
> 1) Cleanup code bases, consolidate current logic into 3 stages:
> - Compute ideal sharing of queues
> - Select to-be-preempt candidates
> - Send preemption/kill events to scheduler
> 2) Add a new interface: {{PreemptionCandidatesSelectionPolicy}} for above
> "select to-be-preempt candidates" part. Move existing how to select
> candidates logics to {{FifoPreemptionCandidatesSelectionPolicy}}.
> 3) Allow multiple PreemptionCandidatesSelectionPolicies work together in a
> chain. Preceding PreemptionCandidatesSelectionPolicy has higher priority to
> select candidates, and later PreemptionCandidatesSelectionPolicy can make
> decisions according to already selected candidates and pre-computed queue
> ideal shares of resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
