[
https://issues.apache.org/jira/browse/YARN-4390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257178#comment-15257178
]
Wangda Tan commented on YARN-4390:
----------------------------------
[~jianhe].
Thanks for review:
bq. Looks like the approach is to loop over almost all containers several times
on the cluster for every preemption cycle (3 secs by default), to see whether
some containers can be preempted to make room for the reserved container on the
same node. Will this cause too much overhead in a large cluster where we have a
large amount of containers ? A unit test may test the time cost for this mega
loop.
Will run benchmark test for this.
bq. Does this equal to node.getUnallocatedResource?
Yes it is, I cannot remember why I chose not to use it, will update.
bq. Insn't FifoCandidatesSelector the first selector and the selectedCandidates
is empty ?
No, it could be second selector as well.
{code}
if (selectCandidatesForResevedContainers) {
candidatesSelectionPolicies.add(
new ReservedContainerCandidatesSelector(this));
}
// initialize candidates preemption selection policies
candidatesSelectionPolicies.add(
new FifoCandidatesSelector(this));
{code}
> Do surgical preemption based on reserved container in CapacityScheduler
> -----------------------------------------------------------------------
>
> Key: YARN-4390
> URL: https://issues.apache.org/jira/browse/YARN-4390
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Affects Versions: 3.0.0, 2.8.0, 2.7.3
> Reporter: Eric Payne
> Assignee: Wangda Tan
> Attachments: YARN-4390-design.1.pdf, YARN-4390-test-results.pdf,
> YARN-4390.1.patch, YARN-4390.2.patch, YARN-4390.3.branch-2.patch,
> YARN-4390.3.patch, YARN-4390.4.patch, YARN-4390.5.patch
>
>
> There are multiple reasons why preemption could unnecessarily preempt
> containers. One is that an app could be requesting a large container (say
> 8-GB), and the preemption monitor could conceivably preempt multiple
> containers (say 8, 1-GB containers) in order to fill the large container
> request. These smaller containers would then be rejected by the requesting AM
> and potentially given right back to the preempted app.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
