[
https://issues.apache.org/jira/browse/YARN-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378954#comment-15378954
]
Naganarasimha G R commented on YARN-5342:
-----------------------------------------
Hi [~sunilg] & [~wangda],
Few thoughts on the approach mentioned by both of you
{quote}
if (Resources.greaterThan(rc,
application.getCSContext().getClusterResource(),
application.getCSContext().getClusterResourceUsage()
.getPending(node.getPartition()), Resources.none()) || StringUtils
.equals(node.getPartition(), RMNodeLabelsManager.NO_LABEL)) {
application.resetMissedNonPartitionedRequestSchedulingOpportunity(
priority);
}
{quote}
Issue i see is we are trying to reverting the counter at application level but
trying to evaluate the pending resources of current node partition. What if
multiple non exclusive partitions are there?
Similar things apply for the approaches captured by Sunil too.
I beleive idea here is to delay the allocation in non exclusive mode till some
time so that preemption doesnt kick in. so would it be a good idea to depend on
partition of the node ?
> Improve non-exclusive node partition resource allocation in Capacity Scheduler
> ------------------------------------------------------------------------------
>
> Key: YARN-5342
> URL: https://issues.apache.org/jira/browse/YARN-5342
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Sunil G
> Attachments: YARN-5342.1.patch
>
>
> In the previous implementation, one non-exclusive container allocation is
> possible when the missed-opportunity >= #cluster-nodes. And
> missed-opportunity will be reset when container allocated to any node.
> This will slow down the frequency of container allocation on non-exclusive
> node partition: *When a non-exclusive partition=x has idle resource, we can
> only allocate one container for this app in every
> X=nodemanagers.heartbeat-interval secs for the whole cluster.*
> In this JIRA, I propose a fix to reset missed-opporunity only if we have >0
> pending resource for the non-exclusive partition OR we get allocation from
> the default partition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
