[
https://issues.apache.org/jira/browse/YARN-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068542#comment-16068542
]
Wei Yan commented on YARN-4161:
-------------------------------
{quote}
I got your point. I was trying to point out to an existing configuration in CS
named yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-assignments.
I thought of keep this syntax and it seems more of similar nature to me.
{quote}
i c. In that case, I agree with you to change to the same syntax as the
offswitch one.
{quote}
Thanks for clarifying. I understood the reason behind that. On an another note,
maxAssign=0 is meaning less. Correct? So value less than 0 could be considered
for infinite. And a boolean variable to switch-on / off the feature.
{quote}
Yes. so you ok with existing two config fields?
{quote}
I think assignMultiple should not impact existing
"maximum-offswitch-assignments" feature. I can see that current patch will skip
when assignMultiple is configured as false.
{quote}
Yes, now checking is little confused. I refactored to more clearer.
{code}
private boolean canAllocateMore(CSAssignment assignment, int offswitchCount,
int assignedContainers) {
// Current assignment shouldn't be empty
if (assignment == null
|| Resources.equals(assignment.getResource(), Resources.none())) {
return false;
}
// offswitch assignment should be under threshold
if (offswitchCount >= offswitchPerHeartbeatLimit) {
return false;
}
// assignMultiple should be ON, and assignedContainers should be under
threshold
return !(!assignMultiple
|| (maxAllocationPerNode != -1 && maxAllocationPerNode <=
assignedContainers));
}
{code}
{quote}
Could we rename assignMultiple to something like
assign-multiple-containers-per-heartbeat
{quote}
As mentioned above, I'll rename configs to follow CS's offswitch one.
{quote}
Are you planning to consider this per queue as well ? Or only at CS level from
top for now.
{quote}
For now, I more prefer to do it in scheduler-level. As this feature is
something that works in cluster-level, to balance the workload across NMs. I
don't have a concrete case that we need queue-level. Do u have cases that it
would be better to do per-queue config?
> Capacity Scheduler : Assign single or multiple containers per heart beat
> driven by configuration
> ------------------------------------------------------------------------------------------------
>
> Key: YARN-4161
> URL: https://issues.apache.org/jira/browse/YARN-4161
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacity scheduler
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Labels: oct16-medium
> Attachments: YARN-4161.patch, YARN-4161.patch.1
>
>
> Capacity Scheduler right now schedules multiple containers per heart beat if
> there are more resources available in the node.
> This approach works fine however in some cases its not distribute the load
> across the cluster hence throughput of the cluster suffers. I am adding
> feature to drive that using configuration by that we can control the number
> of containers assigned per heart beat.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
