[
https://issues.apache.org/jira/browse/YARN-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301158#comment-15301158
]
Karthik Kambatla commented on YARN-5035:
----------------------------------------
bq. Looks to me like you can never assign the last container on a node because
Resources.multiplyTo() will round down
assignmultiple and maxassign come into play only after we allocate one
container. So, we shouldn't run into this. Also, the intent is for a rough half
of the resources so we allocate multiple containers without overloading any one
machine. Let me augment the test in any case to make sure we fully allocate the
node.
bq. Also you have an "upto" in the MD code that you should fix.
Given the allocation is roughly half the containers. I should change this to
"about".
> FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on
> ----------------------------------------------------------------------------
>
> Key: YARN-5035
> URL: https://issues.apache.org/jira/browse/YARN-5035
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.8.0
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: yarn-5035-1.patch
>
>
> When assignMultiple is turned on, today the number of containers allocated to
> node within a single heartbeat is determined by maxAssign. Picking a higher
> value for this prevents the spreading of the workload, and a smaller number
> leads to slow ramp up.
> The proposal is to make it dynamic - half of the unallocated resources on the
> node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
