[
https://issues.apache.org/jira/browse/YARN-6775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083166#comment-16083166
]
Wangda Tan commented on YARN-6775:
----------------------------------
bq. I think it is necessary because we need to keep track of the largest
reservation for which canAssignToUser() returns false.
It's more like a coding-style comment, I don't see any correctness around that,
so I'm fine with your current approach as well.
bq. Couldn't this be mutating between the various lookups that are already
occurring in today's assignContainers()?
Yeah you're correct.
bq. I'm open to keeping track of the minimum though if you feel there would be
sufficient gain.
I see your points, I think we can delay this till we have sufficient data.
> CapacityScheduler: Improvements to assignContainers()
> -----------------------------------------------------
>
> Key: YARN-6775
> URL: https://issues.apache.org/jira/browse/YARN-6775
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Affects Versions: 2.8.1, 3.0.0-alpha3
> Reporter: Nathan Roberts
> Assignee: Nathan Roberts
> Attachments: YARN-6775.001.patch
>
>
> There are several things in assignContainers() that are done multiple times
> even though the result cannot change (canAssignToUser, canAssignToQueue). Add
> some local caching to take advantage of this fact.
> Will post patch shortly. Patch includes a simple throughput test that
> demonstrates when we have users at their user-limit, the number of
> NodeUpdateSchedulerEvents we can process can be improved from 13K/sec to
> 50K/sec.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
