[
https://issues.apache.org/jira/browse/YARN-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617403#comment-13617403
]
Thomas Graves commented on YARN-276:
------------------------------------
I agree with Zhijie on this, if we are changing it to use actual used am
resource we don't need maxActiveApplications as it will just cause more
confusion/unneeded logic. One thing we need to think about is what to do with
the web services api as it has maxActiveApplications in it. If it no longer
applies we might need to make v2 or perhaps we come up with a better way to
hide the internal details.
I didn't look at the patch in great details so perhaps I missed something, but
why are we checking both the cluster level and the queue level? It seems like
queue level should be enough and is generally where we are most concerned about
this. If somehow one queue does go over, it shouldn't really restrict another
queue from using its share.
> Capacity Scheduler can hang when submit many jobs concurrently
> --------------------------------------------------------------
>
> Key: YARN-276
> URL: https://issues.apache.org/jira/browse/YARN-276
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Affects Versions: 3.0.0, 2.0.1-alpha
> Reporter: nemon lou
> Assignee: nemon lou
> Attachments: YARN-276.patch, YARN-276.patch, YARN-276.patch,
> YARN-276.patch, YARN-276.patch, YARN-276.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In hadoop2.0.1,When i submit many jobs concurrently at the same time,Capacity
> scheduler can hang with most resources taken up by AM and don't have enough
> resources for tasks.And then all applications hang there.
> The cause is that "yarn.scheduler.capacity.maximum-am-resource-percent" not
> check directly.Instead ,this property only used for maxActiveApplications.
> And maxActiveApplications is computed by minimumAllocation (not by Am
> actually used).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
