[
https://issues.apache.org/jira/browse/YARN-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630939#comment-13630939
]
nemon lou commented on YARN-276:
--------------------------------
[~tgraves]
Here is the initial thoughts on checking cluster level AM resource percent in
each leafqueue:
Leaf queue'capacity is computed based on absoluteMaxCapacity.
Considering we have 10 leaf queues,each with a value of 100%
absoluteMaxCapacity and 10% maxAMResourcePerQueuePercent configured,
there is still a chance that all leaf queue's resources taken up by AM before
reaching the 10% maxAMResourcePerQueuePercent limit.
Note that a cluster basis' am resource percent only works in leaf queue if no
am resource percent configured for this leaf queue.
As Thomas Graves mentioned,cluster level checking will causing one queue
restrict another.I will remove cluster level checking.
> Capacity Scheduler can hang when submit many jobs concurrently
> --------------------------------------------------------------
>
> Key: YARN-276
> URL: https://issues.apache.org/jira/browse/YARN-276
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Affects Versions: 3.0.0, 2.0.1-alpha
> Reporter: nemon lou
> Assignee: nemon lou
> Attachments: YARN-276.patch, YARN-276.patch, YARN-276.patch,
> YARN-276.patch, YARN-276.patch, YARN-276.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In hadoop2.0.1,When i submit many jobs concurrently at the same time,Capacity
> scheduler can hang with most resources taken up by AM and don't have enough
> resources for tasks.And then all applications hang there.
> The cause is that "yarn.scheduler.capacity.maximum-am-resource-percent" not
> check directly.Instead ,this property only used for maxActiveApplications.
> And maxActiveApplications is computed by minimumAllocation (not by Am
> actually used).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
