[
https://issues.apache.org/jira/browse/YARN-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549949#comment-14549949
]
Arun Suresh commented on YARN-3633:
-----------------------------------
Thanks for the patch [~ragarwal],
Assuming we allow, as per the patch, the first AM to be scheduled, then, as per
the example you specified in the description, the AM will take up 3GB in an 5GB
queue... presuming each worker task requires more resources that the AM (I am
guessing this should be true for most cases), then no other task can be
scheduled on that queue. and remaining queues are anyway log-jammed since the
maxAMshare logic would kick in.
Wondering if its a valid scenario..
> With Fair Scheduler, cluster can logjam when there are too many queues
> ----------------------------------------------------------------------
>
> Key: YARN-3633
> URL: https://issues.apache.org/jira/browse/YARN-3633
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: Rohit Agarwal
> Assignee: Rohit Agarwal
> Priority: Critical
> Attachments: YARN-3633.patch
>
>
> It's possible to logjam a cluster by submitting many applications at once in
> different queues.
> For example, let's say there is a cluster with 20GB of total memory. Let's
> say 4 users submit applications at the same time. The fair share of each
> queue is 5GB. Let's say that maxAMShare is 0.5. So, each queue has at most
> 2.5GB memory for AMs. If all the users requested AMs of size 3GB - the
> cluster logjams. Nothing gets scheduled even when 20GB of resources are
> available.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
