[
https://issues.apache.org/jira/browse/YARN-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287660#comment-15287660
]
Karthik Kambatla commented on YARN-4866:
----------------------------------------
Thanks for working on this, Yufei. Comments on the patch:
# In addition to checking the cluster resources, should we also check against
queue's max resources for this queue and any parents? For the latter, a helper
method might be needed.
# Nit: Can we avoid naming the variables with a "not" for readability? For
instance, may be we could use overMaxAMShareLimit instead of notOverLimit and
exhaustsVcores instead of notTakeAllVCore and negate the expressions as
necessary?
# Nit: Also, do we really need the second boolean. How about modifying the
newly added if condition to if (!overMaxAMShareLimit && policy is not DRF) and
just override overMaxAMShareLimit? The method itself could return
!overMaxAMShareLimit?
# Nit: After the changes, if the code needs more explanation, should we add a
comment before the if condition so it is clear why the second check?
> FairScheduler: AMs can consume all vcores leading to a livelock when using
> FAIR policy
> --------------------------------------------------------------------------------------
>
> Key: YARN-4866
> URL: https://issues.apache.org/jira/browse/YARN-4866
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.8.0
> Reporter: Karthik Kambatla
> Assignee: Yufei Gu
> Attachments: YARN-4866.001.patch, YARN-4866.002.patch,
> YARN-4866.003.patch, YARN-4866.004.patch
>
>
> The maxAMShare uses the queue's policy for enforcing limits. When using FAIR
> policy, this considers only memory. If there are fewer vcores on the cluster,
> the AMs can end up taking all the vcores leading to a livelock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
