[
https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297096#comment-15297096
]
Yufei Gu commented on YARN-4212:
--------------------------------
The way to calculate the vcore in fair policy might not be a good idea. After
discussion with [~kasha], we think the best solution is to force the policy of
parents to be drf when policy of children are drf. So we decide to do one more
sanity check while loading the allocation configure file. [~kasha] suggested a
more systematic sanity check by having a function that returns allowed parent
policies based on the input policy. For example, if one queue is fair, its
parent queue policy should be fair or drf. The complete relationships will be
the following:
- drf: drf
- fair: fair, drf
- fifo: fair, drf
> FairScheduler: Parent queues with 'Fair' policy should compute shares of all
> resources for its children during a recompute
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-4212
> URL: https://issues.apache.org/jira/browse/YARN-4212
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Arun Suresh
> Assignee: Yufei Gu
> Labels: fairscheduler
> Attachments: YARN-4212.1.patch
>
>
> The Fair Scheduler, while performing a {{recomputeShares()}} during an
> {{update()}} call, uses the parent queues policy to distribute shares to its
> children.
> If the parent queues policy is 'fair', it only computes weight for memory and
> sets the vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and
> multiple leaf queues with policy 'drf', Any app submitted to the child queues
> with vcore requirement > 1 will always be above fairshare, since during the
> recomputeShare process, the child queues were all assigned 0 for fairshare
> vcores.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
