[
https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820419#comment-15820419
]
Yufei Gu edited comment on YARN-4212 at 1/12/17 8:09 AM:
---------------------------------------------------------
Thanks [~rchiang] and [~kasha]'s review.
I uploaded patch 006 for all your comments, some comments:
- Change {{Set<String>}} to {{Set<Class>}} instead of {{Set<SchedulingPolicy>}}
to avoid unnecessary creation of new objects while adding items to the set.
- {{checkIfParentPolicyAllowed}} doesn't need to be recursive because of
simplicity of what policies are allowed, basically we can consider {drf, fair,
fifo} as a total order set. I modify it to a non-recursive version, and we can
change it back to recursion whenever necessary.
- Add preorder reinitialization for existing queues while reloading the alloc
file.
- Add test cases for reloading the alloc file and for policy violation in
different intermediate levels.
was (Author: yufeigu):
Thanks [~rchiang] and [~kasha]'s review.
I uploaded patch 006 for all your comments, some comments:
- Change {{Set<String>}} to {{Set<Class>}} instead of {{Set<SchedulingPolicy>}}
to avoid unnecessary new objects while adding items to the set.
- {{checkIfParentPolicyAllowed}} doesn't need to be recursive because of
simplicity of what policies are allowed, basically we can consider {drf, fair,
fifo} as a total order set. I modify it to a un-recursive version, and we can
modify it to recursion whenever necessary.
- Add preorder reinitializing for existing queues while reloading the alloc
file.
- Add test cases for reloading the alloc file and for policy violation in
different levels.
> FairScheduler: Parent queues is not allowed to be 'Fair' policy if its
> children have the "drf" policy
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-4212
> URL: https://issues.apache.org/jira/browse/YARN-4212
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Arun Suresh
> Assignee: Yufei Gu
> Labels: fairscheduler
> Attachments: YARN-4212.002.patch, YARN-4212.003.patch,
> YARN-4212.004.patch, YARN-4212.005.patch, YARN-4212.006.patch,
> YARN-4212.1.patch
>
>
> The Fair Scheduler, while performing a {{recomputeShares()}} during an
> {{update()}} call, uses the parent queues policy to distribute shares to its
> children.
> If the parent queues policy is 'fair', it only computes weight for memory and
> sets the vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and
> multiple leaf queues with policy 'drf', Any app submitted to the child queues
> with vcore requirement > 1 will always be above fairshare, since during the
> recomputeShare process, the child queues were all assigned 0 for fairshare
> vcores.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
