[
https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797873#comment-16797873
]
Wilfred Spiegelenburg commented on YARN-8967:
---------------------------------------------
1) I missed that one too, fixed now
3) The two for loops run over different lists. Take this example:
{code}
<queuePlacementPolicy>
<rule name='specified' />
<rule name='nestedUserQueue'>
<rule name='primaryGroup'/>
</rule>
</queuePlacementPolicy>
{code}
The first for loop run over the top level list of nodes (entries: specified and
nestedQueue). The second loop runs over the children of each entry in that
list. You cannot see the children of the top level nodes until you call
{{getChildren()}} on it. For that you need to cast the Node to an Element. I
thus cannot collapse the loop into one loop. The list also does not have an
iterator to change it to a for-each construct. The xml files also return child
Nodes that are not of the Element type for a correct configuration which means
we have to filter while traversing the list.
4) I added the same test case. We now handle that case and having no parent
rule for the nestedUserQueue correctly.
I have great difficulty removing the create and init for the first rule as I
don't know at the point that I find the first rule that I am going to find a
second one. I would need to wait until after the loop to create/init which
makes the code even more complex.
5) I had that to start with and changed it because the IDE kept complaining.
Not sure why but it works now without complaints and without the getter
methods. I might have had slightly different access modifiers. Looks far more
like a wrapper class now.
I also found that we do not correctly test and handle the cases which have
entries which are not _rules_. I updated the test cases and found that we had a
possible NPE due to the way we process the policy. These ones are covered in
{{testBrokenRules()}} and the updated tests in
{{testNestedUserQueueParsingErrors()}}
> Change FairScheduler to use PlacementRule interface
> ---------------------------------------------------
>
> Key: YARN-8967
> URL: https://issues.apache.org/jira/browse/YARN-8967
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler, fairscheduler
> Reporter: Wilfred Spiegelenburg
> Assignee: Wilfred Spiegelenburg
> Priority: Major
> Attachments: YARN-8967.001.patch, YARN-8967.002.patch,
> YARN-8967.003.patch, YARN-8967.004.patch, YARN-8967.005.patch,
> YARN-8967.006.patch, YARN-8967.007.patch, YARN-8967.008.patch,
> YARN-8967.009.patch, YARN-8967.010.patch
>
>
> The PlacementRule interface was introduced to be used by all schedulers as
> per YARN-3635. The CapacityScheduler is using it but the FairScheduler is not
> and is using its own rule definition.
> YARN-8948 cleans up the implementation and removes the CS references which
> should allow this change to go through.
> This would be the first step in using one placement rule engine for both
> schedulers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
