[
https://issues.apache.org/jira/browse/YARN-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316213#comment-17316213
]
Peter Bacsko edited comment on YARN-10564 at 4/7/21, 11:51 AM:
---------------------------------------------------------------
[~gandras] thanks for the patch.
>From coding POV it looks ok, this is more like a high level review.
There's are some things I just can't figure out (maybe I'm in a bad shape
today).
1. Let's say you set the capacity 6w for {{root.a.*}}. Then a dynamic queue
{{root.a.newparent.newchild}} get created. How does the weight settings
propagate to "newparent" and "newchild"? I kept looking at the code, but it's
just not obvious. I can see that "root.a" will have an entry in
{{templateEntries}}, but then what?
2. I can't deciper this part:
{noformat}
for (int i = 0; i <= wildcardLevel; ++i) {
queuePathParts.set(queuePathParts.size() - 1 - i, WILDCARD_QUEUE);
}
{noformat}
What's happening here?
3. There is a variable called "supportedWildcardLevel". What is "supported"
means in this context? Later on we set it to {{Math.min(queueHierarchyParts -
1, MAX_WILDCARD_LEVEL);}}. It seems to me that it is either 0 or 1, because
{{MAX_WILDCARD_LEVEL}} is 1. I assume most of the time it's going to be 1? I
don't understand what it is meant to represent.
was (Author: pbacsko):
[~gandras] thanks for the patch.
>From coding POV it looks ok, this is more like a high level review.
There's are some things I just can't figure out (maybe I'm in a bad shape
today).
1. Let's say you set the capacity 6w for {{root.a.*}}. Then a dynamic queue
{{root.a.newparent.newchild}} get created. How does the weight settings
propagate to "newparent" and "newchild"? I kept looking at the code, but it's
just not obvious. I can see that "root.a" will have an entry in
{{templateEntries}}, but then what?
2. I can't deciper this part:
{noformat}
for (int i = 0; i <= wildcardLevel; ++i) {
queuePathParts.set(queuePathParts.size() - 1 - i, WILDCARD_QUEUE);
}
{noformat}
What's happening here?
3. There is a variable called "supportedWildcardLevel". What is "supported"
means in this context? Later on we set it to {{Math.min(queueHierarchyParts -
1, MAX_WILDCARD_LEVEL);}}. It seems to me that it is either 0 or 1, because
{{MAX_WILDCARD_LEVEL}} is 1. I assume most of the time it's going to be 1?
Mentally I don't understand what it is meant to represent.
> Support Auto Queue Creation template configurations
> ---------------------------------------------------
>
> Key: YARN-10564
> URL: https://issues.apache.org/jira/browse/YARN-10564
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Andras Gyori
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10564.001.patch, YARN-10564.002.patch,
> YARN-10564.003.patch, YARN-10564.004.patch, YARN-10564.005.patch,
> YARN-10564.poc.001.patch
>
>
> Similar to how the template configuration works for ManagedParents, we need
> to support templates for the new auto queue creation logic. Proposition is to
> allow wildcards in template configs such as:
> {noformat}
> yarn.scheduler.capacity.root.*.*.weight 10{noformat}
> which would mean, that set weight to 10 of every leaf of every parent under
> root.
> We should possibly take an approach, that could support arbitrary depth of
> template configuration, because we might need to lift the limitation of auto
> queue nesting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
