[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[
https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803880#comment-15803880
]
Ying Zhang commented on YARN-4415:
--
Hi [~Naganarasimha], [~leftnoteasy], [~xinxianyin], we've encountered the same
issue during our test. Noticed that this JIRA has been opened for a while. I
understand the reason [~leftnoteasy] and [~xinxianyin] have for choosing 0 or
100 as default max capacity value if not set. But the current issue is we use 0
as default max capacity internally (using macro CSQueueUtils.EPSILON) when
allocating resource but in RM Scheduler UI showing 100 as max capacity (due to
the reason class PartitionQueueCapacitiesInfo use 100 as default value in this
case). Would we change to use same default value here to avoid the
inconsistency?
{quote}
But I think there's one thing we need to fix:
When queue.accessible-node-labels == *,
QueueCapacitiesInfo#QueueCapacitiesInfo(QueueCapacities) should call
RMNodeLabelsManager.getClusterNodeLabelNames to get all labels instead of
calling getExistingNodeLabels. So after we add/remove labels, queue's
capacities in webUI/REST response will be updated as well.
{quote}
[~leftnoteasy], I'm not sure I understand what you mean, but it might be good
that we keep using getExistingNodeLabels so that only the node label partitions
that the queue has access to can be shown in RM Scheduler UI.
> Scheduler Web Ui shows max capacity for the queue is 100% but when we submit
> application doesnt get assigned
>
>
> Key: YARN-4415
> URL: https://issues.apache.org/jira/browse/YARN-4415
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler, resourcemanager
>Affects Versions: 2.7.2
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: App info with diagnostics info.png,
> capacity-scheduler.xml, screenshot-1.png
>
>
> Steps to reproduce the issue :
> Scenario 1:
> # Configure a queue(default) with accessible node labels as *
> # create a exclusive partition *xxx* and map a NM to it
> # ensure no capacities are configured for default for label xxx
> # start an RM app with queue as default and label as xxx
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue
> Scenario 2:
> # create a nonexclusive partition *sharedPartition* and map a NM to it
> # ensure no capacities are configured for default queue
> # start an RM app with queue as *default* and label as *sharedPartition*
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue for *sharedPartition*
> For both issues cause is the same default max capacity and abs max capacity
> is set to Zero %
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[
https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057134#comment-15057134
]
Xianyin Xin commented on YARN-4415:
---
Hi [~leftnoteasy], thanks for you comments.
{quote}
You can see that there're different pros and cons to choose default values of
the two options. Frankly I don't have strong preference for all these choices.
But since we have decided default values since 2.6, I would suggest don't
change the default values.
{quote}
i understand and respect your choice. The pros and cons are just the two sides
of a coin, we must choose one. But i just feel it strange that the
access-labels are "\*" but in fact we can't access it. so in this case "\*"
means nothing except that it is just a symbol, or a abbreviation of all labels.
(what i mean is it has something contradiction with intuition when one sees
"*", i think naga has the same sense). You can claim that the access-labels and
max-capacities are two things and if we want to use it, we must set the two
separately and explicitly. If we finally choose such the way it works, i will
reserve my opinion. At last, thanks again. :)
> Scheduler Web Ui shows max capacity for the queue is 100% but when we submit
> application doesnt get assigned
>
>
> Key: YARN-4415
> URL: https://issues.apache.org/jira/browse/YARN-4415
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler, resourcemanager
>Affects Versions: 2.7.2
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: App info with diagnostics info.png,
> capacity-scheduler.xml, screenshot-1.png
>
>
> Steps to reproduce the issue :
> Scenario 1:
> # Configure a queue(default) with accessible node labels as *
> # create a exclusive partition *xxx* and map a NM to it
> # ensure no capacities are configured for default for label xxx
> # start an RM app with queue as default and label as xxx
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue
> Scenario 2:
> # create a nonexclusive partition *sharedPartition* and map a NM to it
> # ensure no capacities are configured for default queue
> # start an RM app with queue as *default* and label as *sharedPartition*
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue for *sharedPartition*
> For both issues cause is the same default max capacity and abs max capacity
> is set to Zero %
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[ https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15051245#comment-15051245 ] Naganarasimha G R commented on YARN-4415: - Hi [~wangda], bq. User doesn't need to update configurations a lot if new labels added (Assume partition will be shared to all queues) User has to change configurations a lot if new labels added (Assume partition will be shared to few queues only) Sorry was not able to get your thoughts here ... whats the difference you are trying to indicate between update and change configurations ? If Maximum-capacity for partitions is set to 100 what needs to be modified ? how is it different from the default max capacity configuration for the default partition ? I understand guaranteed capacity needs to be set to zero, but why max cap needs to be modified when its shared to few queues? > Scheduler Web Ui shows max capacity for the queue is 100% but when we submit > application doesnt get assigned > > > Key: YARN-4415 > URL: https://issues.apache.org/jira/browse/YARN-4415 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Affects Versions: 2.7.2 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: App info with diagnostics info.png, > capacity-scheduler.xml, screenshot-1.png > > > Steps to reproduce the issue : > Scenario 1: > # Configure a queue(default) with accessible node labels as * > # create a exclusive partition *xxx* and map a NM to it > # ensure no capacities are configured for default for label xxx > # start an RM app with queue as default and label as xxx > # application is stuck but scheduler ui shows 100% as max capacity for that > queue > Scenario 2: > # create a nonexclusive partition *sharedPartition* and map a NM to it > # ensure no capacities are configured for default queue > # start an RM app with queue as *default* and label as *sharedPartition* > # application is stuck but scheduler ui shows 100% as max capacity for that > queue for *sharedPartition* > For both issues cause is the same default max capacity and abs max capacity > is set to Zero % -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[
https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15049581#comment-15049581
]
Wangda Tan commented on YARN-4415:
--
[~Naganarasimha]/[~xinxianyin].
Let me try to summary what we were discussing.
There're 2 different configurations:
1) Accessible-node-labels for queue
2) Maximum-capacity for partitions
There're 4 different combinations for default values:
a. 1)=*, 2)=100
Pros:
- User doesn't need to update configurations a lot if new labels added (Assume
partition will be shared to all queues)
Cons:
- User has to change configurations a lot if new labels added (Assume partition
will be shared to few queues only)
b. 1)=*, 2)=0
Pros:
- User doesn't need to update configurations a lot if new labels added (Assume
partition will be shared to few queues only)
Cons:
- User has to change configurations a lot if new labels added (Assume partition
will be shared to all queues)
c. 1)=, 2=100
Same as b.
d. 1)=, 2=0
Same as b.
You can see that there're different pros and cons to choose default values of
the two options. Frankly I don't have strong preference for all these choices.
But since we have decided default values since 2.6, I would suggest don't
change the default values.
But I think there's one thing we need to fix:
When queue.accessible-node-labels == *,
{{QueueCapacitiesInfo#QueueCapacitiesInfo(QueueCapacities)}} should call
RMNodeLabelsManager.getClusterNodeLabelNames to get all labels instead of
calling {{getExistingNodeLabels}}. So after we add/remove labels, queue's
capacities in webUI/REST response will be updated as well.
> Scheduler Web Ui shows max capacity for the queue is 100% but when we submit
> application doesnt get assigned
>
>
> Key: YARN-4415
> URL: https://issues.apache.org/jira/browse/YARN-4415
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler, resourcemanager
>Affects Versions: 2.7.2
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: App info with diagnostics info.png,
> capacity-scheduler.xml, screenshot-1.png
>
>
> Steps to reproduce the issue :
> Scenario 1:
> # Configure a queue(default) with accessible node labels as *
> # create a exclusive partition *xxx* and map a NM to it
> # ensure no capacities are configured for default for label xxx
> # start an RM app with queue as default and label as xxx
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue
> Scenario 2:
> # create a nonexclusive partition *sharedPartition* and map a NM to it
> # ensure no capacities are configured for default queue
> # start an RM app with queue as *default* and label as *sharedPartition*
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue for *sharedPartition*
> For both issues cause is the same default max capacity and abs max capacity
> is set to Zero %
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[ https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048968#comment-15048968 ] Naganarasimha G R commented on YARN-4415: - sure [~xinxianyin] will start with the patch ones at least scenario is clear to others. If all acknowledge the issue then we can go ahead else may be effort will go wasted if my understanding or view is wrong. > Scheduler Web Ui shows max capacity for the queue is 100% but when we submit > application doesnt get assigned > > > Key: YARN-4415 > URL: https://issues.apache.org/jira/browse/YARN-4415 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Affects Versions: 2.7.2 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: App info with diagnostics info.png, > capacity-scheduler.xml, screenshot-1.png > > > Steps to reproduce the issue : > Scenario 1: > # Configure a queue(default) with accessible node labels as * > # create a exclusive partition *xxx* and map a NM to it > # ensure no capacities are configured for default for label xxx > # start an RM app with queue as default and label as xxx > # application is stuck but scheduler ui shows 100% as max capacity for that > queue > Scenario 2: > # create a nonexclusive partition *sharedPartition* and map a NM to it > # ensure no capacities are configured for default queue > # start an RM app with queue as *default* and label as *sharedPartition* > # application is stuck but scheduler ui shows 100% as max capacity for that > queue for *sharedPartition* > For both issues cause is the same default max capacity and abs max capacity > is set to Zero % -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[ https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045800#comment-15045800 ] Wangda Tan commented on YARN-4415: -- Thanks for looking at this issue, [~Naganarasimha]/[~xinxianyin]. I think QueueCapacitiesInfo should not assume maxCapacity will be > eps. We have normalizations while setting values to QueueCapacities, so we should copy exactly same value from QueueCapacities to QueueCapacitiesInfo (cap it between 0 and 1 is fine). It's a valid use case that a queue has max capacity = 0, for example, reservation system (YARN-1051) could dynamically adjust queue capacities. bq. IMO we need to fetch the capacities of a partition for a given queue from its parent, if capacities are not configured for it. And if its not configured to its parent then from its parent's capacities. if the root itself doesnt have then its should be 0 as capacity and 100 as max capacity I may not fully understand why we need to fetch parent queue's capacities while setting QueueCapacitiesInfo. As I mentioned above, QueueCapacities should have everything considered and calculated at QueueCapacities (including parent queue's capacities), correct? Thoughts? > Scheduler Web Ui shows max capacity for the queue is 100% but when we submit > application doesnt get assigned > > > Key: YARN-4415 > URL: https://issues.apache.org/jira/browse/YARN-4415 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Affects Versions: 2.7.2 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: App info with diagnostics info.png, screenshot-1.png > > > Steps to reproduce the issue : > Scenario 1: > # Configure a queue(default) with accessible node labels as * > # create a exclusive partition *xxx* and map a NM to it > # ensure no capacities are configured for default for label xxx > # start an RM app with queue as default and label as xxx > # application is stuck but scheduler ui shows 100% as max capacity for that > queue > Scenario 2: > # create a nonexclusive partition *sharedPartition* and map a NM to it > # ensure no capacities are configured for default queue > # start an RM app with queue as *default* and label as *sharedPartition* > # application is stuck but scheduler ui shows 100% as max capacity for that > queue for *sharedPartition* > For both issues cause is the same default max capacity and abs max capacity > is set to Zero % -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[
https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045338#comment-15045338
]
Naganarasimha G R commented on YARN-4415:
-
Hi [~xinxianyin]
Thanks for the feedback, but your points partially matches to the description
what have given in description
bq. This would cause confusion because the access-labels inherited from parent
have 0 max capacities. If the case is true, i agree that the inherited
access-labels has 100 max capacities by default.
I am lil confused with your decription here, but what i am trying to specify
is: max capacity of a accessible node label(xxx is accessible to queue as * is
configured) for a queue should be 100 and not 0 which is not currently
happening as the max capacity is not configured for the current queue nor its
parent.
bq. But for the two scenarios in the description, i feel the final result is
reasonable because you didnt set the access-labels for the queue and its parent
doesn't have the access-labels also, so the label is not accessable explicitly
by the queue.
i want to correct here, what i have not set is *capacities* but accessible node
labels for the queue has been set as {{*}}. so the label is accessible but the
practically resources are configured to zero by default. If label is not
accessible then it would have thrown exception while submitting the application
but it dint.
bq. But the info that the web ui shows is wrong if the above analysis is right.
i think the cause is from follow sentence in {{QueueCapacitiesInfo.java}}
Its not because of this change but its caused due in
{{CapacitySchedulerPage.LeafQueueInfoBlock.renderQueueCapacityInfo}} when we
try to fetch {{lqinfo.getCapacities().getPartitionQueueCapacitiesInfo(label)}}
we fetch PartitionQueueCapacitiesInfo with default values which sets default
max capacities as 100.
IMO we need to fetch the capacities of a partition for a given queue from its
parent, if capacities are not configured for it. And if its not configured to
its parent then from its parent's capacities. if the root itself doesnt have
then its should be 0 as capacity and 100 as max capacity
> Scheduler Web Ui shows max capacity for the queue is 100% but when we submit
> application doesnt get assigned
>
>
> Key: YARN-4415
> URL: https://issues.apache.org/jira/browse/YARN-4415
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler, resourcemanager
>Affects Versions: 2.7.2
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: App info with diagnostics info.png, screenshot-1.png
>
>
> Steps to reproduce the issue :
> Scenario 1:
> # Configure a queue(default) with accessible node labels as *
> # create a exclusive partition *xxx* and map a NM to it
> # ensure no capacities are configured for default for label xxx
> # start an RM app with queue as default and label as xxx
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue
> Scenario 2:
> # create a nonexclusive partition *sharedPartition* and map a NM to it
> # ensure no capacities are configured for default queue
> # start an RM app with queue as *default* and label as *sharedPartition*
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue for *sharedPartition*
> For both issues cause is the same default max capacity and abs max capacity
> is set to Zero %
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[ https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046147#comment-15046147 ] Xianyin Xin commented on YARN-4415: --- sorry [~Naganarasimha], i missed that, thanks for you correction. For the case of labels "*", child queues should have accessibility of all labels, and the max capacity should be guaranteed to be 100 as a default value if the admin didn't specify the access-labels list for the child queue (at which case the child queues should inherit from its parent). > Scheduler Web Ui shows max capacity for the queue is 100% but when we submit > application doesnt get assigned > > > Key: YARN-4415 > URL: https://issues.apache.org/jira/browse/YARN-4415 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Affects Versions: 2.7.2 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: App info with diagnostics info.png, screenshot-1.png > > > Steps to reproduce the issue : > Scenario 1: > # Configure a queue(default) with accessible node labels as * > # create a exclusive partition *xxx* and map a NM to it > # ensure no capacities are configured for default for label xxx > # start an RM app with queue as default and label as xxx > # application is stuck but scheduler ui shows 100% as max capacity for that > queue > Scenario 2: > # create a nonexclusive partition *sharedPartition* and map a NM to it > # ensure no capacities are configured for default queue > # start an RM app with queue as *default* and label as *sharedPartition* > # application is stuck but scheduler ui shows 100% as max capacity for that > queue for *sharedPartition* > For both issues cause is the same default max capacity and abs max capacity > is set to Zero % -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[
https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044498#comment-15044498
]
Xianyin Xin commented on YARN-4415:
---
Sorry for the late, [~Naganarasimha].
I don't know i understand correctly, so pls correct me if i'm wrong. Now
there're two cases, 1), if we have set the access-labels for a queue in xml,
and 2), we didnt set the access-labels for a queue. For case 1), the
access-labels and the configured capacities (0 for capacity and 100 max by
default) are imported, and for case 2), the access-labels of the queue is
inherited from its parent, but the capacities of the labels are 0 since
{{setupConfigurableCapacities()}} only considers the configured access-labels
in xml.
{code}
this.accessibleLabels =
csContext.getConfiguration().getAccessibleNodeLabels(getQueuePath());
this.defaultLabelExpression = csContext.getConfiguration()
.getDefaultNodeLabelExpression(getQueuePath());
// inherit from parent if labels not set
if (this.accessibleLabels == null && parent != null) {
this.accessibleLabels = parent.getAccessibleNodeLabels();
}
// inherit from parent if labels not set
if (this.defaultLabelExpression == null && parent != null
&& this.accessibleLabels.containsAll(parent.getAccessibleNodeLabels()))
{
this.defaultLabelExpression = parent.getDefaultNodeLabelExpression();
}
// After we setup labels, we can setup capacities
setupConfigurableCapacities();
{code}
This would cause confusion because the access-labels inherited from parent have
0 max capacities. If the case is true, i agree that the inherited access-labels
has 100 max capacities by default.
But for the two scenarios in the descrition, i feel the final result is
reasonable because you didnt set the access-labels for the queue and its parent
doesn't have the access-labels also, so the label is not accessable explicitly
by the queue. But the info that the web ui shows is wrong if the above analysis
is right. i think the cause is from follow sentence in
{QueueCapacitiesInfo.java},
{code}
if (maxCapacity < CapacitySchedulerQueueInfo.EPSILON || maxCapacity > 1f)
maxCapacity = 1f;
{code}
where it set the {{maxCapacity}} to 1 for case {{maxCapacity == 0}} which is
just the case 2) above.
cc [~leftnoteasy].
> Scheduler Web Ui shows max capacity for the queue is 100% but when we submit
> application doesnt get assigned
>
>
> Key: YARN-4415
> URL: https://issues.apache.org/jira/browse/YARN-4415
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler, resourcemanager
>Affects Versions: 2.7.2
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: App info with diagnostics info.png, screenshot-1.png
>
>
> Steps to reproduce the issue :
> Scenario 1:
> # Configure a queue(default) with accessible node labels as *
> # create a exclusive partition *xxx* and map a NM to it
> # ensure no capacities are configured for default for label xxx
> # start an RM app with queue as default and label as xxx
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue
> Scenario 2:
> # create a nonexclusive partition *sharedPartition* and map a NM to it
> # ensure no capacities are configured for default queue
> # start an RM app with queue as *default* and label as *sharedPartition*
> # application is stuck but scheduler ui shows 100% as max capacity for that
> queue for *sharedPartition*
> For both issues cause is the same default max capacity and abs max capacity
> is set to Zero %
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[ https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042635#comment-15042635 ] Naganarasimha G R commented on YARN-4415: - [~wangda] , [~sunilg] & [~xinxianyin] Any thoughts about my previous comment ? > Scheduler Web Ui shows max capacity for the queue is 100% but when we submit > application doesnt get assigned > > > Key: YARN-4415 > URL: https://issues.apache.org/jira/browse/YARN-4415 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Affects Versions: 2.7.2 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: App info with diagnostics info.png, screenshot-1.png > > > Steps to reproduce the issue : > Scenario 1: > # Configure a queue(default) with accessible node labels as * > # create a exclusive partition *xxx* and map a NM to it > # ensure no capacities are configured for default for label xxx > # start an RM app with queue as default and label as xxx > # application is stuck but scheduler ui shows 100% as max capacity for that > queue > Scenario 2: > # create a nonexclusive partition *sharedPartition* and map a NM to it > # ensure no capacities are configured for default queue > # start an RM app with queue as *default* and label as *sharedPartition* > # application is stuck but scheduler ui shows 100% as max capacity for that > queue for *sharedPartition* > For both issues cause is the same default max capacity and abs max capacity > is set to Zero % -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4415) Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned
[ https://issues.apache.org/jira/browse/YARN-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038332#comment-15038332 ] Naganarasimha G R commented on YARN-4415: - As per offline discussion with [~wangda] he had mentioned that it was done with intent that the default max capacity of a partition is set to zero to avoid configuring the queue. IMHO i feel its much easier if we assume max capacity is 100% and calculate abs max based on its parent queue's max cap for following reasons # It will have the same behavior as that of default partition hence less confusion # May be my understanding is wrong but i feel its easier to add new partitions without touching the CS.xml as we can set the accessible nodelabels to * and assume 100% as the max capacity and 0% as guranteed capacity. And also we need to update the documentation with the default values > Scheduler Web Ui shows max capacity for the queue is 100% but when we submit > application doesnt get assigned > > > Key: YARN-4415 > URL: https://issues.apache.org/jira/browse/YARN-4415 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler, resourcemanager >Affects Versions: 2.7.2 >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: App info with diagnostics info.png, screenshot-1.png > > > Steps to reproduce the issue : > Scenario 1: > # Configure a queue(default) with accessible node labels as * > # create a exclusive partition *xxx* and map a NM to it > # ensure no capacities are configured for default for label xxx > # start an RM app with queue as default and label as xxx > # application is stuck but scheduler ui shows 100% as max capacity for that > queue > Scenario 2: > # create a nonexclusive partition *sharedPartition* and map a NM to it > # ensure no capacities are configured for default queue > # start an RM app with queue as *default* and label as *sharedPartition* > # application is stuck but scheduler ui shows 100% as max capacity for that > queue for *sharedPartition* > For both issues cause is the same default max capacity and abs max capacity > is set to Zero % -- This message was sent by Atlassian JIRA (v6.3.4#6332)
