[jira] [Updated] (YARN-4634) Scheduler UI/Metrics need to consider cases like non-queue label mappings
[ https://issues.apache.org/jira/browse/YARN-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-4634: -- Attachment: 0005-YARN-4634.patch Attaching an updated patch as discussed. [~leftnoteasy], could you please take a look. > Scheduler UI/Metrics need to consider cases like non-queue label mappings > - > > Key: YARN-4634 > URL: https://issues.apache.org/jira/browse/YARN-4634 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.7.1 >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-4634.patch, 0002-YARN-4634.patch, > 0003-YARN-4634.patch, 0004-YARN-4634.patch, 0005-YARN-4634.patch > > > Currently when label-queue mappings are not available, there are few > assumptions taken in UI and in metrics. > In above case where labels are enabled and available in cluster but without > any queue mappings, UI displays queues under labels. This is not correct. > Currently labels enabled check and availability of labels are considered to > render scheduler UI. Henceforth we also need to check whether > - queue-mappings are available > - nodes are mapped with labels with proper exclusivity flags on > This ticket also will try to see the default configurations in queue when > labels are not mapped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4634) Scheduler UI/Metrics need to consider cases like non-queue label mappings
[ https://issues.apache.org/jira/browse/YARN-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-4634: -- Attachment: 0004-YARN-4634.patch Updating patch correcting findbugs warning. > Scheduler UI/Metrics need to consider cases like non-queue label mappings > - > > Key: YARN-4634 > URL: https://issues.apache.org/jira/browse/YARN-4634 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.7.1 >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-4634.patch, 0002-YARN-4634.patch, > 0003-YARN-4634.patch, 0004-YARN-4634.patch > > > Currently when label-queue mappings are not available, there are few > assumptions taken in UI and in metrics. > In above case where labels are enabled and available in cluster but without > any queue mappings, UI displays queues under labels. This is not correct. > Currently labels enabled check and availability of labels are considered to > render scheduler UI. Henceforth we also need to check whether > - queue-mappings are available > - nodes are mapped with labels with proper exclusivity flags on > This ticket also will try to see the default configurations in queue when > labels are not mapped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4634) Scheduler UI/Metrics need to consider cases like non-queue label mappings
[ https://issues.apache.org/jira/browse/YARN-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-4634: -- Attachment: 0003-YARN-4634.patch Updating patch as per the above mentioned comments. > Scheduler UI/Metrics need to consider cases like non-queue label mappings > - > > Key: YARN-4634 > URL: https://issues.apache.org/jira/browse/YARN-4634 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.7.1 >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-4634.patch, 0002-YARN-4634.patch, > 0003-YARN-4634.patch > > > Currently when label-queue mappings are not available, there are few > assumptions taken in UI and in metrics. > In above case where labels are enabled and available in cluster but without > any queue mappings, UI displays queues under labels. This is not correct. > Currently labels enabled check and availability of labels are considered to > render scheduler UI. Henceforth we also need to check whether > - queue-mappings are available > - nodes are mapped with labels with proper exclusivity flags on > This ticket also will try to see the default configurations in queue when > labels are not mapped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4634) Scheduler UI/Metrics need to consider cases like non-queue label mappings
[
https://issues.apache.org/jira/browse/YARN-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-4634:
--
Attachment: 0002-YARN-4634.patch
Thank you [~leftnoteasy]
Addressing this case in this new patch. I tested few cases as well
Base configuration is:
a. Labels are added in cluster
b. Few nodes were configured with label.
Now ,
*scenario 1.*
c. accessible labels for queues were kept default in cs xml
d. no capacity configurations for labels in any queue.
Here Scheduler page will show queues direct (It wont be under any labels)
*scenario 2.*
c. accessible labels for queues were kept default in cs xml
d. capacity configurations were added for some labels in one queue.
Here Scheduler page will show queues under any labels.
This covers the scenario which you mentioned..
*scenario 3.*
c. accessible labels for queues were configured in cs xml
Here Scheduler page will show queues under any labels as we have some
accessible label configurations.
*One observation*:
{{QueueCapacities#clearConfigurableFields}} only resets the capacity for all
its labels to 0 during reinitialize. So {{getExistingNodeLabels}} will still
have this labels. Now assume we removed some label from a queue, and did
reinitialize. Because of above mentioned behavior, we can get this removed
label in {{getExistingNodeLabels}}. Eventhough capacity is 0 for this label, i
think its not correct.If you feel its a pblm, i think we can handle in another
ticket. Thoughts?
> Scheduler UI/Metrics need to consider cases like non-queue label mappings
> -
>
> Key: YARN-4634
> URL: https://issues.apache.org/jira/browse/YARN-4634
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 2.7.1
>Reporter: Sunil G
>Assignee: Sunil G
> Attachments: 0001-YARN-4634.patch, 0002-YARN-4634.patch
>
>
> Currently when label-queue mappings are not available, there are few
> assumptions taken in UI and in metrics.
> In above case where labels are enabled and available in cluster but without
> any queue mappings, UI displays queues under labels. This is not correct.
> Currently labels enabled check and availability of labels are considered to
> render scheduler UI. Henceforth we also need to check whether
> - queue-mappings are available
> - nodes are mapped with labels with proper exclusivity flags on
> This ticket also will try to see the default configurations in queue when
> labels are not mapped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-4634) Scheduler UI/Metrics need to consider cases like non-queue label mappings
[ https://issues.apache.org/jira/browse/YARN-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-4634: -- Attachment: 0001-YARN-4634.patch Attaching an initial version of patch which fixes UI rendering. > Scheduler UI/Metrics need to consider cases like non-queue label mappings > - > > Key: YARN-4634 > URL: https://issues.apache.org/jira/browse/YARN-4634 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.7.1 >Reporter: Sunil G >Assignee: Sunil G > Attachments: 0001-YARN-4634.patch > > > Currently when label-queue mappings are not available, there are few > assumptions taken in UI and in metrics. > In above case where labels are enabled and available in cluster but without > any queue mappings, UI displays queues under labels. This is not correct. > Currently labels enabled check and availability of labels are considered to > render scheduler UI. Henceforth we also need to check whether > - queue-mappings are available > - nodes are mapped with labels with proper exclusivity flags on > This ticket also will try to see the default configurations in queue when > labels are not mapped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4634) Scheduler UI/Metrics need to consider cases like non-queue label mappings
[ https://issues.apache.org/jira/browse/YARN-4634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-4634: -- Issue Type: Sub-task (was: Bug) Parent: YARN-2492 > Scheduler UI/Metrics need to consider cases like non-queue label mappings > - > > Key: YARN-4634 > URL: https://issues.apache.org/jira/browse/YARN-4634 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 2.7.1 >Reporter: Sunil G >Assignee: Sunil G > > Currently when label-queue mappings are not available, there are few > assumptions taken in UI and in metrics. > In above case where labels are enabled and available in cluster but without > any queue mappings, UI displays queues under labels. This is not correct. > Currently labels enabled check and availability of labels are considered to > render scheduler UI. Henceforth we also need to check whether > - queue-mappings are available > - nodes are mapped with labels with proper exclusivity flags on > This ticket also will try to see the default configurations in queue when > labels are not mapped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
