[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[ https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227484#comment-15227484 ] Sunil G commented on YARN-4699: --- Thank you very much [~leftnoteasy] for the review and commit... > Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to > change label of a node > > > Key: YARN-4699 > URL: https://issues.apache.org/jira/browse/YARN-4699 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.7.2 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch, > AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png, > ForLabelY-AfterSwitch.png > > > Scenario is as follows: > a. 2 nodes are available in the cluster (node1 with label "x", node2 with > label "y") > b. Submit an application to node1 for label "x". > c. Change node1 label to "y" by using *replaceLabelsOnNode* command. > d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute > Capacity" etc. "x" still shows some capacity. > e. Change node1 label back to "x" and verify UI and REST o/p > Output: > 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels > is changed for a node. > 2. UI tab for respective label shows wrong GREEN color in these cases. > 3. REST o/p is wrong for each label after executing above scenario. > Attaching screen shots also. This ticket will try to cover UI and REST o/p > fix when label is changed runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[ https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227386#comment-15227386 ] Hudson commented on YARN-4699: -- FAILURE: Integrated in Hadoop-trunk-Commit #9565 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9565/]) YARN-4699. Scheduler UI and REST o/p is not in sync when (wangda: rev 21eb4284487d6f8e4beedb8a0c3168e952f224fc) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CSQueueUtils.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerNodeLabelUpdate.java > Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to > change label of a node > > > Key: YARN-4699 > URL: https://issues.apache.org/jira/browse/YARN-4699 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.7.2 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Fix For: 2.8.0 > > Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch, > AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png, > ForLabelY-AfterSwitch.png > > > Scenario is as follows: > a. 2 nodes are available in the cluster (node1 with label "x", node2 with > label "y") > b. Submit an application to node1 for label "x". > c. Change node1 label to "y" by using *replaceLabelsOnNode* command. > d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute > Capacity" etc. "x" still shows some capacity. > e. Change node1 label back to "x" and verify UI and REST o/p > Output: > 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels > is changed for a node. > 2. UI tab for respective label shows wrong GREEN color in these cases. > 3. REST o/p is wrong for each label after executing above scenario. > Attaching screen shots also. This ticket will try to cover UI and REST o/p > fix when label is changed runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[ https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223714#comment-15223714 ] Wangda Tan commented on YARN-4699: -- bq. Otherwise I need to create another Manager class which implements dispatcher too. Thoughts? I think your latest patch is the best solution. +1 to latest patch, thanks. > Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to > change label of a node > > > Key: YARN-4699 > URL: https://issues.apache.org/jira/browse/YARN-4699 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.7.2 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch, > AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png, > ForLabelY-AfterSwitch.png > > > Scenario is as follows: > a. 2 nodes are available in the cluster (node1 with label "x", node2 with > label "y") > b. Submit an application to node1 for label "x". > c. Change node1 label to "y" by using *replaceLabelsOnNode* command. > d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute > Capacity" etc. "x" still shows some capacity. > e. Change node1 label back to "x" and verify UI and REST o/p > Output: > 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels > is changed for a node. > 2. UI tab for respective label shows wrong GREEN color in these cases. > 3. REST o/p is wrong for each label after executing above scenario. > Attaching screen shots also. This ticket will try to cover UI and REST o/p > fix when label is changed runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[ https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223675#comment-15223675 ] Sunil G commented on YARN-4699: --- Thank you [~leftnoteasy] for the comments. I have one more doubt here. Since we use NullNodeLabelManager for tests, dispatcher is going null. So the event to RM will never be sent. Otherwise I need to create another Manager class which implements dispatcher too. Thoughts? > Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to > change label of a node > > > Key: YARN-4699 > URL: https://issues.apache.org/jira/browse/YARN-4699 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.7.2 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch, > AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png, > ForLabelY-AfterSwitch.png > > > Scenario is as follows: > a. 2 nodes are available in the cluster (node1 with label "x", node2 with > label "y") > b. Submit an application to node1 for label "x". > c. Change node1 label to "y" by using *replaceLabelsOnNode* command. > d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute > Capacity" etc. "x" still shows some capacity. > e. Change node1 label back to "x" and verify UI and REST o/p > Output: > 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels > is changed for a node. > 2. UI tab for respective label shows wrong GREEN color in these cases. > 3. REST o/p is wrong for each label after executing above scenario. > Attaching screen shots also. This ticket will try to cover UI and REST o/p > fix when label is changed runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[ https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223662#comment-15223662 ] Wangda Tan commented on YARN-4699: -- [~sunilg], Yes you're correct, we should call replaceLabelsOnNode. However, I think we don't have to call both of cs.handle(... NodeLabelsUpdateSchedulerEvent...) and replaceLabelsOnNode. Instead, I would suggest to call replaceLabelsOnNode, and monitoring SchedulerNode.getNodePartition get updated. Thoughts? > Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to > change label of a node > > > Key: YARN-4699 > URL: https://issues.apache.org/jira/browse/YARN-4699 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.7.2 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch, > AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png, > ForLabelY-AfterSwitch.png > > > Scenario is as follows: > a. 2 nodes are available in the cluster (node1 with label "x", node2 with > label "y") > b. Submit an application to node1 for label "x". > c. Change node1 label to "y" by using *replaceLabelsOnNode* command. > d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute > Capacity" etc. "x" still shows some capacity. > e. Change node1 label back to "x" and verify UI and REST o/p > Output: > 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels > is changed for a node. > 2. UI tab for respective label shows wrong GREEN color in these cases. > 3. REST o/p is wrong for each label after executing above scenario. > Attaching screen shots also. This ticket will try to cover UI and REST o/p > fix when label is changed runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[
https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15223423#comment-15223423
]
Sunil G commented on YARN-4699:
---
Hi [~leftnoteasy]
Thank you very much for sharing the comments:
I have one doubt here. As per my change in patch, I use below code:
{code}
CSQueueUtils.updateUsedCapacity(resourceCalculator,
labelManager.getResourceByLabel(nodeLabel, Resources.none()),
minimumAllocation, queueUsage, queueCapacities, nodeLabel);
{code}
And here I invoke {{ labelManager.getResourceByLabel(nodeLabel,
Resources.none()}}. label resource in NodeLabelManager will get updated only in
CommonNodeLabelsManager and its happening via {{replaceLabelsOnNode}}. Yes, all
scheduler metrics will work via the event {{NodeLabelsUpdateSchedulerEvent}}
you mentioned. But If I am not invoking {{replaceLabelsOnNode}},
{{getResourceByLabel}} will return me empty resource. Could you pls help to
suggest if I am missing something here. Thank You.
> Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to
> change label of a node
>
>
> Key: YARN-4699
> URL: https://issues.apache.org/jira/browse/YARN-4699
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.7.2
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch,
> AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png,
> ForLabelY-AfterSwitch.png
>
>
> Scenario is as follows:
> a. 2 nodes are available in the cluster (node1 with label "x", node2 with
> label "y")
> b. Submit an application to node1 for label "x".
> c. Change node1 label to "y" by using *replaceLabelsOnNode* command.
> d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute
> Capacity" etc. "x" still shows some capacity.
> e. Change node1 label back to "x" and verify UI and REST o/p
> Output:
> 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels
> is changed for a node.
> 2. UI tab for respective label shows wrong GREEN color in these cases.
> 3. REST o/p is wrong for each label after executing above scenario.
> Attaching screen shots also. This ticket will try to cover UI and REST o/p
> fix when label is changed runtime.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[
https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220681#comment-15220681
]
Wangda Tan commented on YARN-4699:
--
[~sunilg],
Thanks for working on this, fix looks good to me. I think following code in
test may not needed:
bq. mgr.replaceLabelsOnNode(ImmutableMap.of(nm1.getNodeId(), toSet("z")));
IIRC, if we do:
bq. cs.handle(new
NodeLabelsUpdateSchedulerEvent(ImmutableMap.of(nm1.getNodeId(), ..
Labels-related resources should be updated synchronously.
> Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to
> change label of a node
>
>
> Key: YARN-4699
> URL: https://issues.apache.org/jira/browse/YARN-4699
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.7.2
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch,
> AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png,
> ForLabelY-AfterSwitch.png
>
>
> Scenario is as follows:
> a. 2 nodes are available in the cluster (node1 with label "x", node2 with
> label "y")
> b. Submit an application to node1 for label "x".
> c. Change node1 label to "y" by using *replaceLabelsOnNode* command.
> d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute
> Capacity" etc. "x" still shows some capacity.
> e. Change node1 label back to "x" and verify UI and REST o/p
> Output:
> 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels
> is changed for a node.
> 2. UI tab for respective label shows wrong GREEN color in these cases.
> 3. REST o/p is wrong for each label after executing above scenario.
> Attaching screen shots also. This ticket will try to cover UI and REST o/p
> fix when label is changed runtime.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[ https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218644#comment-15218644 ] Sunil G commented on YARN-4699: --- TestRMHA is passing locally. I added some sleep in b/w for existing test case since resource updation to label was slower. [~leftnoteasy] thoughts? > Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to > change label of a node > > > Key: YARN-4699 > URL: https://issues.apache.org/jira/browse/YARN-4699 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.7.2 >Reporter: Sunil G >Assignee: Sunil G >Priority: Critical > Attachments: 0001-YARN-4699.patch, 0002-YARN-4699.patch, > AfterAppFInish-LabelY-Metrics.png, ForLabelX-AfterSwitch.png, > ForLabelY-AfterSwitch.png > > > Scenario is as follows: > a. 2 nodes are available in the cluster (node1 with label "x", node2 with > label "y") > b. Submit an application to node1 for label "x". > c. Change node1 label to "y" by using *replaceLabelsOnNode* command. > d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute > Capacity" etc. "x" still shows some capacity. > e. Change node1 label back to "x" and verify UI and REST o/p > Output: > 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels > is changed for a node. > 2. UI tab for respective label shows wrong GREEN color in these cases. > 3. REST o/p is wrong for each label after executing above scenario. > Attaching screen shots also. This ticket will try to cover UI and REST o/p > fix when label is changed runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[
https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203557#comment-15203557
]
Hadoop QA commented on YARN-4699:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
41s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 6s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 86m 49s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 73m 58s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 177m 21s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| JDK v1.8.0_74 Timed out junit tests |
org.apache.hadoop.yarn.server.resourcemanager.TestRMHA |
| JDK v1.7.0_95 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12794426/0002-YARN-4699.patch |
| JIRA Issue | YARN-4699 |
| Optional
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[
https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200371#comment-15200371
]
Hadoop QA commented on YARN-4699:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 8s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 34s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 5s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m 3s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m 53s {color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 153m 21s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| JDK v1.7.0_95 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12789619/0001-YARN-4699.patch |
| JIRA
[jira] [Commented] (YARN-4699) Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node
[
https://issues.apache.org/jira/browse/YARN-4699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187580#comment-15187580
]
Sunil G commented on YARN-4699:
---
HI [~leftnoteasy]
This patch solves all issues related to capacity when we do
{{-replaceLabelsOnNode}} on running and used NodeManager. We had to update
capacity also which was missing.
Could pls share your thoughts.
> Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to
> change label of a node
>
>
> Key: YARN-4699
> URL: https://issues.apache.org/jira/browse/YARN-4699
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 2.7.2
>Reporter: Sunil G
>Assignee: Sunil G
>Priority: Critical
> Attachments: 0001-YARN-4699.patch, AfterAppFInish-LabelY-Metrics.png,
> ForLabelX-AfterSwitch.png, ForLabelY-AfterSwitch.png
>
>
> Scenario is as follows:
> a. 2 nodes are available in the cluster (node1 with label "x", node2 with
> label "y")
> b. Submit an application to node1 for label "x".
> c. Change node1 label to "y" by using *replaceLabelsOnNode* command.
> d. Verify Scheduler UI for metrics such as "Used Capacity", "Absolute
> Capacity" etc. "x" still shows some capacity.
> e. Change node1 label back to "x" and verify UI and REST o/p
> Output:
> 1. "Used Capacity", "Absolute Capacity" etc are not decremented once labels
> is changed for a node.
> 2. UI tab for respective label shows wrong GREEN color in these cases.
> 3. REST o/p is wrong for each label after executing above scenario.
> Attaching screen shots also. This ticket will try to cover UI and REST o/p
> fix when label is changed runtime.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
