[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739064#comment-14739064
]
Naganarasimha G R commented on YARN-4106:
-
+1, Applied the latest patch, ran the test cases and applied YARN-2729 on top
of this patch and script was also running successfully. Latest patch LGTM.
[~leftnoteasy] As far as this patch its fine but was wondering to increase
usability do we need to support YARN-2728, ??Support for disabling the
Centralized NodeLabel validation in Distributed Node Label Configuration
setup?? ?
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739172#comment-14739172
]
Hudson commented on YARN-4106:
--
FAILURE: Integrated in Hadoop-trunk-Commit #8430 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/8430/])
YARN-4106. NodeLabels for NM in distributed mode is not updated even after
clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev
77666105b4557d5706e5844a4ca286917d966c5f)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739052#comment-14739052
]
Wangda Tan commented on YARN-4106:
--
+1 to latest patch.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739097#comment-14739097
]
Naganarasimha G R commented on YARN-4106:
-
bq. Without a centralized node partition collection, capacity planning will be
not straightforward.
yes idea is to still have this collection but only difference being if the
labels sent from NM is not present @ RM, then may be Labels Manager can support
additional method which adds the missing labels first and then updates the
mapping. thoughts?
Yes it makes more sense when we support node constraint, but if we want to
support more flexibility then we can think of supporting this too.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739113#comment-14739113
]
Wangda Tan commented on YARN-4106:
--
bq. ... may be Labels Manager can support additional method which adds the
missing labels first and then updates the mapping
Doing this could be hard to manage: for example, how to deal with node label
removal, you can do that when reference count of a label becomes zero, but
resource request could be rejected if we remove a existing label. I would not
prefer to add more flexibility to node partition, since it will likely break
something.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739078#comment-14739078
]
Wangda Tan commented on YARN-4106:
--
[~Naganarasimha],
bq. Wangda Tan As far as this patch its fine but was wondering to increase
usability do we need to support YARN-2728, Support for disabling the
Centralized NodeLabel validation in Distributed Node Label Configuration setup ?
Since we only support node partition, and node partition relates to capacity
planning, etc. Without a centralized node partition collection, capacity
planning will be not straightforward.
I think YARN-2728 makes more sense when we support node constraint.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739309#comment-14739309
]
Hudson commented on YARN-4106:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #374 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/374/])
YARN-4106. NodeLabels for NM in distributed mode is not updated even after
clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev
77666105b4557d5706e5844a4ca286917d966c5f)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739642#comment-14739642
]
Hudson commented on YARN-4106:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #354 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/354/])
YARN-4106. NodeLabels for NM in distributed mode is not updated even after
clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev
77666105b4557d5706e5844a4ca286917d966c5f)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739454#comment-14739454
]
Hudson commented on YARN-4106:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2316 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2316/])
YARN-4106. NodeLabels for NM in distributed mode is not updated even after
clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev
77666105b4557d5706e5844a4ca286917d966c5f)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
* hadoop-yarn-project/CHANGES.txt
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739383#comment-14739383
]
Hudson commented on YARN-4106:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #1106 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/1106/])
YARN-4106. NodeLabels for NM in distributed mode is not updated even after
clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev
77666105b4557d5706e5844a4ca286917d966c5f)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
* hadoop-yarn-project/CHANGES.txt
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739625#comment-14739625
]
Hudson commented on YARN-4106:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2293 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2293/])
YARN-4106. NodeLabels for NM in distributed mode is not updated even after
clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev
77666105b4557d5706e5844a4ca286917d966c5f)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739290#comment-14739290
]
Hudson commented on YARN-4106:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #368 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/368/])
YARN-4106. NodeLabels for NM in distributed mode is not updated even after
clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev
77666105b4557d5706e5844a4ca286917d966c5f)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java
* hadoop-yarn-project/CHANGES.txt
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Fix For: 2.8.0
>
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738175#comment-14738175
]
Hadoop QA commented on YARN-4106:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 41s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 2s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 56s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 36s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 27s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 11s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 7m 34s | Tests passed in
hadoop-yarn-server-nodemanager. |
| | | 46m 26s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12755060/0008-YARN-4106.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f153710 |
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9074/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9074/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9074/console |
This message was automatically generated.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738065#comment-14738065
]
Hadoop QA commented on YARN-4106:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 40s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 56s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 5s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 37s | The applied patch generated 1
new checkstyle issues (total was 38, now 39). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 15s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 7m 42s | Tests passed in
hadoop-yarn-server-nodemanager. |
| | | 46m 47s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12755044/0007-YARN-4106.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f153710 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9073/artifact/patchprocess/diffcheckstylehadoop-yarn-server-nodemanager.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9073/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9073/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9073/console |
This message was automatically generated.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch, 0007-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737546#comment-14737546
]
Wangda Tan commented on YARN-4106:
--
[~bibinchundatt], Jenkins cannot show findbugs report properly. You should try
to run "mvn clean compile findbugs:findbugs" under nodemanager project, and
check target/findbugs.xml.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736025#comment-14736025
]
Bibin A Chundatt commented on YARN-4106:
Hi [~leftnoteasy]
Thnks for comments. Updates patch uploaded
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736112#comment-14736112
]
Hadoop QA commented on YARN-4106:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 37s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 55s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 4s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 36s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 30s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 1m 15s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 7m 36s | Tests passed in
hadoop-yarn-server-nodemanager. |
| | | 46m 34s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-server-nodemanager |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12754800/0006-YARN-4106.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a153b96 |
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9051/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9051/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9051/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9051/console |
This message was automatically generated.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735207#comment-14735207
]
Wangda Tan commented on YARN-4106:
--
Thanks update [~bibinchundatt], and comments from [~Naganarasimha]. Few minor
comments:
1) Make failLabelResendInterval to final before we can configure it.
2) testConfigTimer sleep time is too much, I don't know if Clock can be used in
Timer, I think you can set NM_NODE_LABELS_PROVIDER_FETCH_INTERVAL_MS to lower
value, like 1000, and sleep 1500 ms.
3) With changes in your patch, testNodeLabelsFromConfig doesn't need sleep any
more?
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736158#comment-14736158
]
Bibin A Chundatt commented on YARN-4106:
Findbug report not showing anything looks like build report problem
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> ---
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch,
> 0006-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
> *This jira also handles the below issue too*
> Timer Task not getting triggered in Nodemanager for Label update in
> nodemanager for distributed scheduling
> Task is supposed to trigger every
> {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733416#comment-14733416
]
Hadoop QA commented on YARN-4106:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 9s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 46s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 12s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 28s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 13s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 7m 41s | Tests failed in
hadoop-yarn-server-nodemanager. |
| | | 46m 4s | |
\\
\\
|| Reason || Tests ||
| Timed out tests |
org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12754457/0005-YARN-4106.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 1dbd8e3 |
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9021/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9021/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9021/console |
This message was automatically generated.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
>
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733442#comment-14733442 ] Bibin A Chundatt commented on YARN-4106: Ran locally the same testcases its working fine. Looks like not related to the patch uploaded. Please do review. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733251#comment-14733251 ] Bibin A Chundatt commented on YARN-4106: [~leftnoteasy] Thnks for looking into the issue and [~Naganarasimha] for clearing first point. Will try to update testcase for trigger check too. Testcases is working perfectly fine locally for me in eclipse. i will recheck and update. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731620#comment-14731620
]
Wangda Tan commented on YARN-4106:
--
[~bibinchundatt],
Thanks for update,
Some comments:
- Not sure if I completely understand the logic. {{updateNodeLabelsFromConfig}}
of {{ConfigurationNodeLabelsProvider}} is invoked only when TimerTask
scheduled. And TimerTask of AbstractNodeLabelsProvider delayed started for
{{intervalTime}}. Does this mean we cannot get labels when initializing
ConfigurationNodeLabelsProvider? If so, I think we need to call
updateNodeLabelsFromConfig in serviceInit of ConfigurationNodeLabelsProvider.
(Another choice is make the delay time from intervalTime to 0, but it runs in
another thread, so the behavior is undetermined. To make it simple, I suggest
to add it in serviceInit).
- Add a test for new behavior of this JIRA?
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
>
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731697#comment-14731697
]
Naganarasimha G R commented on YARN-4106:
-
Thanks for the comments [~leftnoteasy],
bq. Does this mean we cannot get labels when initializing
ConfigurationNodeLabelsProvider?
Actually {{timerTask.run();}} in {{serviceStart}} is responsible to fetch the
labels for the very first time. Reason for this modification is, if some class
extends this in future and forgets to set the labels during the start then it
will be missed hence trying to take care of this in AbstractProvider class
itself and secondly we cannot choose {{intervalTime to 0}} because there is
configuration that timer task need not run at all, so timer itself will not be
created in this case, considering all this, he has modified as per the patch.
bq. Add a test for new behavior of this JIRA
Well for second issue : i felt changes were related to the design level and
scenarios are the same. only thing we have missed is to check whether the timer
is triggered after the configured interval and the new labels are set, i think
we can have one test case for this. Apart from this were you referring to the
test case for first issue also ?
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
>
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch, 0004-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730432#comment-14730432
]
Hadoop QA commented on YARN-4106:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 41s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 59s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 5s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 37s | The applied patch generated 3
new checkstyle issues (total was 38, now 41). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 14s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 7m 3s | Tests failed in
hadoop-yarn-server-nodemanager. |
| | | 46m 9s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.nodemanager.nodelabels.TestConfigurationNodeLabelsProvider |
| | hadoop.yarn.server.nodemanager.TestNodeManagerReboot |
| | hadoop.yarn.server.nodemanager.TestNodeManagerResync |
| | hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12754152/0003-YARN-4106.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / c83d13c |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9007/artifact/patchprocess/diffcheckstylehadoop-yarn-server-nodemanager.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9007/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9007/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9007/console |
This message was automatically generated.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
>
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch,
> 0003-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730380#comment-14730380
]
Naganarasimha G R commented on YARN-4106:
-
[~bibinchundatt],
Good catch, Basically this was added initially for testing and later removed
but got missed over here so basically its sufficient that {{intervalTime}}
itself is used as {{delay}} in {{AbstractNodeLabelsProvider.serviceStart}} and
also we can remove {{startTime}} & in
{{ConfigurationNodeLabelsProvider.serviceInit}} we can remove {{startTime}} and
instead of calling {{updateNodeLabelsFromConfig}} in {{serviceInit}} we can
override and invoke {{timertask.run}} in {{serviceStart()}}. Please correct the
test cases accordingly.
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
>
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730338#comment-14730338
]
Bibin A Chundatt commented on YARN-4106:
Hi [~leftnoteasy]
Also checking distributed labels further found that Timer task is not getting
triggers as expected.
In {{ConfigurationNodeLabelsProvider#serviceInit}} {{startTime }} is getting
set wrongly
{code}
if (intervalTime != DISABLE_NODE_LABELS_PROVIDER_FETCH_TIMER) {
startTime = new Date().getTime() + intervalTime;
}
{code}
Need to remove the same.
[~Naganarasimha] have a look at the same will update patch further after your
comments..
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
>
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch
>
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726816#comment-14726816 ] Naganarasimha G R commented on YARN-4106: - Thanks for raising this issue [~bibinchundatt], This issue is happening because we are trying to avoid sending labels everytime to RM as if sent everytime RM needs to validate it and refresh the Label mapping for each Node's HB and its a unnecessary overload @ RM side. So only in this failure case, wherein the Labels are not accepted by RM then may be we can send the labels as part of each HB (or on periodic intervals) send the labels from NM to RM . Thoughts ? cc/ [~leftnoteasy] > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14727590#comment-14727590 ] Wangda Tan commented on YARN-4106: -- [~Naganarasimha], I think do the sync on periodic intervals seems reasonable to me. Maybe we can do this: when a label is not accepted by RM, it will retry every fetch-interval. If a label is accepted by RM, it won't resend unless labels updated in the node. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14727678#comment-14727678 ] Naganarasimha G R commented on YARN-4106: - Thanks for the thoughts [~leftnoteasy], Initially we also thought of the same approach but only thing is: At design level NMDistributedNodeLabelsHandler just asks for labels from provider and AbstractNodeLabelsProvider has the configuration of fetch-interval which periodically fetches the labels using the timer, so design wise would it be better to use the fetch-interval configuration in NMDistributedNodeLabelsHandler for only in the failure case ? or is it good to introduce some new configuration only for failure case here, thoughts ? > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[
https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726802#comment-14726802
]
Bibin A Chundatt commented on YARN-4106:
Observation
=
Logs shown below are when Cluster node labels are not available in RM
{code}
2015-09-02 10:21:02,766 ERROR
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node Labels
{1} reported from NM with ID host-:64318 was rejected from RM with
exception message as : Not all labels being replaced contained by known label
collections,
please check, new labels=[1]
{code}
# On restart of NM its updating fine
# Looks like once failure happens NM not resending nodelabel status in
heartbeat to RM
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
>
>
> Key: YARN-4106
> URL: https://issues.apache.org/jira/browse/YARN-4106
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
>
> NodeLabels for NM in distributed mode is not updated even after
> clusterNodelabel addition in RM
> Steps to reproduce
> ===
> # Configure nodelabel in distributed mode
> yarn.node-labels.configuration-type=distributed
> provider = config
> yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms
> # Start RM the NM
> # Once NM is registration is done add nodelabels in RM
> Nodelabels not getting updated in RM side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
