[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514794#comment-14514794
]
Hadoop QA commented on YARN-2740:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 59s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 7 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 37s | There were no new javac warning
messages. |
| {color:red}-1{color} | javadoc | 10m 3s | The applied patch generated 43
additional warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 4m 3s | The applied patch generated 3
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 7s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 25s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 2m 2s | Tests passed in
hadoop-yarn-common. |
| {color:red}-1{color} | yarn tests | 18m 49s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 64m 44s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.yarn.server.resourcemanager.TestApplicationCleanup
|
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
|
| | hadoop.yarn.server.resourcemanager.TestResourceTrackerService |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestRMNMRPCResponseId |
| | hadoop.yarn.server.resourcemanager.TestMoveApplication |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterService |
| | hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler |
| |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
|
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesHttpStaticUserPermissions
|
| | hadoop.yarn.server.resourcemanager.monitor.TestSchedulingMonitor |
| | hadoop.yarn.server.resourcemanager.TestRMAdminService |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication |
| |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId |
| | hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched |
| | hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens |
| | hadoop.yarn.server.resourcemanager.TestApplicationACLs |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels |
| | hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
| | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| | hadoop.yarn.server.resourcemanager.TestRMRestart |
| | hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
|
| | hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens |
| | hadoop.yarn.server.resourcemanager.TestClientRMService |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect |
| | hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA |
| | hadoop.yarn.server.resourcemanager.TestResourceManager |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
|
| | hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher |
| | hadoop.yarn.server.resourcemanager.TestRM |
| |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy
|
| |
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14515134#comment-14515134 ] Wangda Tan commented on YARN-2740: -- Have you checked it in your local environment? ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch, YARN-2740.20150423-1-merged.patch, YARN-2740.20150423-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14515121#comment-14515121
]
Hadoop QA commented on YARN-2740:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 6s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 7 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 52s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 32s | The applied patch generated 3
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 10s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 22s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 2m 12s | Tests passed in
hadoop-yarn-common. |
| {color:red}-1{color} | yarn tests | 58m 10s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 105m 44s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.yarn.server.resourcemanager.TestRM |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12728466/YARN-2740.20150423-1-merged.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6bae596 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7515/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7515/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7515/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7515/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7515/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7515/console |
This message was automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch,
YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch,
YARN-2740.20150423-1-merged.patch, YARN-2740.20150423-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14515130#comment-14515130 ] Naganarasimha G R commented on YARN-2740: - Hi [~wangda], seems like the test failure is not related to the patch. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch, YARN-2740.20150423-1-merged.patch, YARN-2740.20150423-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14515839#comment-14515839 ] Wangda Tan commented on YARN-2740: -- Just verified in local environment, TestRM can get passed. Committing... ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch, YARN-2740.20150423-1-merged.patch, YARN-2740.20150423-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512105#comment-14512105 ] Naganarasimha G R commented on YARN-2740: - Hi [~wangda], seems like jenkins results are fine can we get this in ? ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch, YARN-2740.20150423-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509455#comment-14509455 ] Wangda Tan commented on YARN-2740: -- Seems like env issue, re-kicked Jenkins. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch, YARN-2740.20150423-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509451#comment-14509451
]
Hadoop QA commented on YARN-2740:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 42s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 7 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:red}-1{color} | javac | 7m 45s | The applied patch generated 122
additional warning messages. |
| {color:red}-1{color} | javadoc | 9m 56s | The applied patch generated 9
additional warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 30s | The applied patch generated 3
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 3s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 25s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 2m 3s | Tests passed in
hadoop-yarn-common. |
| {color:red}-1{color} | yarn tests | 19m 31s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 66m 32s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler |
| | hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler |
| | hadoop.yarn.server.resourcemanager.TestApplicationACLs |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
|
| | hadoop.yarn.server.resourcemanager.TestApplicationCleanup |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue |
| | hadoop.yarn.server.resourcemanager.TestRM |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes |
| | hadoop.yarn.server.resourcemanager.TestResourceTrackerService |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification |
| | hadoop.yarn.server.resourcemanager.security.TestAMRMTokens |
| | hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerFairShare |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
|
| | hadoop.yarn.server.resourcemanager.TestMoveApplication |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerEventLog |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher |
| | hadoop.yarn.server.resourcemanager.TestRMProxyUsersConf |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesHttpStaticUserPermissions
|
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
| | hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler |
| | hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterService |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs |
| |
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy
|
| | hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels |
| | hadoop.yarn.server.resourcemanager.scheduler.TestQueueMetrics |
| |
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509449#comment-14509449 ] Wangda Tan commented on YARN-2740: -- Thanks [~Naganarasimha], will commit once Jenkins get back. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch, YARN-2740.20150423-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14509648#comment-14509648
]
Hadoop QA commented on YARN-2740:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 44s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 7 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 34s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 35s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 25s | The applied patch generated 3
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 2s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 25s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 2m 2s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 52m 24s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 98m 43s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12727647/YARN-2740.20150423-1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 3952054 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7479/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7479/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7479/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7479/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7479/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7479/console |
This message was automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch,
YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch,
YARN-2740.20150423-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507029#comment-14507029
]
Hadoop QA commented on YARN-2740:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 35s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 7 new or modified test files. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 10 line(s) that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 31s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 22s | The applied patch generated 5
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 0s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 23s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 2m 0s | Tests passed in
hadoop-yarn-common. |
| {color:red}-1{color} | yarn tests | 54m 14s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 100m 10s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12727201/YARN-2740.20150422-2.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b08908a |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7444/artifact/patchprocess/whitespace.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7444/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7444/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7444/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7444/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7444/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7444//console |
This message was automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch,
YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507396#comment-14507396 ] Naganarasimha G R commented on YARN-2740: - unit test failure is not related to this patch and can fix white space issue but not clear about the check style output, will correct it once i get some confirmation from allen ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507436#comment-14507436 ] Wangda Tan commented on YARN-2740: -- Latest patch LGTM, +1. checkstyle result lack of details and some minor formatting suggestions, will commit today. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch, YARN-2740.20150422-2.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505709#comment-14505709 ] Wangda Tan commented on YARN-2740: -- General LGTM, some minor comments. 1) Mark YarnConfiguration.isDistributedNodeLabelConfiguration to @Private 2) It's better to cover remove label case since remove label = remove label in the cluster + remove label in nodes, add a test to make sure it works in distributed mode, same as TestRMAdminService/TestRMWebServicesNodeLabels 3) RMWebServices.replaceLabelsOnNode(s) should be merged to avoid we need to maintain both. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505891#comment-14505891
]
Naganarasimha G R commented on YARN-2740:
-
Thanks for the comment [~wangda]
bq. 2) It's better to cover remove label case since remove label = remove label
in the cluster + remove label in nodes, add a test to make sure it works in
distributed mode, same as TestRMAdminService/TestRMWebServicesNodeLabels.
IIUC you want {{prevent removing clusterNodeLabel while distributed enabled}}
and add test cases for it ?
bq. 3) RMWebServices.replaceLabelsOnNode(s) should be merged to avoid we need
to maintain both.
I dont mind working on it here but 2 queries
* Is it that we should still support existing 2 rest api's for
replaceLabelsOnNode on a single node and other for multiple nodes and ensure
the common part is extracted to a new method ?
* I understand that it will make it easy for committer to limit number of
checkins but Is it good in terms of maintainability to include code changes in
the patch which is not related to this jira description ?
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch,
YARN-2740.20150421-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505972#comment-14505972 ] Naganarasimha G R commented on YARN-2740: - bq. No, we've decided to support remove node labels while distributed config enabled, I think it's better to add test to make sure it works in distributed mode. Since remove label = remove label in the cluster + remove label in nodes, it's a defensive test case to avoid future code changes break this. ok got it, had misread your comment earlier bq. We can continue to support both of the 2, just extra common part should be enough. ok, Will extract the common part to a method . ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505953#comment-14505953 ] Wangda Tan commented on YARN-2740: -- bq. IIUC you want prevent removing clusterNodeLabel while distributed enabled and add test cases for it ? No, we've decided to support remove node labels while distributed config enabled, I think it's better to add test to make sure it works in distributed mode. Since remove label = remove label in the cluster + remove label in nodes, it's a defensive test case to avoid future code changes break this. bq. Is it that we should still support existing 2 rest api's for replaceLabelsOnNode on a single node and other for multiple nodes and ensure the common part is extracted to a new method ? We can continue to support both of the 2, just extra common part should be enough. bq. I understand that it will make it easy for committer to limit number of checkins but Is it good in terms of maintainability to include code changes in the patch which is not related to this jira description ? I think it's a very small refactoring, doesn't change any of functionalities, doing it with the JIRA should be fine. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503750#comment-14503750 ] Wangda Tan commented on YARN-2740: -- Retriggered Jenkins. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503792#comment-14503792
]
Wangda Tan commented on YARN-2740:
--
Hi [~Naganarasimha],
After reread our conversation, I felt I misled you cause a typo: {{I think it's
better now prevent admin remove clusterNodeLabel when distributed enabled.}},
{{now}} should be {{not}}. I suggest not to prevent removing clusterNodeLabel
while distributed enabled. The reason is what I mentioned above.
bq. I think it's not a big problem, NM doesn't need to know x being removed,
the logic should be, NM reports label, and RM allocate according to label, NM
should just move on if adding label failed as what we done in YARN-2495. My
opinion here is not add extra RM-NM communicate.
Sorry for that.
bq. based on this i mentioned RM-NM communicate/notify would be required as
labels are sent only on change in NM side and it will not be able show that
there is error in reporting labels. In btw have raised new jira YARN-3506 for
this error handling reported in YARN-2495
This may not be a big issue, I suggest to keep it as-is temporarily since it
needs change NM/RM interface. We can address other issues before come to the
problem.
Thoughts?
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503955#comment-14503955
]
Hadoop QA commented on YARN-2740:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726618/YARN-2740.20150420-1.patch
against trunk revision d50e8f0.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7408//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7408//console
This message is automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504188#comment-14504188 ] Naganarasimha G R commented on YARN-2740: - [~wangda], Seems like most of the test cases are failing due to address bind exception which is not related to this patch ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch, YARN-2740.20150421-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504162#comment-14504162
]
Hadoop QA commented on YARN-2740:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726731/YARN-2740.20150421-1.patch
against trunk revision 44872b7.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
The following test timeouts occurred in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiTestTests
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7411//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7411//console
This message is automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch,
YARN-2740.20150421-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503515#comment-14503515
]
Hadoop QA commented on YARN-2740:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726618/YARN-2740.20150420-1.patch
against trunk revision f967fd2.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestLeveldbRMStateStore
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7401//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7401//console
This message is automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch, YARN-2740.20150420-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14500975#comment-14500975
]
Naganarasimha G R commented on YARN-2740:
-
Thanks for the comments [~wangda],
bq. I think it's not a big problem, NM doesn't need to know x being removed,
the logic should be, NM reports label, and RM allocate according to label, NM
should just move on if adding label failed
Well IIUC, based on your reply to first point ??prevent admin remove
clusterNodeLabel when distributed enabled?? we need worry about this second
point right? as user will not be able to remove cluster node label
bq. as what we done in YARN-2495. My opinion here is not add extra RM-NM
communicate.
As per last discussion in YARN-2495 you had given a concluded as per this
[comment
|https://issues.apache.org/jira/browse/YARN-2495?focusedCommentId=14358109page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14358109]
that
{quote}
* Show/log diagnostic in RM (nodes) page and NM page, saying label is invalid.
(Need modify web UI, can be done in a separated task)
* Make the node's labels to be empty, so that applications can continue use it.
{quote}
based on this i mentioned RM-NM communicate/notify would be required as labels
are sent only on change in NM side and it will not be able show that there is
error in reporting labels. In btw have raised new jira YARN-3506 for this error
handling reported in YARN-2495
Test failure is not related to this patch and will work on {{prevent admin
remove clusterNodeLabel when distributed enabled.}} and resubmit the patch.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498372#comment-14498372
]
Wangda Tan commented on YARN-2740:
--
1. CommonNodeLabelsManager:
1.1) {{@VisibleForTesting}} for isDistributedNodeLabelConfiguration is not
necessary?
1.2) How to set isDistributedNodeLabelConfiguration:
It's better to add a helper method in YarnConfiguration instead of doing
following check:
{code}
isDistributedNodeLabelConfiguration =
YarnConfiguration.DISTRIBUTED_NODELABEL_CONFIGURATION_TYPE.equals(conf
.get(YarnConfiguration.NODELABEL_CONFIGURATION_TYPE,
YarnConfiguration.DEFAULT_NODELABEL_CONFIGURATION_TYPE));
{code}
I noticed in ResourceTrackerService can leavage the helper method I suggest too:
{code}
String nodeLabelConfigurationType =
conf.get(YarnConfiguration.NODELABEL_CONFIGURATION_TYPE,
YarnConfiguration.DEFAULT_NODELABEL_CONFIGURATION_TYPE);
isDistributesNodeLabelsConf =
YarnConfiguration.DISTRIBUTED_NODELABEL_CONFIGURATION_TYPE
.equals(nodeLabelConfigurationType);
{code}
1.3) Better to add a brief comment about why we don't send message to store
2. FileSystemNodeLabelStore:
Add a note about why ignoreNodeLabelsMapping is needed
3. AdminService:
Should we throw exception when distributedConfiguration enabled for
removeClusterNodeLabels? remove will change labels on node, after removed, node
heartbeat with the removed partition will be identified as error, it seems
reasonable to me. Admin should be able to control valid-partitions in the
cluster.
Same as RMWebServices.
Test looks good to me, but may need to change if you agree with 2/3.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498528#comment-14498528 ] Naganarasimha G R commented on YARN-2740: - Thanks for the review [~wangda], bq. Should we throw exception when distributedConfiguration enabled for removeClusterNodeLabels? remove will change labels on node, after removed, node heartbeat with the removed partition will be identified as error, it seems reasonable to me. Admin should be able to control valid-partitions in the cluster. Actually dint get completely your opinion about throw exception when distributedConfiguration enabled for removeClusterNodeLabels; Did you want to throw ? if you want to allow admin to remove ClusterNodeLabels then there is one case which i can see potential problem: Assume NM informs RM with a valid node label x through HB/Resgistration and then admin removes x from cluster node labels. But this is not communicated back to NM and NM will not send labels as part of HB *unless there is change in labels in NM side*. So NM is not aware of x being removed at all. I agree we need to allow Admin to control valid partitions but in that case we need to add some logic in RM to request NM to resubmit labels. Please provide ur views. Will correct other issues as part of next patch. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499033#comment-14499033 ] Wangda Tan commented on YARN-2740: -- Thanks for your quick update [~Naganarasimha]. bq. Actually dint get completely your opinion about throw exception when distributedConfiguration enabled for removeClusterNodeLabels; Did you want to throw ? I think it's better now prevent admin remove clusterNodeLabel when distributed enabled. bq. if you want to allow admin to remove ClusterNodeLabels then there is one case which i can see potential problem: Assume NM informs RM with a valid node label x through HB/Resgistration and then admin removes x from cluster node labels. But this is not communicated back to NM and NM will not send labels as part of HB unless there is change in labels in NM side. So NM is not aware of x being removed at all. I agree we need to allow Admin to control valid partitions but in that case we need to add some logic in RM to request NM to resubmit labels. Please provide ur views. I think it's not a big problem, NM doesn't need to know x being removed, the logic should be, NM reports label, and RM allocate according to label, NM should just move on if adding label failed as what we done in YARN-2495. My opinion here is not add extra RM-NM communicate. I will review patch again when we decided above one. ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch, YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch, YARN-2740.20150417-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499099#comment-14499099
]
Hadoop QA commented on YARN-2740:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726016/YARN-2740.20150417-1.patch
against trunk revision 4308910.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7369//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7369//console
This message is automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch,
YARN-2740.20150417-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14490406#comment-14490406
]
Hadoop QA commented on YARN-2740:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12724645/YARN-2740.20150411-2.patch
against trunk revision 7660da9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7305//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7305//console
This message is automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14490575#comment-14490575
]
Hadoop QA commented on YARN-2740:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12724689/YARN-2740.20150411-3.patch
against trunk revision 2cc9514.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7308//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7308//console
This message is automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch, YARN-2740.20150411-1.patch,
YARN-2740.20150411-2.patch, YARN-2740.20150411-3.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392066#comment-14392066
]
Naganarasimha G R commented on YARN-2740:
-
Thanks for the review [~wangda],
bq. Beyond CommonNodeLabelsManager shouldn't persist labels on nodes when NM
do heartbeat., it shouldn't recover labels on nodes when RM restart. This is
because RM configured centralized config, add some labels to nodes and change
config to distributed then restart.
Good catch !. This i can achieve in couple of ways
* Modify {{NodeLabelsStore.recover()}} to accept a boolean parameter like
{{boolean skipNodeToLabelsMappings}} and leave the responsibility to the store
(FileSystemNodeLabelsStore need to take care of skipping)
* Add a method in CommonNodeLabelsManager like {{recoverLabelsOnNode}} and let
the store use this instead of {{replaceLabelsOnNode}} and we can handle the
skipping in the new method i.e.
{{CommonNodeLabelsManager.recoverLabelsOnNode}}. If needed to further ensure
that NodeLabelsStore do not call replaceLabelsOnNode we can extract a interface
for the methods used by the NodeLabelsStore and make CommonNodeLabelsManager
implement it.
Please provide your opinion on the suggested approaches and also if you have
any other alternatives in mind.
2nd point will handle in the next patch
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391106#comment-14391106 ] Wangda Tan commented on YARN-2740: -- [~Naganarasimha], Took a look, some comments: 1) Beyond CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat., it shouldn't recover labels on nodes when RM restart. This is because RM configured centralized config, add some labels to nodes and change config to distributed then restart. Need add tests to make sure RM restart will not recover labels on nodes when distributed configuration enabled. 2) isDistributedConfiguration should be a part of CommonNodeLabelsManager so that you don't need to get from configuration it in RMAdminService/RMWebService ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[ https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389707#comment-14389707 ] Naganarasimha G R commented on YARN-2740: - Hi [~wangda], Seems like TestFairScheduler.testContinuousScheduling is intermittently failing and not related to the fix in this jira so can you review this jira further ? ResourceManager side should properly handle node label modifications when distributed node label configuration enabled -- Key: YARN-2740 URL: https://issues.apache.org/jira/browse/YARN-2740 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Wangda Tan Assignee: Naganarasimha G R Fix For: 2.8.0 Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch, YARN-2740.20150327-1.patch According to YARN-2495, when distributed node label configuration is enabled: - RMAdmin / REST API should reject change labels on node operations. - CommonNodeLabelsManager shouldn't persist labels on nodes when NM do heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2740) ResourceManager side should properly handle node label modifications when distributed node label configuration enabled
[
https://issues.apache.org/jira/browse/YARN-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387825#comment-14387825
]
Hadoop QA commented on YARN-2740:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707571/YARN-2740.20150327-1.patch
against trunk revision 1a495fb.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7158//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7158//console
This message is automatically generated.
ResourceManager side should properly handle node label modifications when
distributed node label configuration enabled
--
Key: YARN-2740
URL: https://issues.apache.org/jira/browse/YARN-2740
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Wangda Tan
Assignee: Naganarasimha G R
Fix For: 2.8.0
Attachments: YARN-2740-20141024-1.patch, YARN-2740.20150320-1.patch,
YARN-2740.20150327-1.patch
According to YARN-2495, when distributed node label configuration is enabled:
- RMAdmin / REST API should reject change labels on node operations.
- CommonNodeLabelsManager shouldn't persist labels on nodes when NM do
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
