[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296717#comment-14296717 ] Hudson commented on YARN-3079: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #88 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/88/]) YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) (wangda: rev 7882bc0f1433ae73361cab4207eb0c15abee4586) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/CHANGES.txt Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Fix For: 2.7.0 Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296728#comment-14296728 ] Hudson commented on YARN-3079: -- FAILURE: Integrated in Hadoop-Yarn-trunk #822 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/822/]) YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) (wangda: rev 7882bc0f1433ae73361cab4207eb0c15abee4586) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java * hadoop-yarn-project/CHANGES.txt Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Fix For: 2.7.0 Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14297000#comment-14297000 ] Hudson commented on YARN-3079: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2039 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2039/]) YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) (wangda: rev 7882bc0f1433ae73361cab4207eb0c15abee4586) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Fix For: 2.7.0 Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296803#comment-14296803 ] Hudson commented on YARN-3079: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #89 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/89/]) YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) (wangda: rev 7882bc0f1433ae73361cab4207eb0c15abee4586) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Fix For: 2.7.0 Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296927#comment-14296927 ] Hudson commented on YARN-3079: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2020 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2020/]) YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) (wangda: rev 7882bc0f1433ae73361cab4207eb0c15abee4586) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Fix For: 2.7.0 Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296893#comment-14296893 ] Hudson commented on YARN-3079: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #85 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/85/]) YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) (wangda: rev 7882bc0f1433ae73361cab4207eb0c15abee4586) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Fix For: 2.7.0 Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296077#comment-14296077 ] zhihai xu commented on YARN-3079: - Hi [~rkanter], thanks for the review. I found out since we can reuse the updateMaximumAllocation in updateNodeResource, we don't need create a separate function refreshMaximumAllocation. So I removed refreshMaximumAllocation and move all the code in refreshMaximumAllocation to updateMaximumAllocation which is the same as the original code. The change will also address your concern. Please review it to see whether it looks good to you. thanks zhihai Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296108#comment-14296108 ] Robert Kanter commented on YARN-3079: - Sounds good. +1 Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296197#comment-14296197
]
Hadoop QA commented on YARN-3079:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12695123/YARN-3079.004.patch
against trunk revision d244574.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6446//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6446//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch,
YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296005#comment-14296005 ] Wangda Tan commented on YARN-3079: -- [~rkanter], wanna take a look? I plan to commit this today or tomorrow. Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296006#comment-14296006 ] Robert Kanter commented on YARN-3079: - I'm actually looking at it right now :) Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296018#comment-14296018
]
Robert Kanter commented on YARN-3079:
-
I think we should acquire {{maxAllocWriteLock}} on entering
{{refreshMaximumAllocation()}} and release it on exiting. I could easily see
someone in a future JIRA calling {{refreshMaximumAllocation()}} without
realizing that they need to acquire {{maxAllocWriteLock}} first.
{{maxAllocWriteLock}} is reentrant, so there should be no harm is re-acquiring
it.
+1 after that
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch,
YARN-3079.002.patch, YARN-3079.003.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296228#comment-14296228 ] zhihai xu commented on YARN-3079: - Hi [~leftnoteasy], All these tests(TestFSRMStateStore, TestFSRMStateStore and TestRMDelegationTokens) are passed in my local latest build with my patch. So these failed tests are not related to my patch. thanks zhihai Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296214#comment-14296214 ] Wangda Tan commented on YARN-3079: -- [~zxu], could you verify if failed tests are not related? Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296383#comment-14296383
]
Hadoop QA commented on YARN-3079:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12695175/YARN-3079.004.patch
against trunk revision 5a0051f.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6451//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6451//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch,
YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296414#comment-14296414 ] Wangda Tan commented on YARN-3079: -- Good, committing.. Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296427#comment-14296427 ] Hudson commented on YARN-3079: -- FAILURE: Integrated in Hadoop-trunk-Commit #6956 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6956/]) YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource. (Zhihai Xu via wangda) (wangda: rev 7882bc0f1433ae73361cab4207eb0c15abee4586) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Fix For: 2.7.0 Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch, YARN-3079.004.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294060#comment-14294060 ] Anubhav Dhoot commented on YARN-3079: - LGTM Verified that the nodeUpdate is called from synchronized methods in both CapacityScheduler and FairScheduler so removing and adding the node should not cause any NPE. The only impact seems like we may miss the node in continuousScheduling when it gets removed, which should be fine. Added Tests look good Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294395#comment-14294395 ] Wangda Tan commented on YARN-3079: -- Thanks for updating, [~zxu]. LGTM, +1. Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294678#comment-14294678 ] zhihai xu commented on YARN-3079: - HI [~rkanter], Could you help review and commit the patch? thanks zhihai Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293108#comment-14293108 ] zhihai xu commented on YARN-3079: - thanks [~rchiang] and [~leftnoteasy]'s review. bq. IMHO, I think they're exchangeable, update a node = remove then a Yes, It make sense to keep code clean, since the node will only disappear for a very short time. AbstractYarnScheduler#getMaximumResourceCapability will be called in this period very rarely. Even it happen, the side effect will be very small. The upload a new patch YARN-3079.003.patch which addressed [~leftnoteasy]'s comment. Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch, YARN-3079.003.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14292729#comment-14292729 ] Wangda Tan commented on YARN-3079: -- [~zxu], Thanks for reply, bq. This is discussable. I prefer to keep the current signature beca Make sense bq. I think it is not completely equivalent . because when you call updateMaximumAllocation(oldNo IMHO, I think they're exchangeable, update a node = remove then add. Its state is discrete, so it is safe to make it disappear for very short time. I think it's very important to keep code clean. Beyond this, patch looks good. Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch, YARN-3079.002.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293141#comment-14293141
]
Hadoop QA commented on YARN-3079:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12694732/YARN-3079.003.patch
against trunk revision 6f9fe76.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-kms.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6431//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6431//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch,
YARN-3079.002.patch, YARN-3079.003.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290501#comment-14290501
]
Hadoop QA commented on YARN-3079:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12694338/YARN-3079.002.patch
against trunk revision 8f26d5a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6409//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6409//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6409//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch,
YARN-3079.002.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290042#comment-14290042 ] Anubhav Dhoot commented on YARN-3079: - Should we add a couple of more combinations to the test to ensure coverage? Update node2 where it increases resources but the increase is less than current max and see no change Update node2 where it decreases resources but the original and new are both less than the current max and see no change? Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. Also RMNodeReconnectEvent called from ResourceTrackerService#registerNodeManager will also trigger AbstractYarnScheduler#updateNodeResource being called. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290465#comment-14290465
]
Hadoop QA commented on YARN-3079:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12694331/YARN-3079.002.patch
against trunk revision 8f26d5a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6406//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6406//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6406//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch,
YARN-3079.002.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290444#comment-14290444
]
zhihai xu commented on YARN-3079:
-
thanks [~leftnoteasy] and [~adhoot]'s review,
addressed the comment from [~adhoot] in the new patch YARN-3079.002.patch.
about the comment from [~leftnoteasy],
bq. 1) Suggest to change signature of updateMaximumAllocation(SchedulerNode,
bool) to updateMaximumAllocation(Resource nodeResource, bool), since we only
uses nodeResource here.
This is discussable. I prefer to keep the current signature because the current
signature is more flexible and more meaningful for the other parameter(added
node or removed node). Two nodes can have same nodeResource and you can access
more information from SchedulerNode.
bq. 2) Change resource for a NM is equivalent to
{{updateMaximumAllocation(oldNodeResource, false)}} and
{{updateMaximumAllocation(newNoderesource, true)}}. We can avoid some
duplicated logic.
I think it is not completely equivalent . because when you call
{{updateMaximumAllocation(oldNodeResource, false)}}, you supposed the node is
already removed from the HashMap nodes based on both the implementation of
updateMaximumAllocation and caller of updateMaximumAllocation. But in the
context updateNodeResource, the node whose resource to be changed is still in
the HashMap nodes.
bq. 3) Suggest rename updateMaximumAllocation(void) to
refreshMaximumAllocation() or other name reflects the behavior: scan all
cluster nodes and get maximum allocation.
good suggestion. refreshMaximumAllocation is very good name. addressed this
comment in the new patch YARN-3079.002.patch.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch,
YARN-3079.002.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288324#comment-14288324
]
Wangda Tan commented on YARN-3079:
--
Hi [~zxu],
Thanks for taking up this task, just reviewed your patch, some comments
1) Suggest to change signature of updateMaximumAllocation(SchedulerNode, bool)
to updateMaximumAllocation(Resource nodeResource, bool), since we only uses
nodeResource here.
2) Change resource for a NM is equivalent to
{{updateMaximumAllocation(oldNodeResource, false)}} and
{{updateMaximumAllocation(newNoderesource, true)}}. We can avoid some
duplicated logic.
3) Suggest rename updateMaximumAllocation(void) to refreshMaximumAllocation()
or other name reflects the behavior: scan all cluster nodes and get maximum
allocation.
4) Not related to this fix -- I found only max allocation is protected by R/W
lock, it seems not correct to me, I think we should address it in a separated
JIRA. Will file a ticket later.
Wangda
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286037#comment-14286037
]
Hadoop QA commented on YARN-3079:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12693628/YARN-3079.001.patch
against trunk revision 4271554.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6376//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6376//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6376//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
Also RMNodeReconnectEvent called from
ResourceTrackerService#registerNodeManager will also trigger
AbstractYarnScheduler#updateNodeResource being called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285470#comment-14285470
]
Hadoop QA commented on YARN-3079:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12693547/YARN-3079.001.patch
against trunk revision 6b17eb9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6375//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6375//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6375//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[ https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285394#comment-14285394 ] zhihai xu commented on YARN-3079: - add testMaxAllocationAfterUpdateNodeResource in new patch YARN-3079.001.patch. Scheduler should also update maximumAllocation when updateNodeResource. --- Key: YARN-3079 URL: https://issues.apache.org/jira/browse/YARN-3079 Project: Hadoop YARN Issue Type: Bug Reporter: zhihai xu Assignee: zhihai xu Attachments: YARN-3079.000.patch, YARN-3079.001.patch Scheduler should also update maximumAllocation when updateNodeResource. Otherwise even the node resource is changed by AdminService#updateNodeResource, maximumAllocation won't be changed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3079) Scheduler should also update maximumAllocation when updateNodeResource.
[
https://issues.apache.org/jira/browse/YARN-3079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285425#comment-14285425
]
Hadoop QA commented on YARN-3079:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12693539/YARN-3079.000.patch
against trunk revision 6b17eb9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6374//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6374//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6374//console
This message is automatically generated.
Scheduler should also update maximumAllocation when updateNodeResource.
---
Key: YARN-3079
URL: https://issues.apache.org/jira/browse/YARN-3079
Project: Hadoop YARN
Issue Type: Bug
Reporter: zhihai xu
Assignee: zhihai xu
Attachments: YARN-3079.000.patch, YARN-3079.001.patch
Scheduler should also update maximumAllocation when updateNodeResource.
Otherwise even the node resource is changed by
AdminService#updateNodeResource, maximumAllocation won't be changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
