[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799690#comment-15799690 ] Junping Du commented on YARN-3225: -- Sorry for missing this. I felt the same pain now in releasing 2.8.0. :( > New parameter or CLI for decommissioning node gracefully in RMAdmin CLI > --- > > Key: YARN-3225 > URL: https://issues.apache.org/jira/browse/YARN-3225 > Project: Hadoop YARN > Issue Type: Sub-task > Components: graceful >Reporter: Junping Du >Assignee: Devaraj K > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch, > YARN-3225-4.patch, YARN-3225-5.patch, YARN-3225.patch, YARN-914.patch > > > New CLI (or existing CLI with parameters) should put each node on > decommission list to decommissioning status and track timeout to terminate > the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447509#comment-15447509 ] Andrew Wang commented on YARN-3225: --- FYI for git log greppers that the JIRA ID is missing in the commit message. > New parameter or CLI for decommissioning node gracefully in RMAdmin CLI > --- > > Key: YARN-3225 > URL: https://issues.apache.org/jira/browse/YARN-3225 > Project: Hadoop YARN > Issue Type: Sub-task > Components: graceful >Reporter: Junping Du >Assignee: Devaraj K > Fix For: 2.8.0 > > Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch, > YARN-3225-4.patch, YARN-3225-5.patch, YARN-3225.patch, YARN-914.patch > > > New CLI (or existing CLI with parameters) should put each node on > decommission list to decommissioning status and track timeout to terminate > the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507385#comment-14507385 ] Rohith commented on YARN-3225: -- +1(non-binding) LGTM. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch, YARN-3225-4.patch, YARN-3225-5.patch, YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507123#comment-14507123
]
Hadoop QA commented on YARN-3225:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 54s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 2 line(s) that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 46s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 46s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 28s | The applied patch generated
18 additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 48s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 25s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 7m 17s | Tests passed in
hadoop-yarn-client. |
| {color:green}+1{color} | yarn tests | 2m 1s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 55m 49s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 110m 44s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12727006/YARN-3225-5.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b08908a |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7446/artifact/patchprocess/whitespace.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7446/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7446/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-client test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7446/artifact/patchprocess/testrun_hadoop-yarn-client.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7446/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7446/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7446/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7446//console |
This message was automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225-4.patch, YARN-3225-5.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14507181#comment-14507181 ] Junping Du commented on YARN-3225: -- v5 patch LGTM. The new Jenkins result just report some trivial format issue but failed to report details to improve it. +1. I will go ahead to commit it shortly. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch, YARN-3225-4.patch, YARN-3225-5.patch, YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494023#comment-14494023
]
Hadoop QA commented on YARN-3225:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12724185/YARN-3225-4.patch
against trunk revision b5a0b24.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7328//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7328//console
This message is automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225-4.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494377#comment-14494377
]
Rohith commented on YARN-3225:
--
Thanks [~devaraj.k] for patch and [~djp] for review.
Overall approach and patch looks good to me. One suggestion on the patch is
DecommissionType#FORCEFUL java doc is bit confusing. {{Forceful decommissioning
of nodes}} Since FORCEFUL type is applied only on the nodes which are
*decommissioning* state, RUNNING state nodes can not be forcefully
decommissioned.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225-4.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14490809#comment-14490809
]
Devaraj K commented on YARN-3225:
-
Thanks [~djp] for review.
This below test failure is also not related to the patch.
{code:xml}
org.apache.hadoop.yarn.client.api.impl.TestYarnClient
{code}
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225-4.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487146#comment-14487146
]
Hadoop QA commented on YARN-3225:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12724185/YARN-3225-4.patch
against trunk revision 6495940.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.client.api.impl.TestTimelineClient
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7268//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7268//console
This message is automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225-4.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485071#comment-14485071 ] Devaraj K commented on YARN-3225: - Thanks [~djp] for your review. bq. I think we should support a case that Admin want node get decommissioned whenever all apps on these node get finished. If so, shall we support nigative value (anyone or some special one, like: -1) to specify this case? If the user wants to achieve this, they can give some larger timeout value and wait for all nodes to get decommissioned gracefully(without forceful). Do we really need to provide special handling for this case? bq. For NORMAL, shall we use Decommission nodes in normal (old) way instead or something simpler- Decommission nodes? I feel Decommission nodes in normal way would be ok, no need to mention the 'old' term. What is your opinion on this? bq. IMO, the methods inside a class should't be more public than class itself? If we don't expect other projects to use class, we alwasy don't expect some methods get used. The same problem happen in an old API RefreshNodeRequest.java. I think we may need to fix both? I agree, I will fix both of them. bq. Why do we need this change? recordFactory.newRecordInstance(RefreshNodesRequest.class) will return something with DecommissionType.NORMAL as default. No? It will not give any difference because the NORMAL is the default. I made this change to make it consistent with other decommission types. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch, YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485199#comment-14485199 ] Junping Du commented on YARN-3225: -- Thanks [~devaraj.k] for replying. bq. If the user wants to achieve this, they can give some larger timeout value and wait for all nodes to get decommissioned gracefully(without forceful). Do we really need to provide special handling for this case? It would be great if we can support this case because users doesn't have to think out a large number for an important job and doesn't known when to end. Given this is a trivial effort comparing what you already achieved, we'd better do here instead of filing a separated JIRA. What do you think? bq. I feel Decommission nodes in normal way would be ok, no need to mention the 'old' term. What is your opinion on this? Yes. That sounds good. My previous point is not to mention decommissioning for normal/previous decommission process to get rid of any confusing. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch, YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482298#comment-14482298
]
Junping Du commented on YARN-3225:
--
Thanks [~devaraj.k] for the patch! The latest patch looks good in overall.
Some minor comments:
{code}
+ private long validateTimeout(String strTimeout) {
+long timeout;
+try {
+ timeout = Long.parseLong(strTimeout);
+} catch (NumberFormatException ex) {
+ throw new IllegalArgumentException(INVALID_TIMEOUT_ERR_MSG + strTimeout);
+}
+if (timeout 0) {
+ throw new IllegalArgumentException(INVALID_TIMEOUT_ERR_MSG + timeout);
+}
+return timeout;
+ }
{code}
I think we should support a case that Admin want node get decommissioned
whenever all apps on these node get finished. If so, shall we support nigative
value (anyone or some special one, like: -1) to specify this case?
javadoc in DecommissionType.java
{code}
+ /** Decomissioning nodes **/
+ NORMAL,
+
+ /** Graceful decommissioning of nodes **/
+ GRACEFUL,
+
+ /** Forceful decommissioning of nodes **/
+ FORCEFUL
{code}
For NORMAL, shall we use Decommission nodes in normal (old) way instead or
something simpler- Decommission nodes?
{code}
+@Private
+@Unstable
+public abstract class CheckForDecommissioningNodesRequest {
+ @Public
+ @Unstable
+ public static CheckForDecommissioningNodesRequest newInstance() {
+CheckForDecommissioningNodesRequest request = Records
+.newRecord(CheckForDecommissioningNodesRequest.class);
+return request;
+ }
+}
{code}
IMO, the methods inside a class should't be more public than class itself? If
we don't expect other projects to use class, we alwasy don't expect some
methods get used. The same problem happen in an old API
RefreshNodeRequest.java. I think we may need to fix both?
{code}
@Test
public void testRefreshNodes() throws Exception {
resourceManager.getClientRMService();
-RefreshNodesRequest request = recordFactory
-.newRecordInstance(RefreshNodesRequest.class);
+RefreshNodesRequest request = RefreshNodesRequest
+.newInstance(DecommissionType.NORMAL);
RefreshNodesResponse response = client.refreshNodes(request);
assertNotNull(response);
}
{code}
Why do we need this change?
recordFactory.newRecordInstance(RefreshNodesRequest.class) will return
something with DecommissionType.NORMAL as default. No?
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390379#comment-14390379
]
Hadoop QA commented on YARN-3225:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708647/YARN-3225-3.patch
against trunk revision c69ba81.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7188//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7188//console
This message is automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390670#comment-14390670
]
Devaraj K commented on YARN-3225:
-
This failed test is not related to the patch.
{code:xml}
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
{code}
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225-3.patch,
YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389328#comment-14389328
]
Hadoop QA commented on YARN-3225:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708492/YARN-3225-2.patch
against trunk revision 79f7f2a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7176//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/7176//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7176//console
This message is automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225-2.patch, YARN-3225.patch,
YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379860#comment-14379860
]
Junping Du commented on YARN-3225:
--
Thanks [~devaraj.k] for updating the patch!
Sorry for coming late for some major comments:
In ResourceManagerAdministrationProtocol.java,
{code}
+
+ public RefreshNodesGracefullyResponse refreshNodesGracefully(
+ RefreshNodesGracefullyRequest refreshNodesGracefullyRequest)
+ throws YarnException, IOException;
+
+ @Public
+ @Evolving
+ @Idempotent
+ public RefreshNodesForcefullyResponse refreshNodesForcefully(
+ RefreshNodesForcefullyRequest refreshNodesForcefullyRequest)
+ throws YarnException, IOException;
{code}
I think we don't have to add a new APIs here but can reuse existing
refreshNodes(), we can add additional optional field (like boolean value) to
RefreshNodesRequest to differentiate decommission immediately or with delay
(gracefully). There should be no difference for decommission forcelly and
previous decommission as there should be no side effect to decommission a
decommissioned node (API marked with Idempotent). That could keep API much
simpler.
{code}
+ public CheckForDecommissioningNodesResponse checkForDecommissioningNodes(
+ CheckForDecommissioningNodesRequest checkForDecommissioningNodesRequest)
+ throws YarnException, IOException;
{code}
May be it is better to add getDecommissioningNodes() to return a list of
decommissioning nodes instead of returning a boolean value here? We can print
it out the decommissioning nodes that haven't finished (or a subset of them if
large size) when hitting timeout at the end. That could be helpful for Admin to
understand things going on there.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380102#comment-14380102 ] Junping Du commented on YARN-3225: -- Thanks for quickly response, [~devaraj.k]! bq. If we make the refreshNodesForcefully as same as refreshNodes() API, there could be a chance of becoming node state incosistent if the hosts file updated with the newnodes or removal of existing nodes after the refreshNodesGracefully and before the refreshNodesForcefully. +1. That's good point. bq. I feel we need to have difference that refreshNodes()/refreshNodesGracefully() should read the hosts file and refreshNodesForcefully() would not read the hosts file and only move the node state from DECOMMISSIONING to DECOMMISSIONED. Agree that this sounds better. However, as my suggestion above, we can still keep the same Admin API. In AdminService side, we can handle request (not boolean value now, could be a new enum :)) in three cases separately. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225-1.patch, YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14380040#comment-14380040 ] Devaraj K commented on YARN-3225: - Thanks [~djp] for your comments and review. bq. There should be no difference for decommission forcelly and previous decommission as there should be no side effect to decommission a decommissioned node (API marked with Idempotent) If we make the refreshNodesForcefully as same as refreshNodes() API, there could be a chance of becoming node state incosistent if the hosts file updated with the newnodes or removal of existing nodes after the refreshNodesGracefully and before the refreshNodesForcefully. I feel we need to have difference that refreshNodes()/refreshNodesGracefully() should read the hosts file and refreshNodesForcefully() would not read the hosts file and only move the node state from DECOMMISSIONING to DECOMMISSIONED. bq. May be it is better to add getDecommissioningNodes() to return a list of decommissioning nodes instead of returning a boolean value here? We can print it out the decommissioning nodes that haven't finished (or a subset of them if large size) when hitting timeout at the end. It would be good idea to show the decommissioning nodes, I will include this in the new patch. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225-1.patch, YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377372#comment-14377372
]
Devaraj K commented on YARN-3225:
-
{code:xml}
org.apache.hadoop.yarn.server.resourcemanager.TestRM
{code}
This test failure is not related to the patch.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375763#comment-14375763
]
Hadoop QA commented on YARN-3225:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706513/YARN-3225-1.patch
against trunk revision 0b9f12c.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7072//console
This message is automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376444#comment-14376444
]
Hadoop QA commented on YARN-3225:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706513/YARN-3225-1.patch
against trunk revision 7e6f384.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRM
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7077//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7077//console
This message is automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225-1.patch, YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370370#comment-14370370 ] Junping Du commented on YARN-3225: -- bq. I feel timeout would be enough, anyway we can wait for some other to comment or suggest here. I am fine for keeping timeout here for now. We can discuss the naming issue on other JIRAs later. Shouldn't block the major feature here. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370649#comment-14370649 ] Devaraj K commented on YARN-3225: - ok sure, I will update the patch soon, Thanks. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369690#comment-14369690 ] Junping Du commented on YARN-3225: -- bq. I would prefer to have the name in sync with the client side terminology as we are specifying the client side value as timeout. +1. bq. If we change the event to DECOMMISSION_WITH_DELAY, do you agree to change the client side name from timeout to delay? Hmmm. How about we change both with DECOMMISSION_DELAY_WITH_TIMEOUT? :) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369801#comment-14369801 ] Devaraj K commented on YARN-3225: - I am thinking if we use the term 'delay', the users may assume/think that decommissioning would happen after certain amount of time as specified but here we are trying to achieve this as early as possible and within the max time as we specify. I feel timeout would be enough, anyway we can wait for some other to comment or suggest here. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369616#comment-14369616 ] Devaraj K commented on YARN-3225: - Either way would be ok and both give relevant meaning. I would prefer to have the name in sync with the client side terminology as we are specifying the client side value as timeout. If we change the event to DECOMMISSION_WITH_DELAY, do you agree to change the client side name from timeout to delay? New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360686#comment-14360686 ] Junping Du commented on YARN-3225: -- bq. If we have a constraint that we should issue graceful decommission command from only one RMAdmin CLI then this issue will not be a problem. Can we have this assumption in our first phase (target to catch up 2.8)? IMO, Decommissioning nodes is very restrictive operation that we don't expect multiple could happen at the same time on a cluster. We can improve later if we think this is not good enough. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361089#comment-14361089
]
Junping Du commented on YARN-3225:
--
One additional comments:
For RMNodeEventType, DECOMMISSION_WITH_DELAY sounds better?
{code}
RMNodeEventType.java
@@ -24,6 +24,7 @@
// Source: AdminService
DECOMMISSION,
+ DECOMMISSION_WITH_TIMEOUT,
{code}
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360591#comment-14360591 ] Devaraj K commented on YARN-3225: - I was describing in my previous comment about having two RMAdmin CLI's for increasing the timeout value, one CLI runs with the timeout(say ‘x’) value and continues waiting for timeout, during this time another CLI issues the command with the higher timeout(x). If we keep the CLI(x timeout) running it would issue the forceful decommission with x timeout and new CLI timeout(x) will not reflect. If we have a constraint that we should issue graceful decommission command from only one RMAdmin CLI then this issue will not be a problem. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359239#comment-14359239 ] Junping Du commented on YARN-3225: -- Nice discussion, [~devaraj.k]! bq. If there are some long running containers in the NM and RMAdmin CLI gets terminated before issuing forceful decommission then the NM could in the “DECOMMISSIONING” state irrespective of timeout. AM I missing anything? If users terminate the blocking/pending CLI, then it only means they want to track timeout themselves or they want to adjust timeout value ahead or delay. In this case, the decommissioning nodes either get decommissioned when app finished (a clean quit), or wait user to decommission again later. We can add some alert messages later if some nodes are in decommissioning stage for really long time. The basic idea is we agree to not tracking timeout in RM side for each individual nodes. bq. If we don't pass timeout to RM then how are we going to achieve this? You mean this will be handled later, once the basic things are done. You are right that timeout value could be useful to pass down to AM for preemption containers (however, no any effect on terminating nodes). Let's keep it here and we can leverage it later when we are notifying AM. bq. For making timeout longer, if we use new CLI then there is a chance of forceful decommission happening with the old CLI timeout. Is there any constraint like this needs to be done with the same CLI? Not quite understanding the case described here. Users should terminate the current CLI and launch a new CLI for adjusted timeout values if they want to wait shorter or longer. If it already passed previous timeout values, current CLI should quit already with all nodes decommissioned. Am I missing something here? New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14358460#comment-14358460 ] Devaraj K commented on YARN-3225: - Thanks [~djp] for explanation. If there are some long running containers in the NM and RMAdmin CLI gets terminated before issuing forceful decommission then the NM could in the “DECOMMISSIONING” state irrespective of timeout. AM I missing anything? bq. AMs who has running containers on this NM will get noticed with preemption framework (with timeout), especially AM itself run against on this node. If we don't pass timeout to RM then how are we going to achieve this? You mean this will be handled later, once the basic things are done. bq. the timeout can be updated to shorter or longer. With RMAdmin CLI handling timeout, If we want to make the timeout shorter then we can issue the command in new/same CLI with the shorter timeout, it would be fine. For making timeout longer, if we use new CLI then there is a chance of forceful decommission happening with the old CLI timeout. Is there any constraint like this needs to be done with the same CLI? New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356774#comment-14356774 ] Junping Du commented on YARN-3225: -- bq. Here what would happen to the decommissioning node if the RMAdmin issued refreshNodeGracefully() and gets terminated(exited) before issuing the 'refreshNode forcefully'? This can be done by doing Ctrl+C on the command prompt. The Node will be in decommissioning state forever and becomes unusable for new containers allocation. From v3 version of proposal in umbrella JIRA, If CLI get interrupted, then it won’t keep track of timeout to forcefully decommissioned left nodes. However, nodes in “DECOMMISSIONING” will still get terminated later (after running apps get finished) except admin call recommission CLI on these nodes explicitly. The node in decommissioning state are terminated for 2 trigger events, one is timeout and the other is all application (on that node) get finished (will be covered in YARN-3212). We can document what's that means if user do Ctrl +C to gracefully decommissioning CLI. If application (like LRS) never ends in this situation, then user need to refresh these node forcefully (or gracefully with timeout but not interrupt). Make sense? New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356255#comment-14356255
]
Junping Du commented on YARN-3225:
--
Thanks [~devaraj.k]. Sorry for missing an important comment on current patch:
From many discussions in umbrella JIRA (YARN-914), most of us prefer not to
pass timeout value to YARN RM side but make it proper handled in RMAdmin side
which make things much simpler (please check discussion there for more
details). So the CLI with -g option could be a blocked CLI until all nodes get
decommissioned or timeout.
Psudo logic in this CLI should be something like:
{code}
refreshNodeGracefully(); - mark node to decommissionning
while (!timeout some nodes still in decommissioning) {
checkStatusForDecommissioningNodes();
}
if (timeout) {
refreshNode forcefully for reminding nodes
}
{code}
Thoughts?
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-3225.patch, YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356331#comment-14356331 ] Devaraj K commented on YARN-3225: - bq. not to pass timeout value to YARN RM side but make it proper handled in RMAdmin side which make things much simpler Here what would happen to the decommissioning node if the RMAdmin issued refreshNodeGracefully() and gets terminated(exited) before issuing the 'refreshNode forcefully'? This can be done by doing Ctrl+C on the command prompt. The Node will be in decommissioning state forever and becomes unusable for new containers allocation. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K Attachments: YARN-3225.patch, YARN-914.patch New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352944#comment-14352944
]
Junping Du commented on YARN-3225:
--
Thanks [~devaraj.k] for delivering the patch which is the first one in graceful
decommission effort!
A couple of comments:
In RefreshNodesRequestPBImpl.java,
{code}
@Override
+ public long getTimeout() {
+return getProto().getTimeout();
+ }
+
+ @Override
+ public void setTimeout(long timeout) {
+builder.setTimeout(timeout);
+ }
{code}
The setTimeout() has problem because we didn't set viaProto to false, so if we
getTimeout() afterwards then it will return the old value from old proto.
Suggest to add a method of maybeInitBuilder() just like other PBImpls, also add
a unit test to verify the PBImpl works as expected.
In NodeState.java,
{code}
DECOMMISSION_IN_PROGRESS
{code}
[~jlowe] suggested in umbrella JIRA that it is better to be DECOMMISSIONING. I
had the same feeling so reflect the name in latest proposal. Do you think we
should incorporate that comments here?
In RMAdminCLI.java,
{code}
+ .put(-refreshNodes, new UsageInfo([-g [timeout in ms]],
Refresh the hosts information at the ResourceManager.))
{code}
I think we should add more info to description message - Refresh the hosts
information at the ResourceManager. to explain -g option doing. Isn't it?
Also, per my suggestion above, it is better to specify seconds in timeout. MS
is more precisely, but get more chance for wrong (manually) operation.
Also, it is better to change the patch name to be consist with JIRA number
(YARN-3225).
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352766#comment-14352766
]
Hadoop QA commented on YARN-3225:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703395/YARN-914.patch
against trunk revision 5578e22.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.api.TestPBImplRecords
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestNodesPage
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6893//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6893//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6893//console
This message is automatically generated.
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
Attachments: YARN-914.patch
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327941#comment-14327941 ] Devaraj K commented on YARN-3225: - What would be the timeout units here, are we thinking of any constrained range for timeout value? Thanks New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328629#comment-14328629 ] Devaraj K commented on YARN-3225: - Thanks [~djp] for clarification. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[
https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328627#comment-14328627
]
Devaraj K commented on YARN-3225:
-
I see the same mentioned in the design doc
https://issues.apache.org/jira/secure/attachment/12699496/GracefullyDecommissionofNodeManagerv3.pdf
{quote} Before NMs get decommissioned, the timeout can be updated to shorter or
longer. e.g. admin can terminate the CLI and resubmit it with a different
timeout
value.{quote}
New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
---
Key: YARN-3225
URL: https://issues.apache.org/jira/browse/YARN-3225
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Junping Du
Assignee: Devaraj K
New CLI (or existing CLI with parameters) should put each node on
decommission list to decommissioning status and track timeout to terminate
the nodes that haven't get finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328631#comment-14328631 ] Sunil G commented on YARN-3225: --- Yes [~devaraj.k]. Thank you for the clarification. New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327662#comment-14327662 ] Sunil G commented on YARN-3225: --- Hi [~djp] To understand the idea correctly, do you mean a command is to be added so that a given node can be made as decommissioned. And it can be given a timeout to gracefully verify the same is done. So something like ./yarn -node nodeID -timeout 200 -decommission Pls help to clarify, and I would like pursue this if you are not assigning yourself. :) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327680#comment-14327680 ] Sunil G commented on YARN-3225: --- Sorry, I slightly misunderstood earlier. You meant rmadmin command with a new option such as -g. So one doubt here, can timeout also be passed here? New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14327807#comment-14327807 ] Junping Du commented on YARN-3225: -- Thanks [~sunilg] for the comments! Yes. I mean mradmin command line. I think it could be better to pass a timeout with adding a parameter something like -t. Without this parameter, it will decommission node forcefully just like the old. Thoughts? New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3225) New parameter or CLI for decommissioning node gracefully in RMAdmin CLI
[ https://issues.apache.org/jira/browse/YARN-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328437#comment-14328437 ] Sunil G commented on YARN-3225: --- Another point is, suppose if we fire same command with different time units immediately. And if first timeout is still ongoing, do we need to update timeout? New parameter or CLI for decommissioning node gracefully in RMAdmin CLI --- Key: YARN-3225 URL: https://issues.apache.org/jira/browse/YARN-3225 Project: Hadoop YARN Issue Type: Sub-task Reporter: Junping Du Assignee: Devaraj K New CLI (or existing CLI with parameters) should put each node on decommission list to decommissioning status and track timeout to terminate the nodes that haven't get finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
