[jira] [Updated] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
[
https://issues.apache.org/jira/browse/YARN-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1078:
Attachment: YARN-1078.patch
Attach a patch. The fixes are quite straight forward.
> TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater
> fail on Windows
> -
>
> Key: YARN-1078
> URL: https://issues.apache.org/jira/browse/YARN-1078
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1078.patch
>
>
> The three unit tests fail on Windows due to host name resolution differences
> on Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
> {noformat}
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
> container_0__01_00 identifier is not valid for current Node manager.
> Expected : 127.0.0.1:12345 Found : localhost:12345
> {noformat}
> {noformat}
> testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
> Time elapsed: 8343 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[localhost]:12345> but
> was:<[127.0.0.1]:12345>
> at org.junit.Assert.assertEquals(Assert.java:125)
> at org.junit.Assert.assertEquals(Assert.java:147)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at $Proxy26.registerNodeManager(Unknown Source)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
> at
> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-1078) TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows
Chuan Liu created YARN-1078:
---
Summary: TestNodeManagerResync, TestNodeManagerShutdown, and
TestNodeStatusUpdater fail on Windows
Key: YARN-1078
URL: https://issues.apache.org/jira/browse/YARN-1078
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.0.0, 2.3.0
Reporter: Chuan Liu
Assignee: Chuan Liu
Priority: Minor
The three unit tests fail on Windows due to host name resolution differences on
Windows, i.e. 127.0.0.1 does not resolve to host name "localhost".
{noformat}
org.apache.hadoop.security.token.SecretManager$InvalidToken: Given Container
container_0__01_00 identifier is not valid for current Node manager.
Expected : 127.0.0.1:12345 Found : localhost:12345
{noformat}
{noformat}
testNMConnectionToRM(org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater)
Time elapsed: 8343 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<[localhost]:12345> but
was:<[127.0.0.1]:12345>
at org.junit.Assert.assertEquals(Assert.java:125)
at org.junit.Assert.assertEquals(Assert.java:147)
at
org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyResourceTracker6.registerNodeManager(TestNodeStatusUpdater.java:712)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
at $Proxy26.registerNodeManager(Unknown Source)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:212)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:149)
at
org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater$MyNodeStatusUpdater4.serviceStart(TestNodeStatusUpdater.java:369)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:213)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNMConnectionToRM(TestNodeStatusUpdater.java:985)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1077) TestContainerLaunch fails on Windows
[
https://issues.apache.org/jira/browse/YARN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1077:
Attachment: YARN-1077.2.patch
Attach a second patch that fixes the testInvalidEnvSyntaxDiagnostics at its
root cause instead of just a test fix.
> TestContainerLaunch fails on Windows
>
>
> Key: YARN-1077
> URL: https://issues.apache.org/jira/browse/YARN-1077
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1077.2.patch, YARN-1077.patch
>
>
> Several cases in this unit tests fail on Windows. (Append error log at the
> end.)
> testInvalidEnvSyntaxDiagnostics fails because the difference between cmd and
> bash script error handling. If some command fails in the cmd script, cmd will
> continue execute the the rest of the script command. Error handling needs to
> be explicitly carried out in the script file. The error code of the last
> command will be returned as the error code of the whole script. In this test,
> some error happened in the middle of the cmd script, the test expect an
> exception and non-zero error code. In the cmd script, the intermediate errors
> are ignored. The last command "call" succeeded and there is no exception.
> testContainerLaunchStdoutAndStderrDiagnostics fails due to wrong cmd commands
> used by the test.
> testContainerEnvVariables and testDelayedKill fail due to a regression from
> YARN-906.
> {noformat}
> ---
> Test set:
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch
> ---
> Tests run: 7, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 11.526 sec
> <<< FAILURE!
> testInvalidEnvSyntaxDiagnostics(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 583 sec <<< FAILURE!
> junit.framework.AssertionFailedError: Should catch exception
> at junit.framework.Assert.fail(Assert.java:50)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testInvalidEnvSyntaxDiagnostics(TestContainerLaunch.java:269)
> ...
> testContainerLaunchStdoutAndStderrDiagnostics(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 561 sec <<< FAILURE!
> junit.framework.AssertionFailedError: Should catch exception
> at junit.framework.Assert.fail(Assert.java:50)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testContainerLaunchStdoutAndStderrDiagnostics(TestContainerLaunch.java:314)
> ...
> testContainerEnvVariables(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 4136 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<137> but was:<143>
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:199)
> at junit.framework.Assert.assertEquals(Assert.java:205)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testContainerEnvVariables(TestContainerLaunch.java:500)
> ...
> testDelayedKill(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 2744 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<137> but was:<143>
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:199)
> at junit.framework.Assert.assertEquals(Assert.java:205)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testDelayedKill(TestContainerLaunch.java:601)
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1027) Implement RMHAServiceProtocol
[ https://issues.apache.org/jira/browse/YARN-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743547#comment-13743547 ] Bikas Saha commented on YARN-1027: -- [~vinodkv] Do you have any suggestions? Would be great to have them early because this approach will be important for the remaining changes. So best to spend time on this now and make sure we are in the best position for future development. > Implement RMHAServiceProtocol > - > > Key: YARN-1027 > URL: https://issues.apache.org/jira/browse/YARN-1027 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bikas Saha >Assignee: Karthik Kambatla > Attachments: yarn-1027-1.patch > > > Implement existing HAServiceProtocol from Hadoop common. This protocol is the > single point of interaction between the RM and HA clients/services. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1077) TestContainerLaunch fails on Windows
[
https://issues.apache.org/jira/browse/YARN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1077:
Description:
Several cases in this unit tests fail on Windows. (Append error log at the end.)
testInvalidEnvSyntaxDiagnostics fails because the difference between cmd and
bash script error handling. If some command fails in the cmd script, cmd will
continue execute the the rest of the script command. Error handling needs to be
explicitly carried out in the script file. The error code of the last command
will be returned as the error code of the whole script. In this test, some
error happened in the middle of the cmd script, the test expect an exception
and non-zero error code. In the cmd script, the intermediate errors are
ignored. The last command "call" succeeded and there is no exception.
testContainerLaunchStdoutAndStderrDiagnostics fails due to wrong cmd commands
used by the test.
testContainerEnvVariables and testDelayedKill fail due to a regression from
YARN-906.
{noformat}
---
Test set:
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch
---
Tests run: 7, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 11.526 sec <<<
FAILURE!
testInvalidEnvSyntaxDiagnostics(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
Time elapsed: 583 sec <<< FAILURE!
junit.framework.AssertionFailedError: Should catch exception
at junit.framework.Assert.fail(Assert.java:50)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testInvalidEnvSyntaxDiagnostics(TestContainerLaunch.java:269)
...
testContainerLaunchStdoutAndStderrDiagnostics(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
Time elapsed: 561 sec <<< FAILURE!
junit.framework.AssertionFailedError: Should catch exception
at junit.framework.Assert.fail(Assert.java:50)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testContainerLaunchStdoutAndStderrDiagnostics(TestContainerLaunch.java:314)
...
testContainerEnvVariables(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
Time elapsed: 4136 sec <<< FAILURE!
junit.framework.AssertionFailedError: expected:<137> but was:<143>
at junit.framework.Assert.fail(Assert.java:50)
at junit.framework.Assert.failNotEquals(Assert.java:287)
at junit.framework.Assert.assertEquals(Assert.java:67)
at junit.framework.Assert.assertEquals(Assert.java:199)
at junit.framework.Assert.assertEquals(Assert.java:205)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testContainerEnvVariables(TestContainerLaunch.java:500)
...
testDelayedKill(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
Time elapsed: 2744 sec <<< FAILURE!
junit.framework.AssertionFailedError: expected:<137> but was:<143>
at junit.framework.Assert.fail(Assert.java:50)
at junit.framework.Assert.failNotEquals(Assert.java:287)
at junit.framework.Assert.assertEquals(Assert.java:67)
at junit.framework.Assert.assertEquals(Assert.java:199)
at junit.framework.Assert.assertEquals(Assert.java:205)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testDelayedKill(TestContainerLaunch.java:601)
...
{noformat}
was:
Several cases in this unit tests fail on Windows.
> TestContainerLaunch fails on Windows
>
>
> Key: YARN-1077
> URL: https://issues.apache.org/jira/browse/YARN-1077
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1077.patch
>
>
> Several cases in this unit tests fail on Windows. (Append error log at the
> end.)
> testInvalidEnvSyntaxDiagnostics fails because the difference between cmd and
> bash script error handling. If some command fails in the cmd script, cmd will
> continue execute the the rest of the script command. Error handling needs to
> be explicitly carried out in the script file. The error code of the last
> command will be returned as the error code of the whole script. In this test,
> some error happened in the middle of the cmd script, the test expect an
> exception and non-zero error code. In the cmd script, the intermediate errors
> are ignored. The last command "call" succeeded and there is no exception.
> testContainerLaunchStdoutAndStderrDiagnostics fails due to w
[jira] [Updated] (YARN-1077) TestContainerLaunch fails on Windows
[
https://issues.apache.org/jira/browse/YARN-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated YARN-1077:
Attachment: YARN-1077.patch
> TestContainerLaunch fails on Windows
>
>
> Key: YARN-1077
> URL: https://issues.apache.org/jira/browse/YARN-1077
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Chuan Liu
>Assignee: Chuan Liu
>Priority: Minor
> Attachments: YARN-1077.patch
>
>
> Several cases in this unit tests fail on Windows. (Append error log at the
> end.)
> testInvalidEnvSyntaxDiagnostics fails because the difference between cmd and
> bash script error handling. If some command fails in the cmd script, cmd will
> continue execute the the rest of the script command. Error handling needs to
> be explicitly carried out in the script file. The error code of the last
> command will be returned as the error code of the whole script. In this test,
> some error happened in the middle of the cmd script, the test expect an
> exception and non-zero error code. In the cmd script, the intermediate errors
> are ignored. The last command "call" succeeded and there is no exception.
> testContainerLaunchStdoutAndStderrDiagnostics fails due to wrong cmd commands
> used by the test.
> testContainerEnvVariables and testDelayedKill fail due to a regression from
> YARN-906.
> {noformat}
> ---
> Test set:
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch
> ---
> Tests run: 7, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 11.526 sec
> <<< FAILURE!
> testInvalidEnvSyntaxDiagnostics(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 583 sec <<< FAILURE!
> junit.framework.AssertionFailedError: Should catch exception
> at junit.framework.Assert.fail(Assert.java:50)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testInvalidEnvSyntaxDiagnostics(TestContainerLaunch.java:269)
> ...
> testContainerLaunchStdoutAndStderrDiagnostics(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 561 sec <<< FAILURE!
> junit.framework.AssertionFailedError: Should catch exception
> at junit.framework.Assert.fail(Assert.java:50)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testContainerLaunchStdoutAndStderrDiagnostics(TestContainerLaunch.java:314)
> ...
> testContainerEnvVariables(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 4136 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<137> but was:<143>
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:199)
> at junit.framework.Assert.assertEquals(Assert.java:205)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testContainerEnvVariables(TestContainerLaunch.java:500)
> ...
> testDelayedKill(org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch)
> Time elapsed: 2744 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<137> but was:<143>
> at junit.framework.Assert.fail(Assert.java:50)
> at junit.framework.Assert.failNotEquals(Assert.java:287)
> at junit.framework.Assert.assertEquals(Assert.java:67)
> at junit.framework.Assert.assertEquals(Assert.java:199)
> at junit.framework.Assert.assertEquals(Assert.java:205)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch.testDelayedKill(TestContainerLaunch.java:601)
> ...
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-1077) TestContainerLaunch fails on Windows
Chuan Liu created YARN-1077: --- Summary: TestContainerLaunch fails on Windows Key: YARN-1077 URL: https://issues.apache.org/jira/browse/YARN-1077 Project: Hadoop YARN Issue Type: Bug Affects Versions: 3.0.0, 2.3.0 Reporter: Chuan Liu Assignee: Chuan Liu Priority: Minor Several cases in this unit tests fail on Windows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1074) Clean up YARN CLI app list to show only running apps.
[
https://issues.apache.org/jira/browse/YARN-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743510#comment-13743510
]
Xuan Gong commented on YARN-1074:
-
bq.Shall we have column for info like app start time? If so, we can build
filter on it so that we can list jobs running within 1 hour, 1 day, etc.
We can do that in the web service. We can provide a range of start time or
finished time, then We can filter based on the range.
Also, it is very easy to add the columns like start time or finished time,
because we do save those informations at RMApp.
> Clean up YARN CLI app list to show only running apps.
> -
>
> Key: YARN-1074
> URL: https://issues.apache.org/jira/browse/YARN-1074
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Reporter: Tassapol Athiapinya
>Assignee: Xuan Gong
> Attachments: YARN-1074.1.patch
>
>
> Once a user brings up YARN daemon, runs jobs, jobs will stay in output
> returned by $ yarn application -list even after jobs complete already. We
> want YARN command line to clean up this list. Specifically, we want to remove
> applications with FINISHED state(not Final-State) or KILLED state from the
> result.
> {code}
> [user1@host1 ~]$ yarn application -list
> Total Applications:150
> Application-IdApplication-Name
> Application-Type User Queue State
> Final-State ProgressTracking-URL
> application_1374638600275_0109 Sleep job
> MAPREDUCEuser1 default KILLED
> KILLED 100%host1:54059
> application_1374638600275_0121 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0121
> application_1374638600275_0020 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0020
> application_1374638600275_0038 Sleep job
> MAPREDUCEuser1 default
>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1074) Clean up YARN CLI app list to show only running apps.
[
https://issues.apache.org/jira/browse/YARN-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743498#comment-13743498
]
Junping Du commented on YARN-1074:
--
Shall we have column for info like app start time? If so, we can build filter
on it so that we can list jobs running within 1 hour, 1 day, etc.
btw, I saw TestUberAM timeout intermittent in some jenkins tests before. Do we
have JIRA to track this?
> Clean up YARN CLI app list to show only running apps.
> -
>
> Key: YARN-1074
> URL: https://issues.apache.org/jira/browse/YARN-1074
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Reporter: Tassapol Athiapinya
>Assignee: Xuan Gong
> Attachments: YARN-1074.1.patch
>
>
> Once a user brings up YARN daemon, runs jobs, jobs will stay in output
> returned by $ yarn application -list even after jobs complete already. We
> want YARN command line to clean up this list. Specifically, we want to remove
> applications with FINISHED state(not Final-State) or KILLED state from the
> result.
> {code}
> [user1@host1 ~]$ yarn application -list
> Total Applications:150
> Application-IdApplication-Name
> Application-Type User Queue State
> Final-State ProgressTracking-URL
> application_1374638600275_0109 Sleep job
> MAPREDUCEuser1 default KILLED
> KILLED 100%host1:54059
> application_1374638600275_0121 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0121
> application_1374638600275_0020 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0020
> application_1374638600275_0038 Sleep job
> MAPREDUCEuser1 default
>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1074) Clean up YARN CLI app list to show only running apps.
[
https://issues.apache.org/jira/browse/YARN-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated YARN-1074:
--
Issue Type: Improvement (was: New Feature)
> Clean up YARN CLI app list to show only running apps.
> -
>
> Key: YARN-1074
> URL: https://issues.apache.org/jira/browse/YARN-1074
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client
>Reporter: Tassapol Athiapinya
>Assignee: Xuan Gong
> Attachments: YARN-1074.1.patch
>
>
> Once a user brings up YARN daemon, runs jobs, jobs will stay in output
> returned by $ yarn application -list even after jobs complete already. We
> want YARN command line to clean up this list. Specifically, we want to remove
> applications with FINISHED state(not Final-State) or KILLED state from the
> result.
> {code}
> [user1@host1 ~]$ yarn application -list
> Total Applications:150
> Application-IdApplication-Name
> Application-Type User Queue State
> Final-State ProgressTracking-URL
> application_1374638600275_0109 Sleep job
> MAPREDUCEuser1 default KILLED
> KILLED 100%host1:54059
> application_1374638600275_0121 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0121
> application_1374638600275_0020 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0020
> application_1374638600275_0038 Sleep job
> MAPREDUCEuser1 default
>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-1074) Clean up YARN CLI app list to show only running apps.
[
https://issues.apache.org/jira/browse/YARN-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated YARN-1074:
--
Summary: Clean up YARN CLI app list to show only running apps. (was:
Provides command line to clean up application list)
Editing title from my understanding of the description.
+1 for showing only running apps by default. And adding filters for other
states. Will look at the patch.
> Clean up YARN CLI app list to show only running apps.
> -
>
> Key: YARN-1074
> URL: https://issues.apache.org/jira/browse/YARN-1074
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: client
>Reporter: Tassapol Athiapinya
>Assignee: Xuan Gong
> Attachments: YARN-1074.1.patch
>
>
> Once a user brings up YARN daemon, runs jobs, jobs will stay in output
> returned by $ yarn application -list even after jobs complete already. We
> want YARN command line to clean up this list. Specifically, we want to remove
> applications with FINISHED state(not Final-State) or KILLED state from the
> result.
> {code}
> [user1@host1 ~]$ yarn application -list
> Total Applications:150
> Application-IdApplication-Name
> Application-Type User Queue State
> Final-State ProgressTracking-URL
> application_1374638600275_0109 Sleep job
> MAPREDUCEuser1 default KILLED
> KILLED 100%host1:54059
> application_1374638600275_0121 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0121
> application_1374638600275_0020 Sleep job
> MAPREDUCEuser1 defaultFINISHED
> SUCCEEDED 100% host1:19888/jobhistory/job/job_1374638600275_0020
> application_1374638600275_0038 Sleep job
> MAPREDUCEuser1 default
>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-643) WHY appToken is removed both in BaseFinalTransition and AMUnregisteredTransition AND clientToken is removed in FinalTransition and not BaseFinalTransition
[ https://issues.apache.org/jira/browse/YARN-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743482#comment-13743482 ] Hudson commented on YARN-643: - SUCCESS: Integrated in Hadoop-trunk-Commit #4289 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4289/]) YARN-643. Fixed ResourceManager to remove all tokens consistently on app finish. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1515256) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestAMRMTokens.java > WHY appToken is removed both in BaseFinalTransition and > AMUnregisteredTransition AND clientToken is removed in FinalTransition and > not BaseFinalTransition > -- > > Key: YARN-643 > URL: https://issues.apache.org/jira/browse/YARN-643 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Xuan Gong > Fix For: 2.1.1-beta > > Attachments: YARN-643.1.patch, YARN-643.2.patch, YARN-643.3.patch, > YARN-643.4.patch, YARN-643.5.patch > > > The jira is tracking why appToken and clientToAMToken is removed separately, > and why they are distributed in different transitions, ideally there may be a > common place where these two tokens can be removed at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-643) WHY appToken is removed both in BaseFinalTransition and AMUnregisteredTransition AND clientToken is removed in FinalTransition and not BaseFinalTransition
[ https://issues.apache.org/jira/browse/YARN-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743478#comment-13743478 ] Vinod Kumar Vavilapalli commented on YARN-643: -- +1, looks good. Checking this in. > WHY appToken is removed both in BaseFinalTransition and > AMUnregisteredTransition AND clientToken is removed in FinalTransition and > not BaseFinalTransition > -- > > Key: YARN-643 > URL: https://issues.apache.org/jira/browse/YARN-643 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Xuan Gong > Attachments: YARN-643.1.patch, YARN-643.2.patch, YARN-643.3.patch, > YARN-643.4.patch, YARN-643.5.patch > > > The jira is tracking why appToken and clientToAMToken is removed separately, > and why they are distributed in different transitions, ideally there may be a > common place where these two tokens can be removed at the same time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-49) Improve distributed shell application to work on a secure cluster
[ https://issues.apache.org/jira/browse/YARN-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743477#comment-13743477 ] Vinod Kumar Vavilapalli commented on YARN-49: - No, the last we tested, it wasn't working yet because of some missing token propagation.. > Improve distributed shell application to work on a secure cluster > - > > Key: YARN-49 > URL: https://issues.apache.org/jira/browse/YARN-49 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications/distributed-shell >Reporter: Hitesh Shah >Assignee: Omkar Vinit Joshi > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-49) Improve distributed shell application to work on a secure cluster
[ https://issues.apache.org/jira/browse/YARN-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13743388#comment-13743388 ] Arun C Murthy commented on YARN-49: --- Can we close this then? > Improve distributed shell application to work on a secure cluster > - > > Key: YARN-49 > URL: https://issues.apache.org/jira/browse/YARN-49 > Project: Hadoop YARN > Issue Type: Sub-task > Components: applications/distributed-shell >Reporter: Hitesh Shah >Assignee: Omkar Vinit Joshi > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
