[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214824#comment-15214824 ] Jian He commented on YARN-4117: --- committed to branch-2.8 to > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Fix For: 2.8.0 > > Attachments: RunTestYarnClient.patch, YARN-4117.v0.patch, > YARN-4117.v1.patch, YARN-4117.v2.patch, YARN-4117.v3.patch, > YARN-4117.v4.patch, YARN-4117.v5.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214513#comment-15214513 ] Hudson commented on YARN-4117: -- FAILURE: Integrated in Hadoop-trunk-Commit #9512 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9512/]) Missing file for YARN-4117. (jianhe: rev 8831b18c65c8a61e400346dc308e068381835357) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMProxy.java > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Fix For: 2.8.0 > > Attachments: RunTestYarnClient.patch, YARN-4117.v0.patch, > YARN-4117.v1.patch, YARN-4117.v2.patch, YARN-4117.v3.patch, > YARN-4117.v4.patch, YARN-4117.v5.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214388#comment-15214388 ] Giovanni Matteo Fumarola commented on YARN-4117: Thanks [~jianhe] and [~subru] to help me on this. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Fix For: 2.8.0 > > Attachments: RunTestYarnClient.patch, YARN-4117.v0.patch, > YARN-4117.v1.patch, YARN-4117.v2.patch, YARN-4117.v3.patch, > YARN-4117.v4.patch, YARN-4117.v5.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213758#comment-15213758 ] Hudson commented on YARN-4117: -- FAILURE: Integrated in Hadoop-trunk-Commit #9507 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9507/]) YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy (jianhe: rev 55ae1439233e8585d624b2872e1e4753ef63eebb) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/DefaultRequestInterceptor.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/AMRMProxyService.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Fix For: 2.8.0 > > Attachments: RunTestYarnClient.patch, YARN-4117.v0.patch, > YARN-4117.v1.patch, YARN-4117.v2.patch, YARN-4117.v3.patch, > YARN-4117.v4.patch, YARN-4117.v5.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212861#comment-15212861
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 63m 18s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 63m 33s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 19s
{color} | {color:red} Patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 140m 26s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Failed junit tests | hadoop.yarn.client.TestGetGroups |
| JDK v1.8.0_74 Timed out junit tests |
org.apache.hadoop.yarn.client.cli.TestYarnCLI |
| | org.apache.hadoop.yarn.client.api.impl.TestYarnClient |
| | org.apache.hadoop.yarn.client.api.impl.TestAMRMClient |
| | org.apache.hadoop.yarn.client.api.impl.TestNMClient |
| JDK v1.7.0_95 Failed junit tests | hadoop.yarn.client.TestGetGroups |
| JDK v1.7.0_95 Timed out junit tests |
org.apache.hadoop.yarn.client.cli.TestYarnCLI |
| | org.apache.hadoop.yarn.client.api.impl.TestYarnClient |
| | org.apache.hadoop.yarn.client.api.impl.TestAMRMClient |
| |
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212644#comment-15212644
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
34s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 18s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 39s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 21s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 37s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new +
55 unchanged - 0 fixed = 58 total (was 55) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 23s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 56s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 26s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 57s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_74.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 22s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 12s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} |
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212283#comment-15212283 ] Giovanni Matteo Fumarola commented on YARN-4117: Attached v5 - Rebase of v4 > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch, > YARN-4117.v2.patch, YARN-4117.v3.patch, YARN-4117.v4.patch, YARN-4117.v5.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15211084#comment-15211084
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} YARN-4117 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12793613/YARN-4117.v4.patch |
| JIRA Issue | YARN-4117 |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/10869/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> End to end unit test with mini YARN cluster for AMRMProxy Service
> -
>
> Key: YARN-4117
> URL: https://issues.apache.org/jira/browse/YARN-4117
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
>Reporter: Kishore Chaliparambil
>Assignee: Giovanni Matteo Fumarola
> Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch,
> YARN-4117.v2.patch, YARN-4117.v3.patch, YARN-4117.v4.patch
>
>
> YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to
> end unit test using mini YARN cluster to the AMRMProxy service. This test
> will validate register, allocate and finish application and token renewal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210996#comment-15210996 ] Giovanni Matteo Fumarola commented on YARN-4117: All the tests run fine locally, included the new tests in this patch. I do not think the changes in [YARN-4686|https://issues.apache.org/jira/browse/YARN-4686] are relative to the TestYarnClient.createApp() failure. The problem should be a wrong configuration in Jenkins, that does not allow to execute that code and let it to timeout. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch, > YARN-4117.v2.patch, YARN-4117.v3.patch, YARN-4117.v4.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210882#comment-15210882 ] Jian He commented on YARN-4117: --- I triggered a new jenkins test run, lets see how it goes. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch, > YARN-4117.v2.patch, YARN-4117.v3.patch, YARN-4117.v4.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210867#comment-15210867
]
Eric Payne commented on YARN-4117:
--
[~giovanni.fumarola], thanks for bringing this to my attention. However, I
don't think the failures are due to the changes in YARN-4686. As [~ebadger]
points out in
[YARN-4686|https://issues.apache.org/jira/browse/YARN-4686?focusedCommentId=15169686=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15169686],
bq. TestYarnCLI, TestAMRMClient, TestYarnClient, TestNMClient, and
TestGetGroups are failing in multiple recent precommit builds YARN-4117,
YARN-4630, YARN-4676.
Also, when I run all the yarn tests locally, I don't see the above errors.
For {{TestGetGroups}}, I'm not seeing any errors.
For {{TestYarnClient}}, I see an error, but it's different than the ones seen
in this JIRA:
{noformat}
TestYarnClient.testReservationAPIs:1227
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
The request cannot be satisfied
{noformat}
> End to end unit test with mini YARN cluster for AMRMProxy Service
> -
>
> Key: YARN-4117
> URL: https://issues.apache.org/jira/browse/YARN-4117
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
>Reporter: Kishore Chaliparambil
>Assignee: Giovanni Matteo Fumarola
> Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch,
> YARN-4117.v2.patch, YARN-4117.v3.patch, YARN-4117.v4.patch
>
>
> YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to
> end unit test using mini YARN cluster to the AMRMProxy service. This test
> will validate register, allocate and finish application and token renewal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206975#comment-15206975 ] Giovanni Matteo Fumarola commented on YARN-4117: Hi [~jianhe], I was checking the test result and something weird showed up. [YARN-4117|https://issues.apache.org/jira/browse/YARN-4117] Results: Tests in error: TestAMRMProxy.testAMRMProxyE2E:110->*createApp*:368 ? test timed out after 6000... TestAMRMProxy.testE2ETokenSwap:273->*createApp*:368 ? test timed out after 6000... TestAMRMProxy.testE2ETokenRenewal:204->*createApp*:368 ? test timed out after 6... TestYarnClient.testShouldNotRetryForeverForNonNetworkExceptions:1468 ? test t... TestYarnClient.testAMMRTokens:902->*createApp*:954 ? test timed out after 3... TestGetGroups>GetGroupsTestBase.testMultipleNonExistingUsers:85->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testExistingInterleavedWithNonExistentUsers:95->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testNoUserGiven:53->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testNonExistentUser:76->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testExistingUser:61->GetGroupsTestBase.runTool:119 ? UnknownHost [YARN-4686|https://issues.apache.org/jira/browse/YARN-4686] (*Resolved*) Results: Tests in error: TestYarnClient.testShouldNotRetryForeverForNonNetworkExceptions:1486 ? test t... TestYarnClient.testAMMRTokens:902->*createApp*:954 ? test timed out after 3... TestGetGroups>GetGroupsTestBase.testMultipleNonExistingUsers:85->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testExistingInterleavedWithNonExistentUsers:95->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testNoUserGiven:53->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testNonExistentUser:76->GetGroupsTestBase.runTool:119 ? UnknownHost TestGetGroups>GetGroupsTestBase.testExistingUser:61->GetGroupsTestBase.runTool:119 ? UnknownHost TestYarnClient.testAMRMTokens fails in both scenarios when it tries to create an application – it timeouts because it is stuck. In TestAMRMProxy I used the same code to create an application. Thoughts? /cc [~eepayne] committer of [YARN-4686|https://issues.apache.org/jira/browse/YARN-4686] > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch, > YARN-4117.v2.patch, YARN-4117.v3.patch, YARN-4117.v4.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196637#comment-15196637
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
43s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 7s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 9s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 31s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new +
55 unchanged - 0 fixed = 58 total (was 55) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 16s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 43s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 36s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_74.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 38s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 45s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} |
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196490#comment-15196490
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 38s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 28s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 16s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 1s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 37s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 18s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 31s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new +
55 unchanged - 0 fixed = 58 total (was 55) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 59s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 13s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 51s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_74.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 55s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 10s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} |
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15191478#comment-15191478 ] Giovanni Matteo Fumarola commented on YARN-4117: Increase the timeout for the tests. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch, > YARN-4117.v2.patch, YARN-4117.v3.patch, YARN-4117.v4.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190543#comment-15190543
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 46s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 6s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 42s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 3s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 31s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new +
54 unchanged - 0 fixed = 57 total (was 54) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 9s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 7s {color} |
{color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 6s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_74.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 14s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 23s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} |
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189960#comment-15189960
]
Jian He commented on YARN-4117:
---
looks good to me, just some minor nits:
- I think these two methods are not needed, as it can just use the
amrmProxyEnabled flag directly
{code}
@Private
public boolean isAMRMProxyEnabled() {
return amrmProxyEnabled;
}
@Private
public void setAMRMProxyEnabled(boolean amrmProxyEnabled)
{
this.amrmProxyEnabled = amrmProxyEnabled;
}
{code}
- change the visibility to protected.
{code}
@Private
public void setAMRMProxyService(AMRMProxyService amrmProxyService) {
this.amrmProxyService = amrmProxyService;
}
{code}
> End to end unit test with mini YARN cluster for AMRMProxy Service
> -
>
> Key: YARN-4117
> URL: https://issues.apache.org/jira/browse/YARN-4117
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
>Reporter: Kishore Chaliparambil
>Assignee: Giovanni Matteo Fumarola
> Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch,
> YARN-4117.v2.patch
>
>
> YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to
> end unit test using mini YARN cluster to the AMRMProxy service. This test
> will validate register, allocate and finish application and token renewal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15185215#comment-15185215 ] Giovanni Matteo Fumarola commented on YARN-4117: I just ran all the tests that failed or timed out and they completed fine in my machine. My only concern is about the checkstyle. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch, > YARN-4117.v2.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184420#comment-15184420
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
59s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 47s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 52s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 21s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
48s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 46s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 27s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 34s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new +
55 unchanged - 0 fixed = 58 total (was 55) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 12s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 14s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_74.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 27s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_74.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 2s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 12s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} unit {color} |
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179070#comment-15179070 ] Jian He commented on YARN-4117: --- bq. MiniYarnCluster allocates the ports for the RM during the Start phase, and there was no way to pass the information to the AMRMProxy. I think this approach may not work for HA case where you will have multiple RM scheduler address. could you check how NM talks to RM ? I think NM have the same problem. we may let AMRMProxy follow the same method ? > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch, YARN-4117.v1.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170135#comment-15170135
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
35s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 8s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 14s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 13s
{color} | {color:red} hadoop-yarn-server-tests in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m 28s
{color} | {color:red} hadoop-yarn in the patch failed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 28s {color}
| {color:red} hadoop-yarn in the patch failed with JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m 35s
{color} | {color:red} hadoop-yarn in the patch failed with JDK v1.7.0_95.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 35s {color}
| {color:red} hadoop-yarn in the patch failed with JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m 15s
{color} | {color:red} hadoop-yarn-server-tests in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 12s
{color} | {color:red} hadoop-yarn-server-tests in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 12s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 13s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 4s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 39s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 14s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_95.
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15169591#comment-15169591 ] Giovanni Matteo Fumarola commented on YARN-4117: [~jianhe], thanks for reviewing the patch. 1. Sure; 2. Sure, I will check it. 3. Because I can customize CustomContainerManagerImpl and AMRMProxyService. MiniYarnCluster allocates the ports for the RM during the Start phase, and there was no way to pass the information to the AMRMProxy. For this reason I save the value of the Scheduler address and in the Customizazion I inform the AMRMProxy what is the address it will connect to. 4. The purpose of testE2ETokenSwap is to verified that the AMRMProxy swapped the security token and the AM cannot submit the job directly to the RM. I will update the patch according to the feedback. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168565#comment-15168565
]
Jian He commented on YARN-4117:
---
[~giovanni.fumarola], thanks for working on this, a few comments and questions :
1. could you add comments at the header of each test to explain what the test
is doing so that it can be understood more easily?
2. this test is based on timing, can we check that
AllocateResponse#getAMRMToken actually returns a different token than previous
one ?
{code}
for (int i = 0; i < 5; i++) {
client.allocate(request);
request.setResponseId(request.getResponseId() + 1);
// Time slot to be sure the RM renew the token
Thread.sleep(1500);
}
{code}
3. why are the changes in MiniYarnCluster needed ?
4. what is the purpose of testE2ETokenSwap?
> End to end unit test with mini YARN cluster for AMRMProxy Service
> -
>
> Key: YARN-4117
> URL: https://issues.apache.org/jira/browse/YARN-4117
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager, resourcemanager
>Reporter: Kishore Chaliparambil
>Assignee: Giovanni Matteo Fumarola
> Attachments: YARN-4117.v0.patch
>
>
> YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to
> end unit test using mini YARN cluster to the AMRMProxy service. This test
> will validate register, allocate and finish application and token renewal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149431#comment-15149431 ] Giovanni Matteo Fumarola commented on YARN-4117: I just ran all the tests that failed or timed out and they completed fine in my machine. My only concern is about the checkstyle. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[
https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149342#comment-15149342
]
Hadoop QA commented on YARN-4117:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 54s
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 7s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} trunk passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 55s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 6s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 6s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 31s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn: patch generated 3 new +
55 unchanged - 0 fixed = 58 total (was 55) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s
{color} | {color:green} the patch passed with JDK v1.8.0_72 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 48s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.8.0_72. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 45s {color}
| {color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.8.0_72.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 5s {color}
| {color:red} hadoop-yarn-client in the patch failed with JDK v1.8.0_72.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 15s
{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed with
JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 6s {color} |
{color:red} hadoop-yarn-server-tests in the patch failed with JDK v1.7.0_95.
{color} |
|
[jira] [Commented] (YARN-4117) End to end unit test with mini YARN cluster for AMRMProxy Service
[ https://issues.apache.org/jira/browse/YARN-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149006#comment-15149006 ] Giovanni Matteo Fumarola commented on YARN-4117: Hi [~jianhe] Can you please review this patch? I am asking you because you reviewed and pushed the AMRMProxy on YARN-2884. > End to end unit test with mini YARN cluster for AMRMProxy Service > - > > Key: YARN-4117 > URL: https://issues.apache.org/jira/browse/YARN-4117 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager >Reporter: Kishore Chaliparambil >Assignee: Giovanni Matteo Fumarola > Attachments: YARN-4117.v0.patch > > > YARN-2884 introduces a proxy between AM and RM. This JIRA proposes an end to > end unit test using mini YARN cluster to the AMRMProxy service. This test > will validate register, allocate and finish application and token renewal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
