[jira] [Commented] (YARN-9272) Backport YARN-7738 for refreshing max allocation for multiple resource types
[
https://issues.apache.org/jira/browse/YARN-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801169#comment-16801169
]
Hadoop QA commented on YARN-9272:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} YARN-8200 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
24s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
54s{color} | {color:green} YARN-8200 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
37s{color} | {color:green} YARN-8200 passed with JDK v1.8.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} YARN-8200 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
in YARN-8200 has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} YARN-8200 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} YARN-8200 passed with JDK v1.8.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
36s{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
53s{color} | {color:green} the patch passed with JDK v1.8.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
53s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 5s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 19 new + 447 unchanged - 0 fixed = 466 total (was 447) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed with JDK v1.8.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 64m
17s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}157m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:da67579 |
| JIRA Issue | YARN-9272 |
| JIRA Patch URL |
[jira] [Commented] (YARN-7848) Force removal of docker containers that do not get removed on first try
[
https://issues.apache.org/jira/browse/YARN-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801270#comment-16801270
]
Hadoop QA commented on YARN-7848:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
31m 6s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 59s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 49s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-7848 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963667/YARN-7848.001.patch |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit |
| uname | Linux c48ac68a88d7 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 3f6d6d2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/23807/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23807/testReport/ |
| Max. process+thread count | 311 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23807/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Force removal of docker containers that do not get removed on first try
> ---
>
> Key: YARN-7848
> URL: https://issues.apache.org/jira/browse/YARN-7848
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Eric Badger
>Assignee: Zhaohui Xin
>Priority: Major
> Labels: Docker
> Attachments: YARN-7848.001.patch
>
>
> After
[jira] [Commented] (YARN-9409) Port resource type changes from YARN-7237 to branch-3.0/branch-2
[ https://issues.apache.org/jira/browse/YARN-9409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801179#comment-16801179 ] Jonathan Hung commented on YARN-9409: - Unit test failures should be unrelated. > Port resource type changes from YARN-7237 to branch-3.0/branch-2 > > > Key: YARN-9409 > URL: https://issues.apache.org/jira/browse/YARN-9409 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9409-YARN-8200.branch3.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9269) Minor cleanup in FpgaResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801158#comment-16801158
]
Hadoop QA commented on YARN-9269:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 27 unchanged - 3 fixed = 27 total (was 30) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 30s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
15s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9269 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963657/YARN-9269-005.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 2660c18d6cae 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 710cbc9 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23806/testReport/ |
| Max. process+thread count | 447 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
[jira] [Updated] (YARN-7848) Force removal of docker containers that do not get removed on first try
[ https://issues.apache.org/jira/browse/YARN-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7848: Attachment: YARN-7848.001.patch > Force removal of docker containers that do not get removed on first try > --- > > Key: YARN-7848 > URL: https://issues.apache.org/jira/browse/YARN-7848 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Badger >Assignee: Zhaohui Xin >Priority: Major > Labels: Docker > Attachments: YARN-7848.001.patch > > > After the addition of YARN-5366, containers will get removed after a certain > debug delay. However, this is a one-time effort. If the removal fails for > whatever reason, the container will persist. We need to add a mechanism for a > forced removal of those containers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801359#comment-16801359
]
Prabhu Joseph commented on YARN-9405:
-
Thanks [~eyang].
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-9405-001.patch, YARN-9405-002.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7848) Force removal of docker containers that do not get removed on first try
[ https://issues.apache.org/jira/browse/YARN-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801229#comment-16801229 ] Eric Yang commented on YARN-7848: - Back track to original discussion that if container stuck in Created state, using -f flag is good enough to remove faulty container. The background thread may keep system cleaner, but not everyone is comfortable with that idea at this time. Therefore, focus on improvement to the current needs. > Force removal of docker containers that do not get removed on first try > --- > > Key: YARN-7848 > URL: https://issues.apache.org/jira/browse/YARN-7848 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Badger >Assignee: Zhaohui Xin >Priority: Major > Labels: Docker > Attachments: YARN-7848.001.patch > > > After the addition of YARN-5366, containers will get removed after a certain > debug delay. However, this is a one-time effort. If the removal fails for > whatever reason, the container will persist. We need to add a mechanism for a > forced removal of those containers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7129) Application Catalog for YARN applications
[
https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801230#comment-16801230
]
Hadoop QA commented on YARN-7129:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 48s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
8s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
8s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 5s{color} | {color:orange} root: The patch generated 3 new + 4 unchanged -
0 fixed = 7 total (was 4) {color} |
| {color:green}+1{color} | {color:green} hadolint {color} | {color:green} 0m
0s{color} | {color:green} There were no new hadolint issues. {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 12m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:orange}-0{color} | {color:orange} shelldocs {color} | {color:orange}
0m 14s{color} | {color:orange} The patch generated 136 new + 104 unchanged - 0
fixed = 240 total (was 104) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
14s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 43s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-docker
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
|| || || ||
[jira] [Commented] (YARN-9227) DistributedShell RelativePath is not removed at end
[
https://issues.apache.org/jira/browse/YARN-9227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801083#comment-16801083
]
Giovanni Matteo Fumarola commented on YARN-9227:
[~Prabhu Joseph] thanks for the patch.
Please rebase the patch if needed.
> DistributedShell RelativePath is not removed at end
> ---
>
> Key: YARN-9227
> URL: https://issues.apache.org/jira/browse/YARN-9227
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Affects Versions: 3.1.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Minor
> Attachments: 0001-YARN-9227.patch, 0002-YARN-9227.patch,
> 0003-YARN-9227.patch
>
>
> DistributedShell Job does not remove the relative path which contains jars
> and localized files.
> {code}
> [ambari-qa@ash hadoop-yarn]$ hadoop fs -ls
> /user/ambari-qa/DistributedShell/application_1542665708563_0017
> Found 2 items
> -rw-r--r-- 3 ambari-qa hdfs 46636 2019-01-23 13:37
> /user/ambari-qa/DistributedShell/application_1542665708563_0017/AppMaster.jar
> -rwx--x--- 3 ambari-qa hdfs 4 2019-01-23 13:37
> /user/ambari-qa/DistributedShell/application_1542665708563_0017/shellCommands
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9227) DistributedShell RelativePath is not removed at end
[
https://issues.apache.org/jira/browse/YARN-9227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801390#comment-16801390
]
Prabhu Joseph commented on YARN-9227:
-
[~giovanni.fumarola] Thanks for checking, 0003-YARN-9227.patch applies on trunk.
> DistributedShell RelativePath is not removed at end
> ---
>
> Key: YARN-9227
> URL: https://issues.apache.org/jira/browse/YARN-9227
> Project: Hadoop YARN
> Issue Type: Bug
> Components: distributed-shell
>Affects Versions: 3.1.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Minor
> Attachments: 0001-YARN-9227.patch, 0002-YARN-9227.patch,
> 0003-YARN-9227.patch
>
>
> DistributedShell Job does not remove the relative path which contains jars
> and localized files.
> {code}
> [ambari-qa@ash hadoop-yarn]$ hadoop fs -ls
> /user/ambari-qa/DistributedShell/application_1542665708563_0017
> Found 2 items
> -rw-r--r-- 3 ambari-qa hdfs 46636 2019-01-23 13:37
> /user/ambari-qa/DistributedShell/application_1542665708563_0017/AppMaster.jar
> -rwx--x--- 3 ambari-qa hdfs 4 2019-01-23 13:37
> /user/ambari-qa/DistributedShell/application_1542665708563_0017/shellCommands
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801412#comment-16801412 ] Yufei Gu commented on YARN-8967: Committed to trunk. Thanks [~wilfreds] for the contribution. > Change FairScheduler to use PlacementRule interface > --- > > Key: YARN-8967 > URL: https://issues.apache.org/jira/browse/YARN-8967 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler, fairscheduler >Reporter: Wilfred Spiegelenburg >Assignee: Wilfred Spiegelenburg >Priority: Major > Attachments: YARN-8967.001.patch, YARN-8967.002.patch, > YARN-8967.003.patch, YARN-8967.004.patch, YARN-8967.005.patch, > YARN-8967.006.patch, YARN-8967.007.patch, YARN-8967.008.patch, > YARN-8967.009.patch, YARN-8967.010.patch, YARN-8967.011.patch > > > The PlacementRule interface was introduced to be used by all schedulers as > per YARN-3635. The CapacityScheduler is using it but the FairScheduler is not > and is using its own rule definition. > YARN-8948 cleans up the implementation and removes the CS references which > should allow this change to go through. > This would be the first step in using one placement rule engine for both > schedulers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
Prabhu Joseph created YARN-9405:
---
Summary: TestYarnNativeServices#testExpressUpgrade fails
intermittent
Key: YARN-9405
URL: https://issues.apache.org/jira/browse/YARN-9405
Project: Hadoop YARN
Issue Type: Test
Components: yarn-native-services
Affects Versions: 3.2.0
Reporter: Prabhu Joseph
TestYarnNativeServices#testExpressUpgrade fails intermittent
{code}
org.junit.ComparisonFailure: version mismatch expected: but was:
at org.junit.Assert.assertEquals(Assert.java:115)
at
org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9406) DeleteTask may leak when upgrading from 2.7 to 3.x
[ https://issues.apache.org/jira/browse/YARN-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800455#comment-16800455 ] Jiandan Yang commented on YARN-9406: - For compatibility， Creating FileDeletionTask when not having taskType. > DeleteTask may leak when upgrading from 2.7 to 3.x > -- > > Key: YARN-9406 > URL: https://issues.apache.org/jira/browse/YARN-9406 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6366 adds taskType in DeletionServiceDeleteTaskProto to support multiple > types of deleteTask, but NMProtoUtils#convertProtoToDeletionTask does not > consider non-exist of taskType. This will lead to leak of DeleteTask before > YARN-6366. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9406) DeleteTask may leak when upgrading from 2.7 to 3.x
[ https://issues.apache.org/jira/browse/YARN-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiandan Yang updated YARN-9406: Attachment: YARN-9406.001.patch > DeleteTask may leak when upgrading from 2.7 to 3.x > -- > > Key: YARN-9406 > URL: https://issues.apache.org/jira/browse/YARN-9406 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: YARN-9406.001.patch > > > YARN-6366 adds taskType in DeletionServiceDeleteTaskProto to support multiple > types of deleteTask, but NMProtoUtils#convertProtoToDeletionTask does not > consider non-exist of taskType. This will lead to leak of DeleteTask before > YARN-6366. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph reassigned YARN-9405:
---
Assignee: Prabhu Joseph
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9406) DeleteTask may leak when upgrading from 2.7 to 3.x
Jiandan Yang created YARN-9406: --- Summary: DeleteTask may leak when upgrading from 2.7 to 3.x Key: YARN-9406 URL: https://issues.apache.org/jira/browse/YARN-9406 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Jiandan Yang Assignee: Jiandan Yang YARN-6366 adds taskType in DeletionServiceDeleteTaskProto to support multiple types of deleteTask, but NMProtoUtils#convertProtoToDeletionTask does not consider non-exist of taskType. This will lead to leak of DeleteTask before YARN-6366. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-9406) DeleteTask may leak when upgrading from 2.7 to 3.x
[ https://issues.apache.org/jira/browse/YARN-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiandan Yang resolved YARN-9406. - Resolution: Invalid > DeleteTask may leak when upgrading from 2.7 to 3.x > -- > > Key: YARN-9406 > URL: https://issues.apache.org/jira/browse/YARN-9406 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: YARN-9406.001.patch > > > YARN-6366 adds taskType in DeletionServiceDeleteTaskProto to support multiple > types of deleteTask, but NMProtoUtils#convertProtoToDeletionTask does not > consider non-exist of taskType. This will lead to leak of DeleteTask before > YARN-6366. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9406) DeleteTask may leak when upgrading from 2.7 to 3.x
[ https://issues.apache.org/jira/browse/YARN-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800463#comment-16800463 ] Jiandan Yang commented on YARN-9406: - My mistake, NMProtoUtils has used FileDeletionTask as default. > DeleteTask may leak when upgrading from 2.7 to 3.x > -- > > Key: YARN-9406 > URL: https://issues.apache.org/jira/browse/YARN-9406 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: YARN-9406.001.patch > > > YARN-6366 adds taskType in DeletionServiceDeleteTaskProto to support multiple > types of deleteTask, but NMProtoUtils#convertProtoToDeletionTask does not > consider non-exist of taskType. This will lead to leak of DeleteTask before > YARN-6366. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9356) Add more tests to ratio method in TestResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-9356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800483#comment-16800483 ] Adam Antal commented on YARN-9356: -- Thanks for the patch [~zsiegl]. IMO the comments are not needed, but otherwise it's +1 (non-binding). > Add more tests to ratio method in TestResourceCalculator > - > > Key: YARN-9356 > URL: https://issues.apache.org/jira/browse/YARN-9356 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Zoltan Siegl >Priority: Major > Attachments: YARN-9356.001.patch > > > TestResourceCalculator has some edge-case testcases to verify how division by > zero is handled with ResourceCalculator. > We need other basic tests like we have for other ResourceCalculator methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9405:
Attachment: YARN-9405-001.patch
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9405-001.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800518#comment-16800518
]
Prabhu Joseph commented on YARN-9405:
-
TestCase waits for service state to be STABLE after {{actionUpgradeExpress}}.
It fails if test case thread is faster than {{ServiceManager}} Dispatcher
handles {{StartUpgradeTransition}} and getStatus is returned with STABLE v1
before ServiceManager sets state to EXPRESS_UPGRADING.
{code}
2019-03-25 13:44:05,140 INFO [IPC Server handler 0 on default port 51338]
service.ClientAMService (ClientAMService.java:upgrade(174)) - Upgrading service
to version v2 by pjoseph (auth:SIMPLE)
2019-03-25 13:44:05,161 INFO [Component dispatcher] service.ServiceManager
(ServiceManager.java:setServiceState(504)) - [SERVICE] spec state changed from
STABLE -> EXPRESS_UPGRADING
{code}
It can be fixed by checking service state transitions to EXPRESS_UPGRADING
before checking for STABLE.
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9405-001.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9353) TestNMWebFilter should be renamed
[ https://issues.apache.org/jira/browse/YARN-9353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800488#comment-16800488 ] Adam Antal commented on YARN-9353: -- Assigned to you [~smileLee]. > TestNMWebFilter should be renamed > - > > Key: YARN-9353 > URL: https://issues.apache.org/jira/browse/YARN-9353 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: WEI-HSIAO-LEE >Priority: Trivial > Labels: newbie, newbie++ > > TestNMWebFilter should be renamed to should be renamed to NMWebAppFilter, as > there is no class named NMWebFilter. The javadoc of the class is also > outdated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9353) TestNMWebFilter should be renamed
[ https://issues.apache.org/jira/browse/YARN-9353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800487#comment-16800487 ] Sunil Govindan commented on YARN-9353: -- [~smileLee]added you as a contributor. pls assign urself. > TestNMWebFilter should be renamed > - > > Key: YARN-9353 > URL: https://issues.apache.org/jira/browse/YARN-9353 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: WEI-HSIAO-LEE >Priority: Trivial > Labels: newbie, newbie++ > > TestNMWebFilter should be renamed to should be renamed to NMWebAppFilter, as > there is no class named NMWebFilter. The javadoc of the class is also > outdated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9269) Minor cleanup in FpgaResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800484#comment-16800484
]
Peter Bacsko commented on YARN-9269:
[~devaraj.k] could you review this as well?
> Minor cleanup in FpgaResourceAllocator
> --
>
> Key: YARN-9269
> URL: https://issues.apache.org/jira/browse/YARN-9269
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-9269-001.patch, YARN-9269-002.patch,
> YARN-9269-003.patch, YARN-9269-004.patch
>
>
> Some stuff that we observed:
> * {{addFpga()}} - we check for duplicate devices, but we don't print any
> error/warning if there's any.
> * {{findMatchedFpga()}} should be called {{findMatchingFpga()}}. Also, is
> this method even needed? We already receive an {{FpgaDevice}} instance in
> {{updateFpga()}} which I believe is the same that we're looking up.
> * variable {{IPIDpreference}} is confusing
> * {{availableFpga}} / {{usedFpgaByRequestor}} are instances of
> {{LinkedHashMap}}. What's the rationale behind this? Doesn't a simple
> {{HashMap}} suffice?
> * {{usedFpgaByRequestor}} should be renamed, naming is a bit unclear
> * {{allowedFpgas}} should be an immutable list
> * {{@VisibleForTesting}} methods should be package private
> * get rid of {{*}} imports
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9268) General improvements in FpgaDevice
[
https://issues.apache.org/jira/browse/YARN-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800486#comment-16800486
]
Peter Bacsko commented on YARN-9268:
[~devaraj.k] please check the latest patch
> General improvements in FpgaDevice
> --
>
> Key: YARN-9268
> URL: https://issues.apache.org/jira/browse/YARN-9268
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-9268-001.patch, YARN-9268-002.patch,
> YARN-9268-003.patch, YARN-9268-004.patch, YARN-9268-005.patch,
> YARN-9268-006.patch, YARN-9268-007.patch
>
>
> Need to fix the following in the class {{FpgaDevice}}:
> * It implements {{Comparable}}, but returns 0 in every case. There is no
> natural ordering among FPGA devices, perhaps "acl0" comes before "acl1", but
> this seems too forced and unnecessary.We think this class should not
> implement {{Comparable}} at all, at least not like that.
> * Stores unnecessary fields: devName, busNum, temperature, power usage. For
> one, these are never needed in the code. Secondly, temp and power usage
> changes constantly. It's pointless to store these in this POJO.
> * {{serialVersionUID}} is 1L - let's generate a number for this
> * Use {{int}} instead of {{Integer}} - don't allow nulls. If major/minor
> uniquely identifies the card, then let's demand them in the constructor and
> don't store Integers that can be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9353) TestNMWebFilter should be renamed
[ https://issues.apache.org/jira/browse/YARN-9353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal reassigned YARN-9353: Assignee: WEI-HSIAO-LEE > TestNMWebFilter should be renamed > - > > Key: YARN-9353 > URL: https://issues.apache.org/jira/browse/YARN-9353 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: WEI-HSIAO-LEE >Priority: Trivial > Labels: newbie, newbie++ > > TestNMWebFilter should be renamed to should be renamed to NMWebAppFilter, as > there is no class named NMWebFilter. The javadoc of the class is also > outdated. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9394) Use new API of RackResolver to get better performance
[ https://issues.apache.org/jira/browse/YARN-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800503#comment-16800503 ] Weiwei Yang commented on YARN-9394: --- [~cltlfcjin], when you get time, could you please take a look at the failed UT? Thanks. > Use new API of RackResolver to get better performance > - > > Key: YARN-9394 > URL: https://issues.apache.org/jira/browse/YARN-9394 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.3.0, 3.2.1 >Reporter: Lantao Jin >Assignee: Lantao Jin >Priority: Major > Attachments: YARN-9394.001.patch > > > After adding a new API in RackResolver YARN-9332, some old callers should > switch to new API to get better performance. As an example, Spark > [YarnAllocator|https://github.com/apache/spark/blob/733f2c0b98208815f8408e36ab669d7c07e3767f/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala#L361-L363] > for Dynamic Allocation invokes > [https://github.com/apache/hadoop/blob/6fa229891e06eea62cb9634efde755f40247e816/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java#L550] > to resolve racks in a loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800566#comment-16800566
]
Hadoop QA commented on YARN-9405:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 2s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 17m
58s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9405 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963614/YARN-9405-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 7fb36cbc6d4e 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 128dd91 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23798/testReport/ |
| Max. process+thread count | 753 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23798/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (YARN-9356) Add more tests to ratio method in TestResourceCalculator
[ https://issues.apache.org/jira/browse/YARN-9356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800493#comment-16800493 ] Szilard Nemeth commented on YARN-9356: -- Hi [~zsiegl]! Looks good. I would also add some more testcases, for example using some greater values to verify ratio calculation. You could also use the values the other way around, i.e. resource1's resources are greater than values of resource2. > Add more tests to ratio method in TestResourceCalculator > - > > Key: YARN-9356 > URL: https://issues.apache.org/jira/browse/YARN-9356 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Zoltan Siegl >Priority: Major > Attachments: YARN-9356.001.patch > > > TestResourceCalculator has some edge-case testcases to verify how division by > zero is handled with ResourceCalculator. > We need other basic tests like we have for other ResourceCalculator methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9391) Disable PATH variable to be passed to Docker container
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-9391: -- Attachment: YARN-9391.001.patch > Disable PATH variable to be passed to Docker container > -- > > Key: YARN-9391 > URL: https://issues.apache.org/jira/browse/YARN-9391 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Jim Brennan >Priority: Major > Attachments: YARN-9391.001.patch > > > This is observed from using Apache NiFi docker image. It makes assumption > that PATH variable contains /bin to reference to system utility. Where host > YARN environment PATH variable is default to leaked into container by > accident and not containing /bin path (default configuration). In general, > it seems like node manager should block PATH variable from leaking into > container. Not sure if there is a valid use case that host PATH variable > must leak into container from docker point of view. From Hadoop point of > view, if container is merely a chroot, and container is a mirror image of > host worker dir. It is good to keep host PATH variable the same. > Maybe we want to be more specific that block PATH variable to leak into > Docker container, if it is using ENTRYPOINT only? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9403) GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table
[
https://issues.apache.org/jira/browse/YARN-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9403:
Issue Type: Sub-task (was: Bug)
Parent: YARN-7055
> GET /apps/{appid}/entities/YARN_APPLICATION accesses application table
> instead of entity table
> --
>
> Key: YARN-9403
> URL: https://issues.apache.org/jira/browse/YARN-9403
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: ATSv2
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> {noformat}"GET /apps/{appid}/entities/YARN_APPLICATION"{noformat} accesses
> application table instead of entity table. As per the doc, With this API, you
> can query generic entities identified by cluster ID, application ID and
> per-framework entity type. But it also provides all the apps when entityType
> is set to YARN_APPLICATION. It should only access Entity Table through
> {{GenericEntityReader}}.
> Wrong Output: With YARN_APPLICATION entityType, all applications listed from
> application tables.
> {code}
> [hbase@yarn-ats-3 centos]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/YARN_APPLICATION?user.name=hbase=hbase=word%20count;
> | jq .
> [
> {
> "metrics": [],
> "events": [],
> "createdtime": 1553258922721,
> "idprefix": 0,
> "isrelatedto": {},
> "relatesto": {},
> "info": {
> "UID": "ats!application_1553258815132_0002",
> "FROM_ID": "ats!hbase!word
> count!1553258922721!application_1553258815132_0002"
> },
> "configs": {},
> "type": "YARN_APPLICATION",
> "id": "application_1553258815132_0002"
> },
> {
> "metrics": [],
> "events": [],
> "createdtime": 1553258825918,
> "idprefix": 0,
> "isrelatedto": {},
> "relatesto": {},
> "info": {
> "UID": "ats!application_1553258815132_0001",
> "FROM_ID": "ats!hbase!word
> count!1553258825918!application_1553258815132_0001"
> },
> "configs": {},
> "type": "YARN_APPLICATION",
> "id": "application_1553258815132_0001"
> }
> ]
> {code}
> Right Output: With correct entity type (MAPREDUCE_JOB) it accesses entity
> table for given applicationId and entityType.
> {code}
> [hbase@yarn-ats-3 centos]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/MAPREDUCE_JOB?user.name=hbase=hbase=word%20count;
> | jq .
> [
> {
> "metrics": [],
> "events": [],
> "createdtime": 1553258926667,
> "idprefix": 0,
> "isrelatedto": {},
> "relatesto": {},
> "info": {
> "UID":
> "ats!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002",
> "FROM_ID": "ats!hbase!word
> count!1553258922721!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002"
> },
> "configs": {},
> "type": "MAPREDUCE_JOB",
> "id": "job_1553258815132_0002"
> }
> ]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9408) @Path("/apps/{appid}/appattempts") error message misleads
Prabhu Joseph created YARN-9408:
---
Summary: @Path("/apps/{appid}/appattempts") error message misleads
Key: YARN-9408
URL: https://issues.apache.org/jira/browse/YARN-9408
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.2.0
Reporter: Prabhu Joseph
Assignee: Prabhu Joseph
@Path("/apps/{appid}/appattempts") error message is misleading.
NotFoundException "Unable to find the context flow name, and flow run id, and
user id" is displayed while app attempts is looked.
{code}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0004",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0005",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9403) GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table
[
https://issues.apache.org/jira/browse/YARN-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9403:
Attachment: YARN-9403-001.patch
> GET /apps/{appid}/entities/YARN_APPLICATION accesses application table
> instead of entity table
> --
>
> Key: YARN-9403
> URL: https://issues.apache.org/jira/browse/YARN-9403
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: ATSv2
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9403-001.patch
>
>
> {noformat}"GET /apps/{appid}/entities/YARN_APPLICATION"{noformat} accesses
> application table instead of entity table. As per the doc, With this API, you
> can query generic entities identified by cluster ID, application ID and
> per-framework entity type. But it also provides all the apps when entityType
> is set to YARN_APPLICATION. It should only access Entity Table through
> {{GenericEntityReader}}.
> Wrong Output: With YARN_APPLICATION entityType, all applications listed from
> application tables.
> {code}
> [hbase@yarn-ats-3 centos]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/YARN_APPLICATION?user.name=hbase=hbase=word%20count;
> | jq .
> [
> {
> "metrics": [],
> "events": [],
> "createdtime": 1553258922721,
> "idprefix": 0,
> "isrelatedto": {},
> "relatesto": {},
> "info": {
> "UID": "ats!application_1553258815132_0002",
> "FROM_ID": "ats!hbase!word
> count!1553258922721!application_1553258815132_0002"
> },
> "configs": {},
> "type": "YARN_APPLICATION",
> "id": "application_1553258815132_0002"
> },
> {
> "metrics": [],
> "events": [],
> "createdtime": 1553258825918,
> "idprefix": 0,
> "isrelatedto": {},
> "relatesto": {},
> "info": {
> "UID": "ats!application_1553258815132_0001",
> "FROM_ID": "ats!hbase!word
> count!1553258825918!application_1553258815132_0001"
> },
> "configs": {},
> "type": "YARN_APPLICATION",
> "id": "application_1553258815132_0001"
> }
> ]
> {code}
> Right Output: With correct entity type (MAPREDUCE_JOB) it accesses entity
> table for given applicationId and entityType.
> {code}
> [hbase@yarn-ats-3 centos]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0002/entities/MAPREDUCE_JOB?user.name=hbase=hbase=word%20count;
> | jq .
> [
> {
> "metrics": [],
> "events": [],
> "createdtime": 1553258926667,
> "idprefix": 0,
> "isrelatedto": {},
> "relatesto": {},
> "info": {
> "UID":
> "ats!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002",
> "FROM_ID": "ats!hbase!word
> count!1553258922721!application_1553258815132_0002!MAPREDUCE_JOB!0!job_1553258815132_0002"
> },
> "configs": {},
> "type": "MAPREDUCE_JOB",
> "id": "job_1553258815132_0002"
> }
> ]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800871#comment-16800871
]
Eric Yang commented on YARN-9405:
-
[~Prabhu Joseph] Thank you for fixing this issue. I think this is a good
patch. However, after applying this change, I see another thread race
condition occuring in the same test class sometimes.
{code}
[ERROR] Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
773.903 s <<< FAILURE! - in
org.apache.hadoop.yarn.service.TestYarnNativeServices
[ERROR]
testCreateFlexStopDestroyService(org.apache.hadoop.yarn.service.TestYarnNativeServices)
Time elapsed: 50.97 s <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<3>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at
org.apache.hadoop.yarn.service.TestYarnNativeServices.checkEachCompInstancesInOrder(TestYarnNativeServices.java:873)
at
org.apache.hadoop.yarn.service.TestYarnNativeServices.checkCompInstancesInOrder(TestYarnNativeServices.java:866)
at
org.apache.hadoop.yarn.service.TestYarnNativeServices.testCreateFlexStopDestroyService(TestYarnNativeServices.java:122)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]
TestYarnNativeServices.testCreateFlexStopDestroyService:122->checkCompInstancesInOrder:866->checkEachCompInstancesInOrder:873
expected:<1> but was:<3>
[INFO]
[ERROR] Tests run: 13, Failures: 1, Errors: 0, Skipped: 0
{code}
It looks like similar logic needs to be applied to
testCreateFlexStopDestroyService unit test case. Can we include the same fix
to the flaky test?
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9405-001.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail:
[jira] [Created] (YARN-9407) getComputedResourceLimitForActiveUsers debug line does not show username
steven W created YARN-9407:
--
Summary: getComputedResourceLimitForActiveUsers debug line does
not show username
Key: YARN-9407
URL: https://issues.apache.org/jira/browse/YARN-9407
Project: Hadoop YARN
Issue Type: Improvement
Components: capacityscheduler
Affects Versions: 2.7.3
Reporter: steven W
Debug logging of the capacityscheduler doesn't log the username when printing
the calculated user limit. In general you can deduct what user it is, but it
would be great if the username is added to the logline.
Example logline:
2019-03-25 11:05:08,956 DEBUG capacity.UsersManager
(UsersManager.java:getComputedResourceLimitForActiveUsers(525)) - userLimit is
fetched. userLimit=, userSpecificUserLimit=, schedulingMode=RESPECT_PARTITION_EXCLUSIVITY, partition=
Change code block in function
{color:#6f42c1}getComputedResourceLimitForActiveUsers{color:#001000}
{color}{color}{color:#6f42c1}{color:#001000}in file
{color}{color}java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/UsersManager.java
The actual block printing the line:
LOG.debug("userLimit is fetched. userLimit={}, userSpecificUserLimit={},"
+ " schedulingMode={}, partition={}", userLimitResource,
userSpecificUserLimit, schedulingMode, nodePartition);
should be changed to contain the username
It's a minor change and would help debugging
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7129) Application Catalog for YARN applications
[ https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800851#comment-16800851 ] Steve Loughran commented on YARN-7129: -- if the font has been moved to an artifact download, I'm happy, No opinions one the other bits -will leave to others > Application Catalog for YARN applications > - > > Key: YARN-7129 > URL: https://issues.apache.org/jira/browse/YARN-7129 > Project: Hadoop YARN > Issue Type: New Feature > Components: applications >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN Appstore.pdf, YARN-7129.001.patch, > YARN-7129.002.patch, YARN-7129.003.patch, YARN-7129.004.patch, > YARN-7129.005.patch, YARN-7129.006.patch, YARN-7129.007.patch, > YARN-7129.008.patch, YARN-7129.009.patch, YARN-7129.010.patch, > YARN-7129.011.patch, YARN-7129.012.patch, YARN-7129.013.patch, > YARN-7129.014.patch, YARN-7129.015.patch, YARN-7129.016.patch, > YARN-7129.017.patch, YARN-7129.018.patch, YARN-7129.019.patch, > YARN-7129.020.patch, YARN-7129.021.patch, YARN-7129.022.patch, > YARN-7129.023.patch, YARN-7129.024.patch, YARN-7129.025.patch, > YARN-7129.026.patch, YARN-7129.027.patch, YARN-7129.028.patch, > YARN-7129.029.patch, YARN-7129.030.patch, YARN-7129.031.patch, > YARN-7129.032.patch, YARN-7129.033.patch, YARN-7129.034.patch > > > YARN native services provides web services API to improve usability of > application deployment on Hadoop using collection of docker images. It would > be nice to have an application catalog system which provides an editorial and > search interface for YARN applications. This improves usability of YARN for > manage the life cycle of applications. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9408) @Path("/apps/{appid}/appattempts") error message misleads
[
https://issues.apache.org/jira/browse/YARN-9408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9408:
Description:
{code} @Path("/apps/{appid}/appattempts") {code} error message is misleading.
NotFoundException "Unable to find the context flow name, and flow run id, and
user id" is displayed while app attempts is looked.
{code}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0004",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0005",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
{code}
was:
@Path("/apps/{appid}/appattempts") error message is misleading.
NotFoundException "Unable to find the context flow name, and flow run id, and
user id" is displayed while app attempts is looked.
{code}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0004",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0005",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
{code}
> @Path("/apps/{appid}/appattempts") error message misleads
> -
>
> Key: YARN-9408
> URL: https://issues.apache.org/jira/browse/YARN-9408
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Minor
>
> {code} @Path("/apps/{appid}/appattempts") {code} error message is misleading.
> NotFoundException "Unable to find the context flow name, and flow run id, and
> user id" is displayed while app attempts is looked.
> {code}
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats,
> appId=application_1553258815132_0004",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats,
> appId=application_1553258815132_0005",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9408) @Path("/apps/{appid}/appattempts") error message misleads
[
https://issues.apache.org/jira/browse/YARN-9408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9408:
Description:
{code} @Path("/apps/{appid}/appattempts") {code} error message is misleading.
NotFoundException "Unable to find the context flow name, and flow run id, and
user id" is displayed while app attempts is looked.
{code}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0004",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0005",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats1/apps/application_1553258815132_0001/containers/container_e14_1553258815132_0001_01_01?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats1,
appId=application_1553258815132_0001",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats1/apps/application_1553258815132_0001/appattempts/appattempt_1553258815132_0001_01/containers?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats1,
appId=application_1553258815132_0001",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
{code}
was:
{code} @Path("/apps/{appid}/appattempts") {code} error message is misleading.
NotFoundException "Unable to find the context flow name, and flow run id, and
user id" is displayed while app attempts is looked.
{code}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0004",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
[hbase@yarn-ats-3 ~]$ curl -s
"http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
| jq .
{
"exception": "NotFoundException",
"message": "java.lang.Exception: Unable to find the context flow name, and
flow run id, and user id for clusterId=ats,
appId=application_1553258815132_0005",
"javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
}
{code}
> @Path("/apps/{appid}/appattempts") error message misleads
> -
>
> Key: YARN-9408
> URL: https://issues.apache.org/jira/browse/YARN-9408
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Minor
>
> {code} @Path("/apps/{appid}/appattempts") {code} error message is misleading.
> NotFoundException "Unable to find the context flow name, and flow run id, and
> user id" is displayed while app attempts is looked.
> {code}
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats,
> appId=application_1553258815132_0004",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats,
> appId=application_1553258815132_0005",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> [hbase@yarn-ats-3 ~]$ curl -s
>
[jira] [Updated] (YARN-9408) @Path("/apps/{appid}/appattempts") error message misleads
[
https://issues.apache.org/jira/browse/YARN-9408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9408:
Component/s: ATSv2
> @Path("/apps/{appid}/appattempts") error message misleads
> -
>
> Key: YARN-9408
> URL: https://issues.apache.org/jira/browse/YARN-9408
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Minor
>
> {code} @Path("/apps/{appid}/appattempts") {code} error message is misleading.
> NotFoundException "Unable to find the context flow name, and flow run id, and
> user id" is displayed while app attempts is looked.
> {code}
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1553258815132_0004/appattempts?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats,
> appId=application_1553258815132_0004",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats/apps/application_1553258815132_0005/appattempts?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats,
> appId=application_1553258815132_0005",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats1/apps/application_1553258815132_0001/containers/container_e14_1553258815132_0001_01_01?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats1,
> appId=application_1553258815132_0001",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> [hbase@yarn-ats-3 ~]$ curl -s
> "http://yarn-ats-3:8198/ws/v2/timeline/clusters/ats1/apps/application_1553258815132_0001/appattempts/appattempt_1553258815132_0001_01/containers?user.name=hbase;
> | jq .
> {
> "exception": "NotFoundException",
> "message": "java.lang.Exception: Unable to find the context flow name, and
> flow run id, and user id for clusterId=ats1,
> appId=application_1553258815132_0001",
> "javaClassName": "org.apache.hadoop.yarn.webapp.NotFoundException"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container
[
https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800930#comment-16800930
]
Hadoop QA commented on YARN-9391:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 13s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 32s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
38s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9391 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963640/YARN-9391.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d23a5771029f 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e5d72f5 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23799/testReport/ |
| Max. process+thread count | 317 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23799/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Disable PATH variable to be passed to Docker container
[jira] [Commented] (YARN-9403) GET /apps/{appid}/entities/YARN_APPLICATION accesses application table instead of entity table
[
https://issues.apache.org/jira/browse/YARN-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800936#comment-16800936
]
Hadoop QA commented on YARN-9403:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} pathlen {color} | {color:red} 0m
0s{color} | {color:red} The patch appears to contain 1 files with names longer
than 240 {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 8s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 10s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 5 new +
40 unchanged - 0 fixed = 45 total (was 40) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 5s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
25s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
28s{color} | {color:green} hadoop-yarn-server-timelineservice-hbase-client in
the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9403 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963641/YARN-9403-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 3bded9043cbc 4.4.0-138-generic
[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800957#comment-16800957 ] Jim Brennan commented on YARN-9391: --- [~ebadger], [~eyang] patch 001 is ready for review. > Disable PATH variable to be passed to Docker container > -- > > Key: YARN-9391 > URL: https://issues.apache.org/jira/browse/YARN-9391 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Jim Brennan >Priority: Major > Attachments: YARN-9391.001.patch > > > This is observed from using Apache NiFi docker image. It makes assumption > that PATH variable contains /bin to reference to system utility. Where host > YARN environment PATH variable is default to leaked into container by > accident and not containing /bin path (default configuration). In general, > it seems like node manager should block PATH variable from leaking into > container. Not sure if there is a valid use case that host PATH variable > must leak into container from docker point of view. From Hadoop point of > view, if container is merely a chroot, and container is a mirror image of > host worker dir. It is good to keep host PATH variable the same. > Maybe we want to be more specific that block PATH variable to leak into > Docker container, if it is using ENTRYPOINT only? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800961#comment-16800961 ] Eric Yang commented on YARN-9391: - +1 verified with mapreduce, non-entrypoint mode, and entrypoint mode. > Disable PATH variable to be passed to Docker container > -- > > Key: YARN-9391 > URL: https://issues.apache.org/jira/browse/YARN-9391 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Jim Brennan >Priority: Major > Attachments: YARN-9391.001.patch > > > This is observed from using Apache NiFi docker image. It makes assumption > that PATH variable contains /bin to reference to system utility. Where host > YARN environment PATH variable is default to leaked into container by > accident and not containing /bin path (default configuration). In general, > it seems like node manager should block PATH variable from leaking into > container. Not sure if there is a valid use case that host PATH variable > must leak into container from docker point of view. From Hadoop point of > view, if container is merely a chroot, and container is a mirror image of > host worker dir. It is good to keep host PATH variable the same. > Maybe we want to be more specific that block PATH variable to leak into > Docker container, if it is using ENTRYPOINT only? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9409) Port resource type changes from YARN-7237 to branch-3.0/branch-2
Jonathan Hung created YARN-9409: --- Summary: Port resource type changes from YARN-7237 to branch-3.0/branch-2 Key: YARN-9409 URL: https://issues.apache.org/jira/browse/YARN-9409 Project: Hadoop YARN Issue Type: Sub-task Reporter: Jonathan Hung Assignee: Jonathan Hung -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7129) Application Catalog for YARN applications
[ https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800970#comment-16800970 ] Eric Yang commented on YARN-7129: - Patch 035 removed asm and javaee-api dependencies. Solr did not have some dependencies declared properly in previous release. This has been fixed, and no longer necessary to list them explicitly. > Application Catalog for YARN applications > - > > Key: YARN-7129 > URL: https://issues.apache.org/jira/browse/YARN-7129 > Project: Hadoop YARN > Issue Type: New Feature > Components: applications >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN Appstore.pdf, YARN-7129.001.patch, > YARN-7129.002.patch, YARN-7129.003.patch, YARN-7129.004.patch, > YARN-7129.005.patch, YARN-7129.006.patch, YARN-7129.007.patch, > YARN-7129.008.patch, YARN-7129.009.patch, YARN-7129.010.patch, > YARN-7129.011.patch, YARN-7129.012.patch, YARN-7129.013.patch, > YARN-7129.014.patch, YARN-7129.015.patch, YARN-7129.016.patch, > YARN-7129.017.patch, YARN-7129.018.patch, YARN-7129.019.patch, > YARN-7129.020.patch, YARN-7129.021.patch, YARN-7129.022.patch, > YARN-7129.023.patch, YARN-7129.024.patch, YARN-7129.025.patch, > YARN-7129.026.patch, YARN-7129.027.patch, YARN-7129.028.patch, > YARN-7129.029.patch, YARN-7129.030.patch, YARN-7129.031.patch, > YARN-7129.032.patch, YARN-7129.033.patch, YARN-7129.034.patch, > YARN-7129.035.patch > > > YARN native services provides web services API to improve usability of > application deployment on Hadoop using collection of docker images. It would > be nice to have an application catalog system which provides an editorial and > search interface for YARN applications. This improves usability of YARN for > manage the life cycle of applications. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7129) Application Catalog for YARN applications
[ https://issues.apache.org/jira/browse/YARN-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7129: Attachment: YARN-7129.035.patch > Application Catalog for YARN applications > - > > Key: YARN-7129 > URL: https://issues.apache.org/jira/browse/YARN-7129 > Project: Hadoop YARN > Issue Type: New Feature > Components: applications >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN Appstore.pdf, YARN-7129.001.patch, > YARN-7129.002.patch, YARN-7129.003.patch, YARN-7129.004.patch, > YARN-7129.005.patch, YARN-7129.006.patch, YARN-7129.007.patch, > YARN-7129.008.patch, YARN-7129.009.patch, YARN-7129.010.patch, > YARN-7129.011.patch, YARN-7129.012.patch, YARN-7129.013.patch, > YARN-7129.014.patch, YARN-7129.015.patch, YARN-7129.016.patch, > YARN-7129.017.patch, YARN-7129.018.patch, YARN-7129.019.patch, > YARN-7129.020.patch, YARN-7129.021.patch, YARN-7129.022.patch, > YARN-7129.023.patch, YARN-7129.024.patch, YARN-7129.025.patch, > YARN-7129.026.patch, YARN-7129.027.patch, YARN-7129.028.patch, > YARN-7129.029.patch, YARN-7129.030.patch, YARN-7129.031.patch, > YARN-7129.032.patch, YARN-7129.033.patch, YARN-7129.034.patch, > YARN-7129.035.patch > > > YARN native services provides web services API to improve usability of > application deployment on Hadoop using collection of docker images. It would > be nice to have an application catalog system which provides an editorial and > search interface for YARN applications. This improves usability of YARN for > manage the life cycle of applications. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9409) Port resource type changes from YARN-7237 to branch-3.0/branch-2
[ https://issues.apache.org/jira/browse/YARN-9409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9409: Attachment: YARN-9409-YARN-8200.branch3.001.patch > Port resource type changes from YARN-7237 to branch-3.0/branch-2 > > > Key: YARN-9409 > URL: https://issues.apache.org/jira/browse/YARN-9409 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9409-YARN-8200.branch3.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-9405:
Attachment: YARN-9405-002.patch
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9405-001.patch, YARN-9405-002.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800966#comment-16800966
]
Prabhu Joseph commented on YARN-9405:
-
Thanks [~eyang] for reviewing. Attaching v2 patch with retry to check if
component gets number of expected instances.
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9405-001.patch, YARN-9405-002.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9269) Minor cleanup in FpgaResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-9269:
---
Attachment: YARN-9269-005.patch
> Minor cleanup in FpgaResourceAllocator
> --
>
> Key: YARN-9269
> URL: https://issues.apache.org/jira/browse/YARN-9269
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-9269-001.patch, YARN-9269-002.patch,
> YARN-9269-003.patch, YARN-9269-004.patch, YARN-9269-005.patch
>
>
> Some stuff that we observed:
> * {{addFpga()}} - we check for duplicate devices, but we don't print any
> error/warning if there's any.
> * {{findMatchedFpga()}} should be called {{findMatchingFpga()}}. Also, is
> this method even needed? We already receive an {{FpgaDevice}} instance in
> {{updateFpga()}} which I believe is the same that we're looking up.
> * variable {{IPIDpreference}} is confusing
> * {{availableFpga}} / {{usedFpgaByRequestor}} are instances of
> {{LinkedHashMap}}. What's the rationale behind this? Doesn't a simple
> {{HashMap}} suffice?
> * {{usedFpgaByRequestor}} should be renamed, naming is a bit unclear
> * {{allowedFpgas}} should be an immutable list
> * {{@VisibleForTesting}} methods should be package private
> * get rid of {{*}} imports
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9269) Minor cleanup in FpgaResourceAllocator
[
https://issues.apache.org/jira/browse/YARN-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801043#comment-16801043
]
Peter Bacsko commented on YARN-9269:
Uploaded patch v5, because v4 no longer applies to trunk cleanly.
> Minor cleanup in FpgaResourceAllocator
> --
>
> Key: YARN-9269
> URL: https://issues.apache.org/jira/browse/YARN-9269
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-9269-001.patch, YARN-9269-002.patch,
> YARN-9269-003.patch, YARN-9269-004.patch, YARN-9269-005.patch
>
>
> Some stuff that we observed:
> * {{addFpga()}} - we check for duplicate devices, but we don't print any
> error/warning if there's any.
> * {{findMatchedFpga()}} should be called {{findMatchingFpga()}}. Also, is
> this method even needed? We already receive an {{FpgaDevice}} instance in
> {{updateFpga()}} which I believe is the same that we're looking up.
> * variable {{IPIDpreference}} is confusing
> * {{availableFpga}} / {{usedFpgaByRequestor}} are instances of
> {{LinkedHashMap}}. What's the rationale behind this? Doesn't a simple
> {{HashMap}} suffice?
> * {{usedFpgaByRequestor}} should be renamed, naming is a bit unclear
> * {{allowedFpgas}} should be an immutable list
> * {{@VisibleForTesting}} methods should be package private
> * get rid of {{*}} imports
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9268) General improvements in FpgaDevice
[
https://issues.apache.org/jira/browse/YARN-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801047#comment-16801047
]
Hudson commented on YARN-9268:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16279 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/16279/])
YARN-9268. General improvements in FpgaDevice. Contributed by Peter (devaraj:
rev eeda6891e49bc13ae86d0193f94238b7109e291d)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/discovery/DeviceSpecParser.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/fpga/FpgaResourceAllocator.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/AoclDiagnosticOutputParser.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/TestFpgaDiscoverer.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/TestAoclOutputParser.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/FpgaDiscoverer.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/resources/fpga/TestFpgaResourceHandler.java
> General improvements in FpgaDevice
> --
>
> Key: YARN-9268
> URL: https://issues.apache.org/jira/browse/YARN-9268
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-9268-001.patch, YARN-9268-002.patch,
> YARN-9268-003.patch, YARN-9268-004.patch, YARN-9268-005.patch,
> YARN-9268-006.patch, YARN-9268-007.patch
>
>
> Need to fix the following in the class {{FpgaDevice}}:
> * It implements {{Comparable}}, but returns 0 in every case. There is no
> natural ordering among FPGA devices, perhaps "acl0" comes before "acl1", but
> this seems too forced and unnecessary.We think this class should not
> implement {{Comparable}} at all, at least not like that.
> * Stores unnecessary fields: devName, busNum, temperature, power usage. For
> one, these are never needed in the code. Secondly, temp and power usage
> changes constantly. It's pointless to store these in this POJO.
> * {{serialVersionUID}} is 1L - let's generate a number for this
> * Use {{int}} instead of {{Integer}} - don't allow nulls. If major/minor
> uniquely identifies the card, then let's demand them in the constructor and
> don't store Integers that can be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9409) Port resource type changes from YARN-7237 to branch-3.0/branch-2
[
https://issues.apache.org/jira/browse/YARN-9409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801050#comment-16801050
]
Hadoop QA commented on YARN-9409:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} YARN-8200.branch3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
55s{color} | {color:green} YARN-8200.branch3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} YARN-8200.branch3 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} YARN-8200.branch3 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} YARN-8200.branch3 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 35s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} YARN-8200.branch3 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} YARN-8200.branch3 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 0 new + 111 unchanged - 3 fixed = 111 total (was 114)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 41s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 57m 37s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}103m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:e402791 |
| JIRA Issue | YARN-9409 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963652/YARN-9409-YARN-8200.branch3.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux baf3b313a3b1 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | YARN-8200.branch3 / 6eefb78 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit |
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801066#comment-16801066
]
Hudson commented on YARN-9405:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16280 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/16280/])
YARN-9405. Fixed flaky tests in TestYarnNativeServices. (eyang:
rev 710cbc9bd649123cb0f742e4a91a6a216cb1ac76)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/java/org/apache/hadoop/yarn/service/ServiceTestUtils.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/test/java/org/apache/hadoop/yarn/service/TestYarnNativeServices.java
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-9405-001.patch, YARN-9405-002.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9391) Disable PATH variable to be passed to Docker container
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9391: Affects Version/s: 3.2.0 3.1.1 3.1.2 > Disable PATH variable to be passed to Docker container > -- > > Key: YARN-9391 > URL: https://issues.apache.org/jira/browse/YARN-9391 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.2.0, 3.1.1, 3.1.2 >Reporter: Eric Yang >Assignee: Jim Brennan >Priority: Major > Fix For: 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9391.001.patch > > > This is observed from using Apache NiFi docker image. It makes assumption > that PATH variable contains /bin to reference to system utility. Where host > YARN environment PATH variable is default to leaked into container by > accident and not containing /bin path (default configuration). In general, > it seems like node manager should block PATH variable from leaking into > container. Not sure if there is a valid use case that host PATH variable > must leak into container from docker point of view. From Hadoop point of > view, if container is merely a chroot, and container is a mirror image of > host worker dir. It is good to keep host PATH variable the same. > Maybe we want to be more specific that block PATH variable to leak into > Docker container, if it is using ENTRYPOINT only? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801020#comment-16801020 ] Hudson commented on YARN-9391: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16277 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16277/]) YARN-9391. Fixed node manager environment leaks into Docker containers. (eyang: rev 3c45762a0bfb403e069a03e30d35dd11432ee8b0) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > Disable PATH variable to be passed to Docker container > -- > > Key: YARN-9391 > URL: https://issues.apache.org/jira/browse/YARN-9391 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.2.0, 3.1.1, 3.1.2 >Reporter: Eric Yang >Assignee: Jim Brennan >Priority: Major > Fix For: 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9391.001.patch > > > This is observed from using Apache NiFi docker image. It makes assumption > that PATH variable contains /bin to reference to system utility. Where host > YARN environment PATH variable is default to leaked into container by > accident and not containing /bin path (default configuration). In general, > it seems like node manager should block PATH variable from leaking into > container. Not sure if there is a valid use case that host PATH variable > must leak into container from docker point of view. From Hadoop point of > view, if container is merely a chroot, and container is a mirror image of > host worker dir. It is good to keep host PATH variable the same. > Maybe we want to be more specific that block PATH variable to leak into > Docker container, if it is using ENTRYPOINT only? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9268) General improvements in FpgaDevice
[
https://issues.apache.org/jira/browse/YARN-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated YARN-9268:
Hadoop Flags: Reviewed
> General improvements in FpgaDevice
> --
>
> Key: YARN-9268
> URL: https://issues.apache.org/jira/browse/YARN-9268
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-9268-001.patch, YARN-9268-002.patch,
> YARN-9268-003.patch, YARN-9268-004.patch, YARN-9268-005.patch,
> YARN-9268-006.patch, YARN-9268-007.patch
>
>
> Need to fix the following in the class {{FpgaDevice}}:
> * It implements {{Comparable}}, but returns 0 in every case. There is no
> natural ordering among FPGA devices, perhaps "acl0" comes before "acl1", but
> this seems too forced and unnecessary.We think this class should not
> implement {{Comparable}} at all, at least not like that.
> * Stores unnecessary fields: devName, busNum, temperature, power usage. For
> one, these are never needed in the code. Secondly, temp and power usage
> changes constantly. It's pointless to store these in this POJO.
> * {{serialVersionUID}} is 1L - let's generate a number for this
> * Use {{int}} instead of {{Integer}} - don't allow nulls. If major/minor
> uniquely identifies the card, then let's demand them in the constructor and
> don't store Integers that can be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9272) Backport YARN-7738 for refreshing max allocation for multiple resource types
[ https://issues.apache.org/jira/browse/YARN-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800996#comment-16800996 ] Jonathan Hung commented on YARN-9272: - Attached branch-2 version which is the same as the branch-3 one but with initMockRMWithOldConf argument declared as final (requirement in java 7) > Backport YARN-7738 for refreshing max allocation for multiple resource types > > > Key: YARN-9272 > URL: https://issues.apache.org/jira/browse/YARN-9272 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9272-YARN-8200.001.patch, > YARN-9272-YARN-8200.branch3.001.patch, YARN-9272-YARN-8200.branch3.002.patch > > > Need to port to YARN-8200.branch3 (for branch-3.0) and YARN-8200 (for > branch-2) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9272) Backport YARN-7738 for refreshing max allocation for multiple resource types
[ https://issues.apache.org/jira/browse/YARN-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9272: Attachment: YARN-9272-YARN-8200.001.patch > Backport YARN-7738 for refreshing max allocation for multiple resource types > > > Key: YARN-9272 > URL: https://issues.apache.org/jira/browse/YARN-9272 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9272-YARN-8200.001.patch, > YARN-9272-YARN-8200.branch3.001.patch, YARN-9272-YARN-8200.branch3.002.patch > > > Need to port to YARN-8200.branch3 (for branch-3.0) and YARN-8200 (for > branch-2) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801003#comment-16801003
]
Hadoop QA commented on YARN-9405:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 32s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 17m
31s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9405 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963650/YARN-9405-002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 19541d8e5357 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e5d72f5 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23801/testReport/ |
| Max. process+thread count | 766 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23801/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (YARN-9391) Disable PATH variable to be passed to Docker container
[ https://issues.apache.org/jira/browse/YARN-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801008#comment-16801008 ] Eric Badger commented on YARN-9391: --- +1 lgtm > Disable PATH variable to be passed to Docker container > -- > > Key: YARN-9391 > URL: https://issues.apache.org/jira/browse/YARN-9391 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Jim Brennan >Priority: Major > Attachments: YARN-9391.001.patch > > > This is observed from using Apache NiFi docker image. It makes assumption > that PATH variable contains /bin to reference to system utility. Where host > YARN environment PATH variable is default to leaked into container by > accident and not containing /bin path (default configuration). In general, > it seems like node manager should block PATH variable from leaking into > container. Not sure if there is a valid use case that host PATH variable > must leak into container from docker point of view. From Hadoop point of > view, if container is merely a chroot, and container is a mirror image of > host worker dir. It is good to keep host PATH variable the same. > Maybe we want to be more specific that block PATH variable to leak into > Docker container, if it is using ENTRYPOINT only? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9268) General improvements in FpgaDevice
[
https://issues.apache.org/jira/browse/YARN-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801025#comment-16801025
]
Devaraj K commented on YARN-9268:
-
+1, latest patch looks good to me.
> General improvements in FpgaDevice
> --
>
> Key: YARN-9268
> URL: https://issues.apache.org/jira/browse/YARN-9268
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-9268-001.patch, YARN-9268-002.patch,
> YARN-9268-003.patch, YARN-9268-004.patch, YARN-9268-005.patch,
> YARN-9268-006.patch, YARN-9268-007.patch
>
>
> Need to fix the following in the class {{FpgaDevice}}:
> * It implements {{Comparable}}, but returns 0 in every case. There is no
> natural ordering among FPGA devices, perhaps "acl0" comes before "acl1", but
> this seems too forced and unnecessary.We think this class should not
> implement {{Comparable}} at all, at least not like that.
> * Stores unnecessary fields: devName, busNum, temperature, power usage. For
> one, these are never needed in the code. Secondly, temp and power usage
> changes constantly. It's pointless to store these in this POJO.
> * {{serialVersionUID}} is 1L - let's generate a number for this
> * Use {{int}} instead of {{Integer}} - don't allow nulls. If major/minor
> uniquely identifies the card, then let's demand them in the constructor and
> don't store Integers that can be null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9405) TestYarnNativeServices#testExpressUpgrade fails intermittent
[
https://issues.apache.org/jira/browse/YARN-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801035#comment-16801035
]
Eric Yang commented on YARN-9405:
-
[~Prabhu Joseph] Thanks for the fix. Patch 002 produces consistent results. +1
> TestYarnNativeServices#testExpressUpgrade fails intermittent
>
>
> Key: YARN-9405
> URL: https://issues.apache.org/jira/browse/YARN-9405
> Project: Hadoop YARN
> Issue Type: Test
> Components: yarn-native-services
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: YARN-9405-001.patch, YARN-9405-002.patch
>
>
> TestYarnNativeServices#testExpressUpgrade fails intermittent
> {code}
> org.junit.ComparisonFailure: version mismatch expected: but was:
> at org.junit.Assert.assertEquals(Assert.java:115)
> at
> org.apache.hadoop.yarn.service.TestYarnNativeServices.testExpressUpgrade(TestYarnNativeServices.java:445)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
