[jira] [Commented] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[
https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040669#comment-17040669
]
Hadoop QA commented on YARN-8529:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 5m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 13m
22s{color} | {color:red} root in trunk failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
46s{color} | {color:red} hadoop-yarn in trunk failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 46s{color} | {color:orange} The patch fails to run checkstyle in hadoop-yarn
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
32s{color} | {color:red} hadoop-yarn-api in trunk failed. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 23m
25s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 6m 51s{color}
| {color:red} hadoop-yarn-project_hadoop-yarn generated 330 new + 0 unchanged -
0 fixed = 330 total (was 0) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 21s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 209 new + 0 unchanged - 0 fixed = 209 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
0s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
58s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
44s{color} | {color:green} hadoop-yarn-server-router in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-8529 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993968/YARN-8529.v8.patch |
|
[jira] [Updated] (YARN-8529) Add timeout to RouterWebServiceUtil#invokeRMWebService
[
https://issues.apache.org/jira/browse/YARN-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Minni Mittal updated YARN-8529:
---
Attachment: YARN-8529.v8.patch
> Add timeout to RouterWebServiceUtil#invokeRMWebService
> --
>
> Key: YARN-8529
> URL: https://issues.apache.org/jira/browse/YARN-8529
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Íñigo Goiri
>Assignee: Minni Mittal
>Priority: Major
> Attachments: YARN-8529.v1.patch, YARN-8529.v2.patch,
> YARN-8529.v3.patch, YARN-8529.v4.patch, YARN-8529.v5.patch,
> YARN-8529.v6.patch, YARN-8529.v7.patch, YARN-8529.v8.patch
>
>
> {{RouterWebServiceUtil#invokeRMWebService}} currently has a fixed timeout.
> This should be configurable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040521#comment-17040521
]
Hadoop QA commented on YARN-10140:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.10 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
11s{color} | {color:green} branch-2.10 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} branch-2.10 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} branch-2.10 passed with JDK v1.8.0_242 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} branch-2.10 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} branch-2.10 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} branch-2.10 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} branch-2.10 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} branch-2.10 passed with JDK v1.8.0_242 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed with JDK v1.8.0_242 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed with JDK v1.8.0_242 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
51s{color} | {color:green} hadoop-yarn-server-tests in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:a969cad0a12 |
| JIRA Issue | YARN-10140 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993952/YARN-10140-branch-2.10.003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs
[jira] [Updated] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein updated YARN-10140:
-
Attachment: YARN-10140-branch-2.10.003.patch
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: YARN-10140-branch-2.10.001.patch,
> YARN-10140-branch-2.10.002.patch, YARN-10140-branch-2.10.003.patch
>
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
>
[jira] [Updated] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein updated YARN-10140:
-
Attachment: YARN-10140-branch-2.10.002.patch
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: YARN-10140-branch-2.10.001.patch,
> YARN-10140-branch-2.10.002.patch
>
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
>
[jira] [Commented] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040497#comment-17040497
]
Ahmed Hussein commented on YARN-10140:
--
I discussed the solution with [~daryn]. Since the code is very host, it won't
be optimum to call a synchronized block and check whether realm is set or not.
Therefore, I decide to change my fix: after the initialization of the
{{testMiniKDC}}, I reset {{kerberosName.defaultRealm}}.
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: YARN-10140-branch-2.10.001.patch
>
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
>
[jira] [Commented] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040474#comment-17040474
]
Hadoop QA commented on YARN-10140:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.10 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
3s{color} | {color:green} branch-2.10 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
25s{color} | {color:green} branch-2.10 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
35s{color} | {color:green} branch-2.10 passed with JDK v1.8.0_242 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
47s{color} | {color:green} branch-2.10 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} branch-2.10 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} branch-2.10 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} branch-2.10 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} branch-2.10 passed with JDK v1.8.0_242 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
41s{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
38s{color} | {color:green} the patch passed with JDK v1.8.0_242 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
43s{color} | {color:green} root: The patch generated 0 new + 10 unchanged - 3
fixed = 10 total (was 13) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed with JDK v1.8.0_242 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m
38s{color} | {color:green} hadoop-auth in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
21s{color} | {color:green} hadoop-yarn-server-tests in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License
[jira] [Commented] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040417#comment-17040417
]
Jim Brennan commented on YARN-10140:
Would it be better to just pull back HADOOP-9888?
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: YARN-10140-branch-2.10.001.patch
>
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
>
[jira] [Commented] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040409#comment-17040409
]
Ahmed Hussein commented on YARN-10140:
--
The bug was caused by {{KerberosName.defaultRealm}} which is a static variable
assigned only once. When it gets set when the class is loaded, the
{{KerberosUtil.getDefaultRealm()}} throws an exception implying that
{{KerberosName.defaultRealm}} becomes an empty string. Moving the access of
that field into a getter delays setting the field until {{KerberosUtil}} is
ready.
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: YARN-10140-branch-2.10.001.patch
>
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
[jira] [Updated] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein updated YARN-10140:
-
Attachment: YARN-10140-branch-2.10.001.patch
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
> Attachments: YARN-10140-branch-2.10.001.patch
>
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
>
[jira] [Assigned] (YARN-10140) TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
[
https://issues.apache.org/jira/browse/YARN-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein reassigned YARN-10140:
Assignee: Ahmed Hussein
> TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10
> --
>
> Key: YARN-10140
> URL: https://issues.apache.org/jira/browse/YARN-10140
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
>Affects Versions: 2.10.0
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
>
> For branch-2.10, it seems that {{TestTimelineAuthFilterForV2}} was broken for
> some time.
> Using {{git bisect}} tool, I found that the first bad commit is
> "{{a3470c65d8b4e205c8a16d0c0b8dad10d0134bb8}}" in HADOOP-15959 .
> For trunk, the JUnit passes without failures.
> The stack trace is:
> {code:bash}
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 20.231 s <<< FAILURE! - in
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2
> [ERROR]
> testPutTimelineEntities[0](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 0.877 s <<< ERROR!
> org.apache.hadoop.security.KerberosAuthException: Login failure for user:
> HTTP/localh...@example.com from keytab
> /home/ahussein/workspace/repos/amahadoop-yhadoop-3090/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/target/bf6554f1-1df4-4ce0-8516-e6c337067870
> javax.security.auth.login.LoginException:
> java.lang.IllegalArgumentException: Illegal principal name
> HTTP/localh...@example.com:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to HTTP/localh...@example.com
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1104)
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:312)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.initialize(TestTimelineAuthFilterForV2.java:209)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
>
[jira] [Commented] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[
https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040355#comment-17040355
]
Hadoop QA commented on YARN-10141:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
37s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 4m
23s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
36s{color} | {color:red} hadoop-yarn-server-router in trunk failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
23s{color} | {color:red} hadoop-yarn-server-router in trunk failed. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 47 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
2s{color} | {color:red} The patch 400 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 37s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
34s{color} | {color:green} hadoop-yarn-server-router in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
30s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10141 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993921/YARN-10141.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 7def86655371 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 3f1aad0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/25540/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-YARN-Build/25540/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt
|
| whitespace |
[jira] [Updated] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10141: Attachment: YARN-10141.002.patch > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10124) Remove restriction of ParentQueue capacity zero when childCapacities > 0
[ https://issues.apache.org/jira/browse/YARN-10124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040271#comment-17040271 ] Prabhu Joseph commented on YARN-10124: -- Below are the observations after testing the patch with setting a parent queue to 0% capacity. 1. Leaf queues under the parent queue which is set with 0% capacity won't accept any new apps. The already running apps will get resources based on the queue max capacity. 2. When there is a used resource in the queue, then it is above its hard limit 0%. 2. The used resources is preemptible when the other queue has demand and under the configured capacity, cluster is full. 3. Root queue cannot be set to 0%, it will be always 100%. > Remove restriction of ParentQueue capacity zero when childCapacities > 0 > > > Key: YARN-10124 > URL: https://issues.apache.org/jira/browse/YARN-10124 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-10124-001.patch > > > ParentQueue capacity cannot be set to 0 when child capacities > 0. To disable > a parent queue temporarily, user can only STOP the queue but the capacity of > the queue cannot be used for other queues. Allowing 0 capacity for parent > queue will allow user to use the capacity for other queues and also to retain > the child queue capacity values. (else user has to set all child queue > capacities to 0) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10043) FairOrderingPolicy Improvements
[ https://issues.apache.org/jira/browse/YARN-10043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040219#comment-17040219 ] Manikandan R commented on YARN-10043: - [~snemeth] Can you review the latest patch? > FairOrderingPolicy Improvements > --- > > Key: YARN-10043 > URL: https://issues.apache.org/jira/browse/YARN-10043 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > Attachments: YARN-10043.001.patch, YARN-10043.002.patch, > YARN-10043.003.patch, YARN-10043.004.patch > > > FairOrderingPolicy can be improved by using some of the approaches (only > relevant) implemented in FairSharePolicy of FS. This improvement has > significance in FS to CS migration context. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9593) Updating scheduler conf with comma in config value fails
[
https://issues.apache.org/jira/browse/YARN-9593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G reassigned YARN-9593:
-
Assignee: Tanu Ajmera
> Updating scheduler conf with comma in config value fails
>
>
> Key: YARN-9593
> URL: https://issues.apache.org/jira/browse/YARN-9593
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.9.0, 3.0.0, 3.2.0, 3.1.2
>Reporter: Anthony Hsu
>Assignee: Tanu Ajmera
>Priority: Major
>
> For example:
> {code:java}
> $ yarn schedulerconf -update "root.gridops:acl_administer_queue=user1,user2
> group1,group2"
> Specify configuration key value as confKey=confVal.{code}
> This fails because there is a comma in the config value and the SchedConfCLI
> splits on comma first, expecting each split to a k=v pair.
> {noformat}
> void globalUpdates(String args, SchedConfUpdateInfo updateInfo) {
> if (args == null) {
> return;
> }
> HashMap globalUpdates = new HashMap<>();
> for (String globalUpdate : args.split(",")) {
> putKeyValuePair(globalUpdates, globalUpdate);
> }
> updateInfo.setGlobalParams(globalUpdates);
> }{noformat}
> Cc: [~jhung]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040151#comment-17040151
]
Hudson commented on YARN-10147:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17967 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17967/])
YARN-10147. FPGA plugin can't find the localized aocx file. Contributed
(snemeth: rev 3f1aad05f0510d713bdf1297de6cda380c2bb078)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/IntelFpgaOpenclPlugin.java
* (add)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/resourceplugin/fpga/TestIntelFpgaOpenclPlugin.java
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch,
> YARN-10147-003.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[
https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040138#comment-17040138
]
Adam Antal commented on YARN-10148:
---
Thanks for the patch [~kmarton]!
As for the queue names, do they have some connection between them? I mean C and
C1 is not very descriptive in that sense (maybe foo/bar or A/B queues would be
better, but I'm also fine if you keep as it is).
Also I think it's dangerous to use Queue C and C1 in the new test cases, as
ACLsTestBase also has a "queueC":
{code:java}
protected static final String QUEUEC = "queueC";
{code}
and it just messes things up to name that the same way - e.g. in
{{TestCapacitySchedulerQueueACLs}} class:
{code:java}
@Override
public void updateConfigWithCAndC1Queues(String rootACL, String queueCAcl,
String queueC1Acl) throws IOException {
CapacitySchedulerConfiguration csConf =
(CapacitySchedulerConfiguration) getConf();
csConf.clear();
csConf.setQueues(CapacitySchedulerConfiguration.ROOT,
new String[] {QUEUE_C, QUEUEA, QUEUEB});
...
{code}
Here we mix the queues from different test cases.
Some indentation seems a bit strange to me, but locally running the checkstyle
plugin also did not report them. What I'm specifically thinking:
QueueACLsTestBase#L59-60:
{code:java}
abstract public void updateConfigWithCAndC1Queues(String rootACL,
String queueCAcl, String queueC1Acl) throws IOException;
{code}
QueueACLsTestBase#L137-138:
{code:java}
private void checkAccess(boolean rootAccess, boolean cAccess,
boolean c1Access) throws IOException {
{code}
and all along {{TestFairSchedulerQueueACLs#updateConfigWithCAndC1Queues}}.
Could you take a look at them?
I'd be more informative to have a failure message in
{{QueueACLTestBase#checkAccess}} 's assertEquals statements.
Why were the {{TestAbsoluteResourceConfiguration}} failure messages modified?
(I suppose they were auto-modified because the changes in
{{TestCapacitySchedulerQueueACLs}}). I think they were good, let's take it back.
I found GenericTestUtils#SYSPROP_TEST_DATA_DIR constant for "test.build.data",
you can also that in {{TestFairSchedulerQueueACLs}}.
> Add Unit test for queue ACL for both FS and CS
> --
>
> Key: YARN-10148
> URL: https://issues.apache.org/jira/browse/YARN-10148
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Major
> Attachments: YARN-10148.001.patch, YARN-10148.002.patch
>
>
> Add some unit tests covering the queue ACL evaluation for both FS and CS.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040124#comment-17040124
]
Szilard Nemeth commented on YARN-10147:
---
Thanks [~pbacsko],
Latest patch looks good to me, committed to trunk.
Thanks [~adam.antal] for the review.
[~pbacsko]: Could you please check if it is feasible to backport this patch to
branch-3.2?
Thanks
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch,
> YARN-10147-003.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-10147:
--
Fix Version/s: 3.3.0
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch,
> YARN-10147-003.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040101#comment-17040101
]
Szilard Nemeth commented on YARN-10130:
---
Hi [~adam.antal],
Thanks for this patch.
Some comments:
1. Nit: FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls:
Should be renamed to "checkOutputDirDoesNotContainXml"
2. Would FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls
work correctly if the String "yarnSiteXmlFile" contains a relative name only? I
mean if -y yarn-site.xml is provided, I'm not sure if the following code block
will work correctly:
{code}
File xmlFile = new File(yarnSiteXmlFile);
File xmlParentFolder = xmlFile.getParentFile();
{code}
In this case, I'm not sure about the value of xmlParentFolder, but maybe I'm
wrong.
3. In FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls, when
you throw the IllegalArgumentException, you should use yet another static final
string and not the string literal as you used static string for a different
case in checkFileNotInOutputDir. This way, the code could be more consistent.
4. Nit: Comment:
// check whether the output folder does not contain not yarn-site.xml
// neither capacity-scheduler.xml
Should be: "... nor yarn-site.xml neither..."
5. In checkFileNotInOutputDir: Variable name xmlChild is a bit weird :)
6. In general, I don't get what's going on:
You first call checkOutputDirDoesNotContainsXmls and the first code block
checks if yarn-site.xml can be found in the output directory.
Then you call checkFileNotInOutputDir, once with YARN_SITE_CONFIGURATION_FILE,
then with CS_CONFIGURATION_FILE.
Aren't you checking yarn-site.xml twice, unnecessarily?
Consequently, I don't get what this testcase does:
TestFSConfigToCSConfigArgumentHandler#testYarnSiteOptionInOutputFolder
I think the other 2 testcases are enough.
7. I think the code that throws the exception in checkFileNotInOutputDir has
parameters in a wrong order:
1st param should be: CliOption.OUTPUT_DIR.name
2nd param should be: output
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
>
[jira] [Comment Edited] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040101#comment-17040101
]
Szilard Nemeth edited comment on YARN-10130 at 2/19/20 2:36 PM:
Hi [~adam.antal],
Thanks for this patch.
Some comments:
1. Nit: FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls:
Should be renamed to "checkOutputDirDoesNotContainXml"
2. Would FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls
work correctly if the String "yarnSiteXmlFile" contains a relative name only? I
mean if -y yarn-site.xml is provided, I'm not sure if the following code block
will work correctly:
{code}
File xmlFile = new File(yarnSiteXmlFile);
File xmlParentFolder = xmlFile.getParentFile();
{code}
In this case, I'm not sure about the value of xmlParentFolder, but maybe I'm
wrong.
3. In FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls, when
you throw the IllegalArgumentException, you should use yet another static final
string and not the string literal as you used static string for a different
case in checkFileNotInOutputDir. This way, the code could be more consistent.
4. Nit: Comment:
{code:java}
// check whether the output folder does not contain not yarn-site.xml
// neither capacity-scheduler.xml
{code}
Should be: "... nor yarn-site.xml neither..."
5. In checkFileNotInOutputDir: Variable name xmlChild is a bit weird :)
6. In general, I don't get what's going on:
You first call checkOutputDirDoesNotContainsXmls and the first code block
checks if yarn-site.xml can be found in the output directory.
Then you call checkFileNotInOutputDir, once with YARN_SITE_CONFIGURATION_FILE,
then with CS_CONFIGURATION_FILE.
Aren't you checking yarn-site.xml twice, unnecessarily?
Consequently, I don't get what this testcase does:
TestFSConfigToCSConfigArgumentHandler#testYarnSiteOptionInOutputFolder
I think the other 2 testcases are enough.
7. I think the code that throws the exception in checkFileNotInOutputDir has
parameters in a wrong order:
1st param should be: CliOption.OUTPUT_DIR.name
2nd param should be: output
was (Author: snemeth):
Hi [~adam.antal],
Thanks for this patch.
Some comments:
1. Nit: FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls:
Should be renamed to "checkOutputDirDoesNotContainXml"
2. Would FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls
work correctly if the String "yarnSiteXmlFile" contains a relative name only? I
mean if -y yarn-site.xml is provided, I'm not sure if the following code block
will work correctly:
{code}
File xmlFile = new File(yarnSiteXmlFile);
File xmlParentFolder = xmlFile.getParentFile();
{code}
In this case, I'm not sure about the value of xmlParentFolder, but maybe I'm
wrong.
3. In FSConfigToCSConfigArgumentHandler#checkOutputDirDoesNotContainsXmls, when
you throw the IllegalArgumentException, you should use yet another static final
string and not the string literal as you used static string for a different
case in checkFileNotInOutputDir. This way, the code could be more consistent.
4. Nit: Comment:
// check whether the output folder does not contain not yarn-site.xml
// neither capacity-scheduler.xml
Should be: "... nor yarn-site.xml neither..."
5. In checkFileNotInOutputDir: Variable name xmlChild is a bit weird :)
6. In general, I don't get what's going on:
You first call checkOutputDirDoesNotContainsXmls and the first code block
checks if yarn-site.xml can be found in the output directory.
Then you call checkFileNotInOutputDir, once with YARN_SITE_CONFIGURATION_FILE,
then with CS_CONFIGURATION_FILE.
Aren't you checking yarn-site.xml twice, unnecessarily?
Consequently, I don't get what this testcase does:
TestFSConfigToCSConfigArgumentHandler#testYarnSiteOptionInOutputFolder
I think the other 2 testcases are enough.
7. I think the code that throws the exception in checkFileNotInOutputDir has
parameters in a wrong order:
1st param should be: CliOption.OUTPUT_DIR.name
2nd param should be: output
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[ https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040090#comment-17040090 ] Kinga Marton commented on YARN-10148: - The test failures are unrelated. > Add Unit test for queue ACL for both FS and CS > -- > > Key: YARN-10148 > URL: https://issues.apache.org/jira/browse/YARN-10148 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Attachments: YARN-10148.001.patch, YARN-10148.002.patch > > > Add some unit tests covering the queue ACL evaluation for both FS and CS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10130) FS-CS converter: Do not allow output dir to be the same as input dir
[
https://issues.apache.org/jira/browse/YARN-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-10130:
--
Summary: FS-CS converter: Do not allow output dir to be the same as input
dir (was: Do not allow output dir to be the same as input dir)
> FS-CS converter: Do not allow output dir to be the same as input dir
>
>
> Key: YARN-10130
> URL: https://issues.apache.org/jira/browse/YARN-10130
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Szilard Nemeth
>Assignee: Adam Antal
>Priority: Major
> Attachments: YARN-10130.001.patch, YARN-10130.002.patch,
> YARN-10130.003.patch
>
>
> If the input dir where fair-scheduler.xml / yarn-site.xml sits is the same as
> the output dir (defined by the -o switch), the fs2cs tool overwrites the
> source config files, i.e. yarn-site.xml.
> Reproduce this is easy, just run fs2cs tool with this command:
> {code:java}
> /bin/yarn fs2cs --cluster-resource memory-mb=18044928,vcores=16
> --no-terminal-rule-check -y yarn-site.xml -f fair-scheduler.xml -o .
> {code}
> The following (or similar) is emitted by the tool:
> {code:java}
> WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.WARNING: YARN_OPTS has been replaced by HADOOP_OPTS. Using value of
> YARN_OPTS.20/02/10 12:51:42 INFO converter.FSConfigToCSConfigConverter:
> Output directory for yarn-site.xml and capacity-scheduler.xml is: .20/02/10
> 12:51:42 INFO converter.FSConfigToCSConfigConverter: Conversion rules file is
> not defined, using default conversion config!20/02/10 12:51:42 ERROR
> conf.Configuration: error parsing conf
> yarn-site.xmlcom.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog at
> [row,col,system-id]: [1,0,"yarn-site.xml"] at
> com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:687) at
> com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2220) at
> com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2126)
> at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181) at
> org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3343)
> at
> org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3137) at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3030) at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2996)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2871) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1389) at
> org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) at
> org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1702) at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.createConfiguration(FSConfigToCSConfigConverter.java:166)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverter.convert(FSConfigToCSConfigConverter.java:98)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigArgumentHandler.parseAndConvert(FSConfigToCSConfigArgumentHandler.java:137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.converter.FSConfigToCSConfigConverterMain.main(FSConfigToCSConfigConverterMain.java:40)20/02/10
> 12:51:42 ERROR converter.FSConfigToCSConfigConverterMain: Error while
> starting FS configuration conversion!
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039956#comment-17039956
]
Hadoop QA commented on YARN-10147:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 24m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 4s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 57s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
48s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}100m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10147 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993872/YARN-10147-003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux ad75091647cd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / cb3f3cc |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25539/testReport/ |
| Max. process+thread count | 414 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/25539/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> FPGA plugin can't find the localized aocx file
>
[jira] [Commented] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[
https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039949#comment-17039949
]
Hadoop QA commented on YARN-10148:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
47s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 59s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 2s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue} 0m
47s{color} | {color:blue} ASF License check generated no output? {color} |
| {color:black}{color} | {color:black} {color} | {color:black}146m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService
|
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | YARN-10148 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12993868/YARN-10148.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f05a3fd56d62 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / cb3f3cc |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/25538/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/25538/testReport/ |
| Max. process+thread count | 839 (vs. ulimit of 5500) |
| modules | C:
[jira] [Commented] (YARN-10143) YARN-10101 broke Yarn logs CLI
[
https://issues.apache.org/jira/browse/YARN-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039880#comment-17039880
]
Kinga Marton commented on YARN-10143:
-
Hi [~adam.antal], the patch LGTM, if you will test it with tFile as well and it
will work, than I give +1. (non-binding)
> YARN-10101 broke Yarn logs CLI
> ---
>
> Key: YARN-10143
> URL: https://issues.apache.org/jira/browse/YARN-10143
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.3.0, 3.2.2
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Blocker
> Attachments: YARN-10143.001.patch, YARN-10143.002.patch
>
>
> YARN-10101 broke the Yarn logs CLI.
> In {{LogsCLI#359}} a {{ContainerLogsRequest}} has been created with null set
> as appAttemptId, while the {{LogAggregationFileController}} instances are
> configured badly to handle this case. The new JHS API works as expected for
> defined application attempt, but we should fix this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10147) FPGA plugin can't find the localized aocx file
[
https://issues.apache.org/jira/browse/YARN-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10147:
Attachment: YARN-10147-003.patch
> FPGA plugin can't find the localized aocx file
> --
>
> Key: YARN-10147
> URL: https://issues.apache.org/jira/browse/YARN-10147
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10147-001.patch, YARN-10147-002.patch,
> YARN-10147-003.patch
>
>
> There's a bug in the FPGA plugin which is intended to find the localized
> "aocx" file:
> {noformat}
> ...
> if (localizedResources != null) {
> Optional aocxPath = localizedResources
> .keySet()
> .stream()
> .filter(path -> matchesIpid(path, id))
> .findFirst();
> if (aocxPath.isPresent()) {
> ipFilePath = aocxPath.get().toUri().toString();
> LOG.debug("Found: " + ipFilePath);
> }
> } else {
> LOG.warn("Localized resource is null!");
> }
> return ipFilePath;
> }
> private boolean matchesIpid(Path p, String id) {
> return p.getName().toLowerCase().equals(id.toLowerCase())
> && p.getName().endsWith(".aocx");
> }
> {noformat}
> The method {{matchesIpid()}} works incorrecty: the {{id}} argument is the
> expected filename, but without the extension. Therefore the {{equals()}}
> comparison will always be false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10148) Add Unit test for queue ACL for both FS and CS
[ https://issues.apache.org/jira/browse/YARN-10148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton updated YARN-10148: Attachment: YARN-10148.002.patch > Add Unit test for queue ACL for both FS and CS > -- > > Key: YARN-10148 > URL: https://issues.apache.org/jira/browse/YARN-10148 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Attachments: YARN-10148.001.patch, YARN-10148.002.patch > > > Add some unit tests covering the queue ACL evaluation for both FS and CS. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10124) Remove restriction of ParentQueue capacity zero when childCapacities > 0
[ https://issues.apache.org/jira/browse/YARN-10124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039817#comment-17039817 ] Wilfred Spiegelenburg edited comment on YARN-10124 at 2/19/20 9:11 AM: --- When you set a parent queue to a 0 capacity could that not leave the apps in the leaf queues below it in a state that they are not scheduled? How does it affect the capacity calculation, i.e. is the queue now always below or above its (hard)limits? I am not against removing the limitation but I am thinking about the side effects it could have. was (Author: wilfreds): When you set a parent queue to a 0 capacity could that not leave the apps in the leaf queues below it in a state that they are not scheduled? How does it affect the capacity calculation, i.e. is the queue now always below or above its (hard)limits? > Remove restriction of ParentQueue capacity zero when childCapacities > 0 > > > Key: YARN-10124 > URL: https://issues.apache.org/jira/browse/YARN-10124 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-10124-001.patch > > > ParentQueue capacity cannot be set to 0 when child capacities > 0. To disable > a parent queue temporarily, user can only STOP the queue but the capacity of > the queue cannot be used for other queues. Allowing 0 capacity for parent > queue will allow user to use the capacity for other queues and also to retain > the child queue capacity values. (else user has to set all child queue > capacities to 0) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10124) Remove restriction of ParentQueue capacity zero when childCapacities > 0
[ https://issues.apache.org/jira/browse/YARN-10124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039817#comment-17039817 ] Wilfred Spiegelenburg commented on YARN-10124: -- When you set a parent queue to a 0 capacity could that not leave the apps in the leaf queues below it in a state that they are not scheduled? How does it affect the capacity calculation, i.e. is the queue now always below or above its (hard)limits? > Remove restriction of ParentQueue capacity zero when childCapacities > 0 > > > Key: YARN-10124 > URL: https://issues.apache.org/jira/browse/YARN-10124 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.3.0 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: YARN-10124-001.patch > > > ParentQueue capacity cannot be set to 0 when child capacities > 0. To disable > a parent queue temporarily, user can only STOP the queue but the capacity of > the queue cannot be used for other queues. Allowing 0 capacity for parent > queue will allow user to use the capacity for other queues and also to retain > the child queue capacity values. (else user has to set all child queue > capacities to 0) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10145) Error in creating hbase tables
[
https://issues.apache.org/jira/browse/YARN-10145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039811#comment-17039811
]
yehuanhuan commented on YARN-10145:
---
The error does not occur when building with hbase2.0. To use HBase 2.0.0-beta1,
build from source with option -Dhbase.profile=2.0
> Error in creating hbase tables
> --
>
> Key: YARN-10145
> URL: https://issues.apache.org/jira/browse/YARN-10145
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2
>Affects Versions: 3.2.1
>Reporter: yehuanhuan
>Priority: Major
>
> When using the command(bin/hadoop
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator
> -create) to create the timeline service schema，I get the following error.
> {panel:title=ERROR}
> 2020-02-18 17:16:53,694 ERROR storage.TimelineSchemaCreator: Error in
> creating hbase tables:
> org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError:
> tried to access method shaded.com.google.common.base.Stopwatch.()V from
> class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:212)
> at
> org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)
> at
> org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:162)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:797)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:406)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.entity.EntityTableRW.createTable(EntityTableRW.java:87)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllTables(TimelineSchemaCreator.java:308)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.createAllSchemas(TimelineSchemaCreator.java:278)
> at
> org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator.main(TimelineSchemaCreator.java:147)
> Caused by: java.lang.IllegalAccessError: tried to access method
> shaded.com.google.common.base.Stopwatch.()V from class
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:604)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:588)
> at
> org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:561)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:61)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateMeta(ConnectionManager.java:1211)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1178)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:305)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:156)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:60)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
> {panel}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
