[jira] [Commented] (YARN-11256) buildCommandExecutor function leverages redundant params
[
https://issues.apache.org/jira/browse/YARN-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598200#comment-17598200
]
ASF GitHub Bot commented on YARN-11256:
---
hadoop-yetus commented on PR #4831:
URL: https://github.com/apache/hadoop/pull/4831#issuecomment-1232522166
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 22s | | trunk passed |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 54s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 26s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 1m 30s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 30s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 29s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 41 unchanged - 1 fixed = 41 total (was 42) |
| +1 :green_heart: | mvnsite | 0m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 33s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 23m 23s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 129m 46s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4831/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4831 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f633b55122c6 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2e86271a5d36912ee1a401cf6f2451a4adaeb8d1 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4831/2/testReport/ |
| Max. process+thread count | 527 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-
[jira] [Commented] (YARN-11289) [Federation] Improve NM FederationInterceptor removeAppFromRegistry
[ https://issues.apache.org/jira/browse/YARN-11289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598178#comment-17598178 ] ASF GitHub Bot commented on YARN-11289: --- slfan1989 opened a new pull request, #4836: URL: https://github.com/apache/hadoop/pull/4836 YARN-11289. [Federation] Improve NM FederationInterceptor removeAppFromRegistry. 1.FederationInterceptor#finishApplicationMaster needs to check getFinalApplicationStatus, and clean up when the status is SUCCESS. 2. When FederationInterceptor#shutdown, perform CleanUp on Application. > [Federation] Improve NM FederationInterceptor removeAppFromRegistry > --- > > Key: YARN-11289 > URL: https://issues.apache.org/jira/browse/YARN-11289 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > > [Federation] Improve NM FederationInterceptor removeAppFromRegistry > 1.FederationInterceptor#finishApplicationMaster needs to check > getFinalApplicationStatus, and clean up when the status is SUCCESS. > 2. When FederationInterceptor#shutdown, perform CleanUp on Application. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11289) [Federation] Improve NM FederationInterceptor removeAppFromRegistry
[ https://issues.apache.org/jira/browse/YARN-11289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11289: -- Labels: pull-request-available (was: ) > [Federation] Improve NM FederationInterceptor removeAppFromRegistry > --- > > Key: YARN-11289 > URL: https://issues.apache.org/jira/browse/YARN-11289 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > [Federation] Improve NM FederationInterceptor removeAppFromRegistry > 1.FederationInterceptor#finishApplicationMaster needs to check > getFinalApplicationStatus, and clean up when the status is SUCCESS. > 2. When FederationInterceptor#shutdown, perform CleanUp on Application. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11289) [Federation] Improve NM FederationInterceptor removeAppFromRegistry
fanshilun created YARN-11289: Summary: [Federation] Improve NM FederationInterceptor removeAppFromRegistry Key: YARN-11289 URL: https://issues.apache.org/jira/browse/YARN-11289 Project: Hadoop YARN Issue Type: Improvement Components: federation, nodemanager Affects Versions: 3.4.0 Reporter: fanshilun Assignee: fanshilun [Federation] Improve NM FederationInterceptor removeAppFromRegistry 1.FederationInterceptor#finishApplicationMaster needs to check getFinalApplicationStatus, and clean up when the status is SUCCESS. 2. When FederationInterceptor#shutdown, perform CleanUp on Application. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598163#comment-17598163
]
ASF GitHub Bot commented on YARN-11241:
---
hadoop-yetus commented on PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#issuecomment-1232446408
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 48s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 48s | | trunk passed |
| +1 :green_heart: | compile | 10m 50s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 11s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 21s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 12s | | the patch passed |
| +1 :green_heart: | compile | 10m 9s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 10m 9s | | the patch passed |
| +1 :green_heart: | compile | 9m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 9m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 59s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 286 unchanged - 2
fixed = 286 total (was 288) |
| +1 :green_heart: | mvnsite | 3m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 5s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 0s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 28s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 19s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 25m 16s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 1m 9s | | The patch does not
generate ASF License warnings. |
| | | 207m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4703/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4703 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 3002ead4d5a5 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 02c16cdd8e142bab58887ec3a4f3d7748398afb2 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
[jira] [Commented] (YARN-11272) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk
[ https://issues.apache.org/jira/browse/YARN-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598104#comment-17598104 ] ASF GitHub Bot commented on YARN-11272: --- slfan1989 commented on PR #4781: URL: https://github.com/apache/hadoop/pull/4781#issuecomment-1232252531 @goiri Can you help to merge this pr into trunk branch? I will continue to follow up on [YARN-11273](https://issues.apache.org/jira/browse/YARN-11273), which requires part of the pr junit test. Thank you very much! > [RESERVATION] Federation StateStore: Support storage/retrieval of > Reservations With Zk > -- > > Key: YARN-11272 > URL: https://issues.apache.org/jira/browse/YARN-11272 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, reservation system >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11256) buildCommandExecutor function leverages redundant params
[
https://issues.apache.org/jira/browse/YARN-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598031#comment-17598031
]
ASF GitHub Bot commented on YARN-11256:
---
hadoop-yetus commented on PR #4831:
URL: https://github.com/apache/hadoop/pull/4831#issuecomment-1232086678
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 57s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 1m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 28s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4831/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 3 new + 41 unchanged - 1 fixed = 44 total (was 42) |
| +1 :green_heart: | mvnsite | 0m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 0s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 130m 32s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4831/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4831 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e29e08aac8ea 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3b0ad9d0e0ae247b203894793263849b31bddd7c |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.
[jira] [Commented] (YARN-10965) Centralize queue resource calculation based on CapacityVectors
[
https://issues.apache.org/jira/browse/YARN-10965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598003#comment-17598003
]
ASF GitHub Bot commented on YARN-10965:
---
hadoop-yetus commented on PR #3470:
URL: https://github.com/apache/hadoop/pull/3470#issuecomment-1231999621
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 33s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 19s | | trunk passed |
| +1 :green_heart: | compile | 11m 48s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 17s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 5s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 29s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 55s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 38s | | the patch passed |
| +1 :green_heart: | compile | 9m 5s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 9m 5s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/16/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 2 new +
738 unchanged - 3 fixed = 740 total (was 741) |
| +1 :green_heart: | compile | 8m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 39s | |
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new
+ 651 unchanged - 1 fixed = 651 total (was 652) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 49s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/16/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 92 new + 150 unchanged
- 0 fixed = 242 total (was 150) |
| +1 :green_heart: | mvnsite | 2m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 16s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 58s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 36s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 99m 16s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 1m 9s | | The patch does not
generate ASF License warnings. |
| | | 259m 34s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker
[jira] [Updated] (YARN-11256) buildCommandExecutor function leverages redundant params
[
https://issues.apache.org/jira/browse/YARN-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-11256:
--
Labels: pull-request-available (was: )
> buildCommandExecutor function leverages redundant params
> -
>
> Key: YARN-11256
> URL: https://issues.apache.org/jira/browse/YARN-11256
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Reporter: Samrat Deb
>Assignee: Samrat Deb
>Priority: Minor
> Labels: pull-request-available
>
> Current signature of `buildCommandExecutor` passes redundant params which can
> be reduced by passing container reference
>
> {code:java}
> protected CommandExecutor buildCommandExecutor(String wrapperScriptPath,
> String containerIdStr, String user, Path pidFile, Resource resource,
> File workDir, Map environment){code}
>
> - `containerIdStr` can be derived from
> `container.getContainerId().toString()`
> - `resource` can be derived from `container.getResource()`
> - `environment` can be derived from container reference
> this will allow extention of the function without constraining to paramters
> of the method
> Container reference is safe as it is pass by reference it wont copy the
> entire the container object
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11256) buildCommandExecutor function leverages redundant params
[
https://issues.apache.org/jira/browse/YARN-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597992#comment-17597992
]
ASF GitHub Bot commented on YARN-11256:
---
Samrat002 opened a new pull request, #4831:
URL: https://github.com/apache/hadoop/pull/4831
### Description of PR
[YARN-11256](https://issues.apache.org/jira/browse/YARN-11256)
### How was this patch tested?
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [x] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> buildCommandExecutor function leverages redundant params
> -
>
> Key: YARN-11256
> URL: https://issues.apache.org/jira/browse/YARN-11256
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Reporter: Samrat Deb
>Assignee: Samrat Deb
>Priority: Minor
>
> Current signature of `buildCommandExecutor` passes redundant params which can
> be reduced by passing container reference
>
> {code:java}
> protected CommandExecutor buildCommandExecutor(String wrapperScriptPath,
> String containerIdStr, String user, Path pidFile, Resource resource,
> File workDir, Map environment){code}
>
> - `containerIdStr` can be derived from
> `container.getContainerId().toString()`
> - `resource` can be derived from `container.getResource()`
> - `environment` can be derived from container reference
> this will allow extention of the function without constraining to paramters
> of the method
> Container reference is safe as it is pass by reference it wont copy the
> entire the container object
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11273) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With SQL
[
https://issues.apache.org/jira/browse/YARN-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597972#comment-17597972
]
ASF GitHub Bot commented on YARN-11273:
---
hadoop-yetus commented on PR #4817:
URL: https://github.com/apache/hadoop/pull/4817#issuecomment-1231900198
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 12s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 26s | | trunk passed |
| +1 :green_heart: | compile | 9m 49s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 8m 35s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 7m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 51s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 4m 6s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 14m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 32s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 47s | | the patch passed |
| +1 :green_heart: | compile | 9m 14s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 14s | | the patch passed |
| +1 :green_heart: | compile | 8m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 46s | | the patch passed |
| +1 :green_heart: | mvnsite | 6m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 14s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 51s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 236m 44s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 3m 30s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 49s | |
hadoop-yarn-server-applicationhistoryservice in the patch passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not
generate ASF License warnings. |
| | | 444m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4817/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4817 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient xmllint spotbugs checkstyle |
| uname | Linux 62e201d0d544 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 10072c4835a1375ecd4bded9e17bc1eb629d52d5 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597966#comment-17597966
]
ASF GitHub Bot commented on YARN-11241:
---
hadoop-yetus commented on PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#issuecomment-1231879062
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 10s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 13s | | trunk passed |
| +1 :green_heart: | compile | 10m 34s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 0s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 22s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 58s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 0s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 3s | | the patch passed |
| +1 :green_heart: | compile | 9m 49s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 49s | | the patch passed |
| +1 :green_heart: | compile | 8m 59s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 59s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 53s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4703/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 286 unchanged
- 2 fixed = 287 total (was 288) |
| +1 :green_heart: | mvnsite | 3m 8s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 51s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 18s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 0s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 24m 29s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 204m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4703/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4703 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 7b3e5fe1dc42 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 04811718931f33ea2af27aebf8c5fc4a79be6256 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK
[jira] [Updated] (YARN-11285) LocalizedResources are leaked and its LocalPath are not cleared
[
https://issues.apache.org/jira/browse/YARN-11285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-11285:
-
Description:
LocalizedResources are leaked and its LocalPath are not cleared from NM Local
Directories.
Each container has separate LocalizedResource object and separate local path
like below.
{code}
/mnt/yarn/usercache/hive/filecache/6/2552419:
total 28456
-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
/mnt/yarn/usercache/hive/filecache/6/2552420:
total 28456
-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
/mnt/yarn/usercache/hive/filecache/6/2552421:
total 28456
-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
/mnt/yarn/usercache/hive/filecache/6/2552422:
total 28456
{code}
was:
LocalizedResources are leaked and its LocalPath are not cleared from NM Local
Directories. When multiple containers are initialized at same time,
LocalResourcesTrackerImpl REQUEST handler could create and handle multiple
LocalizedResource object for the same input path due to race condition in
[below
code|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourcesTrackerImpl.java#L149]
{code}
case REQUEST:
LocalResourceRequest req = event.getLocalResourceRequest();
LocalizedResource rsrc = localrsrc.get(req);
if (null == rsrc) {
rsrc = new LocalizedResource(req, dispatcher);
localrsrc.put(req, rsrc);
}
rsrc.handle(event);
{code}
Each container will have separate LocalizedResource object and separate local
path like below.
{code}
/mnt/yarn/usercache/hive/filecache/6/2552419:
total 28456
-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
/mnt/yarn/usercache/hive/filecache/6/2552420:
total 28456
-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
/mnt/yarn/usercache/hive/filecache/6/2552421:
total 28456
-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
/mnt/yarn/usercache/hive/filecache/6/2552422:
total 28456
{code}
> LocalizedResources are leaked and its LocalPath are not cleared
> ---
>
> Key: YARN-11285
> URL: https://issues.apache.org/jira/browse/YARN-11285
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.2.1
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> LocalizedResources are leaked and its LocalPath are not cleared from NM Local
> Directories.
> Each container has separate LocalizedResource object and separate local path
> like below.
> {code}
>/mnt/yarn/usercache/hive/filecache/6/2552419:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552420:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552421:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552422:
>total 28456
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11285) LocalizedResources are leaked and its LocalPath are not cleared
[
https://issues.apache.org/jira/browse/YARN-11285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated YARN-11285:
-
Attachment: (was: TestConcurrency.java)
> LocalizedResources are leaked and its LocalPath are not cleared
> ---
>
> Key: YARN-11285
> URL: https://issues.apache.org/jira/browse/YARN-11285
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.2.1
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> LocalizedResources are leaked and its LocalPath are not cleared from NM Local
> Directories.
> Each container has separate LocalizedResource object and separate local path
> like below.
> {code}
>/mnt/yarn/usercache/hive/filecache/6/2552419:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552420:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552421:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug 7 10:24
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552422:
>total 28456
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11177) Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation
[ https://issues.apache.org/jira/browse/YARN-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597919#comment-17597919 ] ASF GitHub Bot commented on YARN-11177: --- slfan1989 commented on PR #4764: URL: https://github.com/apache/hadoop/pull/4764#issuecomment-1231792111 @goiri Please help to review the code again, thank you very much! > Support getNewReservation, submitReservation, updateReservation, > deleteReservation API's for Federation > --- > > Key: YARN-11177 > URL: https://issues.apache.org/jira/browse/YARN-11177 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6667) Handle containerId duplicate without failing the heartbeat in Federation Interceptor
[ https://issues.apache.org/jira/browse/YARN-6667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597917#comment-17597917 ] ASF GitHub Bot commented on YARN-6667: -- slfan1989 commented on PR #4810: URL: https://github.com/apache/hadoop/pull/4810#issuecomment-1231789354 @goiri Please help to review the code again, Thank you very much! > Handle containerId duplicate without failing the heartbeat in Federation > Interceptor > > > Key: YARN-6667 > URL: https://issues.apache.org/jira/browse/YARN-6667 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Botong Huang >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > > From the actual situation, the probability of this happening is very low. > It can only be caused by the master-slave fail-hover of YARN and the wrong > Epoch parameter configuration. > We will try to be compatible with this situation and let the Application run > as much as possible, using the following measures: > 1. Select a node whose heartbeat does not time out for allocation, and at the > same time require the node to be in the RUNNING state. > 2. If the heartbeat of both RMs does not time out, and both are in the > RUNNING state, select the previously allocated RM for Container processing. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11284) [Federation] Improve UnmanagedAMPoolManager WithoutBlock ServiceStop
[
https://issues.apache.org/jira/browse/YARN-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597912#comment-17597912
]
ASF GitHub Bot commented on YARN-11284:
---
slfan1989 commented on PR #4814:
URL: https://github.com/apache/hadoop/pull/4814#issuecomment-1231777089
@goiri Please help to review the code again, thank you very much!
> [Federation] Improve UnmanagedAMPoolManager WithoutBlock ServiceStop
>
>
> Key: YARN-11284
> URL: https://issues.apache.org/jira/browse/YARN-11284
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> There is a todo in UnmanagedAMPoolManager#ServiceStop
> {code:java}
> TODO: move waiting for the kill to finish into a separate thread, without
> blocking the serviceStop. {code}
> I use a separate thread for this work, no longer Block blocking the
> serviceStop
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597889#comment-17597889
]
ASF GitHub Bot commented on YARN-11241:
---
hadoop-yetus commented on PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#issuecomment-1231704644
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 32s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 57s | | trunk passed |
| +1 :green_heart: | compile | 11m 41s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 10m 43s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 54s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 59s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 12s | | the patch passed |
| +1 :green_heart: | compile | 9m 56s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 56s | | the patch passed |
| +1 :green_heart: | compile | 8m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 50s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4703/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 286 unchanged
- 2 fixed = 287 total (was 288) |
| +1 :green_heart: | mvnsite | 3m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 11s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 4s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 24s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 43s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 24m 42s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| | | 210m 36s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4703/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4703 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux fc9032b63645 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7c52957074d0aefb5fc3a62bb48fc043c5cbf4cd |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK
[jira] [Commented] (YARN-11264) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests
[
https://issues.apache.org/jira/browse/YARN-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597887#comment-17597887
]
ASF GitHub Bot commented on YARN-11264:
---
hadoop-yetus commented on PR #4776:
URL: https://github.com/apache/hadoop/pull/4776#issuecomment-1231696828
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 9 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 33s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 53s | |
branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
no spotbugs output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 20m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 22s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4776/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-tests.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests:
The patch generated 12 new + 19 unchanged - 20 fixed = 31 total (was 39) |
| +1 :green_heart: | mvnsite | 0m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 25s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests has
no data from spotbugs |
| +1 :green_heart: | shadedclient | 19m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 33s | | hadoop-yarn-server-tests in the
patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 94m 41s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4776/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4776 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux 1aef96862822 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b579b2f2fda4d8c3198197e7791367e2c9198be8 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-o
[jira] [Commented] (YARN-11261) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy
[
https://issues.apache.org/jira/browse/YARN-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597882#comment-17597882
]
ASF GitHub Bot commented on YARN-11261:
---
hadoop-yetus commented on PR #4777:
URL: https://github.com/apache/hadoop/pull/4777#issuecomment-1231691151
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 5s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 21s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | compile | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 16s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4777/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-web-proxy.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy:
The patch generated 2 new + 31 unchanged - 31 fixed = 33 total (was 62) |
| +1 :green_heart: | mvnsite | 0m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 0m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 18s | | hadoop-yarn-server-web-proxy in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 100m 2s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4777/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4777 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux fb2af95a5313 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1b7fa27e6f007b9e0150698a6a62db3c45963992 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Res
[jira] [Updated] (YARN-11288) fix the invalid RPATH of container-executor
[ https://issues.apache.org/jira/browse/YARN-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 1783129...@qq.com updated YARN-11288: - Description: In https://issues.apache.org/jira/browse/YARN-10495 make the rpath of container-executor configurable, and default add ../lib/native to rpath. In fact, this path does not take effect because it is not an absolute path. > fix the invalid RPATH of container-executor > --- > > Key: YARN-11288 > URL: https://issues.apache.org/jira/browse/YARN-11288 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: 1783129...@qq.com >Priority: Major > Attachments: unfind_libcrypto.png > > > In https://issues.apache.org/jira/browse/YARN-10495 make the rpath of > container-executor configurable, and default add ../lib/native to rpath. In > fact, this path does not take effect because it is not an absolute path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11288) fix the invalid RPATH of container-executor
[ https://issues.apache.org/jira/browse/YARN-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 1783129...@qq.com updated YARN-11288: - Component/s: yarn > fix the invalid RPATH of container-executor > --- > > Key: YARN-11288 > URL: https://issues.apache.org/jira/browse/YARN-11288 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Reporter: 1783129...@qq.com >Priority: Major > Attachments: unfind_libcrypto.png > > > In https://issues.apache.org/jira/browse/YARN-10495 make the rpath of > container-executor configurable, and default add ../lib/native to rpath. In > fact, this path does not take effect because it is not an absolute path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11288) fix the invalid RPATH of container-executor
[ https://issues.apache.org/jira/browse/YARN-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 1783129...@qq.com updated YARN-11288: - Attachment: unfind_libcrypto.png > fix the invalid RPATH of container-executor > --- > > Key: YARN-11288 > URL: https://issues.apache.org/jira/browse/YARN-11288 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: 1783129...@qq.com >Priority: Major > Attachments: unfind_libcrypto.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597848#comment-17597848
]
ASF GitHub Bot commented on YARN-11241:
---
ashutoshcipher commented on code in PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#discussion_r958434672
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java:
##
@@ -234,31 +234,63 @@ private void verifyLocalFileDeletion(
// ensure filesystems were closed
verify(logAggregationService).closeFileSystems(
any(UserGroupInformation.class));
-List dirList = new ArrayList<>();
-dirList.add(new Path(app1LogDir.toURI()));
-verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
-delSrvc, user, null, dirList)));
-
-String containerIdStr = container11.toString();
-File containerLogDir = new File(app1LogDir, containerIdStr);
-int count = 0;
-int maxAttempts = 50;
-for (String fileType : new String[] { "stdout", "stderr", "syslog" }) {
- File f = new File(containerLogDir, fileType);
+boolean filesShouldBeDeleted =
+
this.conf.getBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP,
+YarnConfiguration.DEFAULT_LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP);
+if (filesShouldBeDeleted) {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
+ }
count = 0;
- while ((f.exists()) && (count < maxAttempts)) {
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
count++;
Thread.sleep(100);
}
- Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
-}
-count = 0;
-while ((app1LogDir.exists()) && (count < maxAttempts)) {
- count++;
- Thread.sleep(100);
+ Assert.assertFalse("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
+} else {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, never()).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertTrue("File [" + f + "] was not deleted", f.exists());
+ }
+ count = 0;
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
+count++;
+Thread.sleep(100);
+ }
+ Assert.assertTrue("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
Review Comment:
Yes, Makes sense. Addressed this in my latest commit.
> Add uncleaning option for local app log file with log-aggregation enabled
> -
>
> Key: YARN-11241
> URL: https://issues.apache.org/jira/browse/YARN-11241
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: log-aggregation
>Reporter: groot
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add uncleaning option for local app log file with log-aggregation enabled
> This will be helpful for debugging purpose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11288) fix the invalid RPATH of container-executor
1783129...@qq.com created YARN-11288: Summary: fix the invalid RPATH of container-executor Key: YARN-11288 URL: https://issues.apache.org/jira/browse/YARN-11288 Project: Hadoop YARN Issue Type: Improvement Reporter: 1783129...@qq.com -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6939) Upgrade JUnit from 4 to 5 in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597840#comment-17597840 ] Akira Ajisaka commented on YARN-6939: - After discussing with groot offline, now I'm okay with adding the junit-platform-launcher dependency for each PR. It will take long time to test locally. > Upgrade JUnit from 4 to 5 in hadoop-yarn > > > Key: YARN-6939 > URL: https://issues.apache.org/jira/browse/YARN-6939 > Project: Hadoop YARN > Issue Type: Test >Reporter: Akira Ajisaka >Assignee: groot >Priority: Major > > Feel free to create sub-tasks for each module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597839#comment-17597839
]
ASF GitHub Bot commented on YARN-11241:
---
aajisaka commented on code in PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#discussion_r958408381
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java:
##
@@ -234,31 +234,63 @@ private void verifyLocalFileDeletion(
// ensure filesystems were closed
verify(logAggregationService).closeFileSystems(
any(UserGroupInformation.class));
-List dirList = new ArrayList<>();
-dirList.add(new Path(app1LogDir.toURI()));
-verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
-delSrvc, user, null, dirList)));
-
-String containerIdStr = container11.toString();
-File containerLogDir = new File(app1LogDir, containerIdStr);
-int count = 0;
-int maxAttempts = 50;
-for (String fileType : new String[] { "stdout", "stderr", "syslog" }) {
- File f = new File(containerLogDir, fileType);
+boolean filesShouldBeDeleted =
+
this.conf.getBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP,
+YarnConfiguration.DEFAULT_LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP);
+if (filesShouldBeDeleted) {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
+ }
count = 0;
- while ((f.exists()) && (count < maxAttempts)) {
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
count++;
Thread.sleep(100);
}
- Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
-}
-count = 0;
-while ((app1LogDir.exists()) && (count < maxAttempts)) {
- count++;
- Thread.sleep(100);
+ Assert.assertFalse("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
+} else {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, never()).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertTrue("File [" + f + "] was not deleted", f.exists());
+ }
+ count = 0;
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
+count++;
+Thread.sleep(100);
+ }
+ Assert.assertTrue("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
Review Comment:
Thanks. Let's run `Thread.sleep(5000)` and check the directory and files
still exist. Now we don't need the 2 while loop.
> Add uncleaning option for local app log file with log-aggregation enabled
> -
>
> Key: YARN-11241
> URL: https://issues.apache.org/jira/browse/YARN-11241
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: log-aggregation
>Reporter: groot
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add uncleaning option for local app log file with log-aggregation enabled
> This will be helpful for debugging purpose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11264) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests
[ https://issues.apache.org/jira/browse/YARN-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597834#comment-17597834 ] ASF GitHub Bot commented on YARN-11264: --- ashutoshcipher commented on PR #4776: URL: https://github.com/apache/hadoop/pull/4776#issuecomment-1231582414 @aajisaka - As discussed, added junit-platform-launcher dependency > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests > - > > Key: YARN-11264 > URL: https://issues.apache.org/jira/browse/YARN-11264 > Project: Hadoop YARN > Issue Type: Sub-task > Components: test, yarn >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11261) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy
[ https://issues.apache.org/jira/browse/YARN-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597804#comment-17597804 ] ASF GitHub Bot commented on YARN-11261: --- ashutoshcipher commented on PR #4777: URL: https://github.com/apache/hadoop/pull/4777#issuecomment-1231572295 @aajisaka - As discussed, added `junit-platform-launcher` dependency > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy > - > > Key: YARN-11261 > URL: https://issues.apache.org/jira/browse/YARN-11261 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11177) Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation
[
https://issues.apache.org/jira/browse/YARN-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597803#comment-17597803
]
ASF GitHub Bot commented on YARN-11177:
---
hadoop-yetus commented on PR #4764:
URL: https://github.com/apache/hadoop/pull/4764#issuecomment-1231571867
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 16s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 57s | | trunk passed |
| +1 :green_heart: | compile | 5m 10s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 59s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 56s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 43s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 20s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 23s | | the patch passed |
| +1 :green_heart: | compile | 4m 49s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 4m 49s | | the patch passed |
| +1 :green_heart: | compile | 3m 50s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 3m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 1s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 52s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 6s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 108m 0s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 3m 48s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 263m 28s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4764/18/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4764 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux d67c4fb87ae9 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 276575beeb4b9361512805674319add383da3eef |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranc
[jira] [Commented] (YARN-11177) Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation
[
https://issues.apache.org/jira/browse/YARN-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597802#comment-17597802
]
ASF GitHub Bot commented on YARN-11177:
---
hadoop-yetus commented on PR #4764:
URL: https://github.com/apache/hadoop/pull/4764#issuecomment-1231570902
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 3m 5s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 31m 3s | | trunk passed |
| +1 :green_heart: | compile | 5m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 4m 5s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 17s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 26s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 16s | | the patch passed |
| +1 :green_heart: | compile | 4m 52s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 4m 52s | | the patch passed |
| +1 :green_heart: | compile | 3m 55s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 3m 55s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 52s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 37s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 0s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 108m 27s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 3m 51s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 264m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4764/17/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4764 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 5a86318d1426 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 276575beeb4b9361512805674319add383da3eef |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranc
[jira] [Commented] (YARN-6939) Upgrade JUnit from 4 to 5 in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597794#comment-17597794 ] groot commented on YARN-6939: - Sure [~aajisaka] , I will do that. > Upgrade JUnit from 4 to 5 in hadoop-yarn > > > Key: YARN-6939 > URL: https://issues.apache.org/jira/browse/YARN-6939 > Project: Hadoop YARN > Issue Type: Test >Reporter: Akira Ajisaka >Assignee: groot >Priority: Major > > Feel free to create sub-tasks for each module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory after YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597791#comment-17597791
]
ASF GitHub Bot commented on YARN-11287:
---
slfan1989 commented on PR #4828:
URL: https://github.com/apache/hadoop/pull/4828#issuecomment-1231557641
@aajisaka @ashutoshcipher Thank you very much for helping to review the code!
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> after YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, test
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6939) Upgrade JUnit from 4 to 5 in hadoop-yarn
[
https://issues.apache.org/jira/browse/YARN-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597790#comment-17597790
]
Akira Ajisaka commented on YARN-6939:
-
Thank you [~groot].
For the contributors, in addition, I want to make sure we won't make any
regressions like YARN-11287. Could you run {{mvn test}} under
hadoop-yarn-project/hadoop-yarn directory and paste the result in the PR?
> Upgrade JUnit from 4 to 5 in hadoop-yarn
>
>
> Key: YARN-6939
> URL: https://issues.apache.org/jira/browse/YARN-6939
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Akira Ajisaka
>Assignee: groot
>Priority: Major
>
> Feel free to create sub-tasks for each module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11177) Support getNewReservation, submitReservation, updateReservation, deleteReservation API's for Federation
[
https://issues.apache.org/jira/browse/YARN-11177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597786#comment-17597786
]
ASF GitHub Bot commented on YARN-11177:
---
hadoop-yetus commented on PR #4764:
URL: https://github.com/apache/hadoop/pull/4764#issuecomment-1231551778
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 1s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 30s | | trunk passed |
| +1 :green_heart: | compile | 4m 15s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 16s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 1s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 56s | | the patch passed |
| +1 :green_heart: | compile | 4m 1s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 4m 1s | | the patch passed |
| +1 :green_heart: | compile | 3m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 3m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 10s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 45s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 20s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 48s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 102m 1s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 3m 39s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 243m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4764/19/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4764 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux ccc47271d808 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 276575beeb4b9361512805674319add383da3eef |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranc
[jira] [Updated] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory after YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-11287:
-
Summary: Fix NoClassDefFoundError:
org/junit/platform/launcher/core/LauncherFactory after YARN-10793 (was: Fix
NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After
YARN-10793)
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> after YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, test
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated YARN-11287:
-
Component/s: build
test
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, test
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597783#comment-17597783
]
ASF GitHub Bot commented on YARN-11287:
---
aajisaka merged PR #4828:
URL: https://github.com/apache/hadoop/pull/4828
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved YARN-11287.
--
Fix Version/s: 3.4.0
Resolution: Fixed
Committed to trunk. Thank you [~slfan1989] for reporting and fixing this issue.
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597782#comment-17597782
]
ASF GitHub Bot commented on YARN-11287:
---
aajisaka commented on PR #4828:
URL: https://github.com/apache/hadoop/pull/4828#issuecomment-1231547633
+1 for the change. Thank you @slfan1989
> I think when upgrading junit test 4 to 5, we should consider global unit
tests.
Qbt job daily runs global unit tests, but I couldn't find any issue
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/967/artifact/out/
Probably qbt didn't run test on hadoop-yarn because the module itself has no
Java tests.
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597765#comment-17597765
]
ASF GitHub Bot commented on YARN-11287:
---
hadoop-yetus commented on PR #4828:
URL: https://github.com/apache/hadoop/pull/4828#issuecomment-1231515820
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 7s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | shadedclient | 69m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | shadedclient | 27m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 4m 36s | |
hadoop-yarn-server-applicationhistoryservice in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 106m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4828/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4828 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 7b53dd05640d 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9f8f7182301c02b06f190e16e2c910f73152101d |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4828/1/testReport/ |
| Max. process+thread count | 1801 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4828/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
Th
[jira] [Commented] (YARN-6939) Upgrade JUnit from 4 to 5 in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597749#comment-17597749 ] groot commented on YARN-6939: - Thank you so much [~aajisaka] . I have mentioned you in few PRs. Please help review them whenever you have free time. I will continue on other modules meanwhile. > Upgrade JUnit from 4 to 5 in hadoop-yarn > > > Key: YARN-6939 > URL: https://issues.apache.org/jira/browse/YARN-6939 > Project: Hadoop YARN > Issue Type: Test >Reporter: Akira Ajisaka >Assignee: groot >Priority: Major > > Feel free to create sub-tasks for each module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[ https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597746#comment-17597746 ] ASF GitHub Bot commented on YARN-11286: --- slfan1989 commented on PR #4824: URL: https://github.com/apache/hadoop/pull/4824#issuecomment-1231495365 @ayushtkn Can you help review this pr? Thank you very much! > Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter > configurable > - > > Key: YARN-11286 > URL: https://issues.apache.org/jira/browse/YARN-11286 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > > AsyncDispatcher#printEventDetailsExecutor thread pool parameters are > hard-coded, extract this part of hard-coded configuration parameters to the > configuration file. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597744#comment-17597744
]
ASF GitHub Bot commented on YARN-11241:
---
ashutoshcipher commented on code in PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#discussion_r958312471
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java:
##
@@ -234,31 +234,63 @@ private void verifyLocalFileDeletion(
// ensure filesystems were closed
verify(logAggregationService).closeFileSystems(
any(UserGroupInformation.class));
-List dirList = new ArrayList<>();
-dirList.add(new Path(app1LogDir.toURI()));
-verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
-delSrvc, user, null, dirList)));
-
-String containerIdStr = container11.toString();
-File containerLogDir = new File(app1LogDir, containerIdStr);
-int count = 0;
-int maxAttempts = 50;
-for (String fileType : new String[] { "stdout", "stderr", "syslog" }) {
- File f = new File(containerLogDir, fileType);
+boolean filesShouldBeDeleted =
+
this.conf.getBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP,
+YarnConfiguration.DEFAULT_LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP);
+if (filesShouldBeDeleted) {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
+ }
count = 0;
- while ((f.exists()) && (count < maxAttempts)) {
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
count++;
Thread.sleep(100);
}
- Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
-}
-count = 0;
-while ((app1LogDir.exists()) && (count < maxAttempts)) {
- count++;
- Thread.sleep(100);
+ Assert.assertFalse("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
+} else {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, never()).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertTrue("File [" + f + "] was not deleted", f.exists());
+ }
+ count = 0;
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
+count++;
+Thread.sleep(100);
+ }
+ Assert.assertTrue("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
Review Comment:
As per my experiment, it need few seconds and 5 secs seems to be kind of
optimal
> Add uncleaning option for local app log file with log-aggregation enabled
> -
>
> Key: YARN-11241
> URL: https://issues.apache.org/jira/browse/YARN-11241
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: log-aggregation
>Reporter: groot
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add uncleaning option for local app log file with log-aggregation enabled
> This will be helpful for debugging purpose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[ https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597741#comment-17597741 ] ASF GitHub Bot commented on YARN-11241: --- ashutoshcipher commented on code in PR #4703: URL: https://github.com/apache/hadoop/pull/4703#discussion_r958308526 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java: ## @@ -172,6 +173,9 @@ public AppLogAggregatorImpl(Dispatcher dispatcher, this.logAggregationContext = logAggregationContext; this.context = context; this.nodeId = nodeId; +this.enableLocalCleanup = +conf.getBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP, +YarnConfiguration.DEFAULT_LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP); Review Comment: Done > Add uncleaning option for local app log file with log-aggregation enabled > - > > Key: YARN-11241 > URL: https://issues.apache.org/jira/browse/YARN-11241 > Project: Hadoop YARN > Issue Type: New Feature > Components: log-aggregation >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Add uncleaning option for local app log file with log-aggregation enabled > This will be helpful for debugging purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[ https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597740#comment-17597740 ] ASF GitHub Bot commented on YARN-11241: --- ashutoshcipher commented on code in PR #4703: URL: https://github.com/apache/hadoop/pull/4703#discussion_r958308369 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml: ## @@ -1548,6 +1548,13 @@ 60 + +Whether to clean up nodemanager logs when log aggregation is enabled Review Comment: Updated the description in my next commit. > Add uncleaning option for local app log file with log-aggregation enabled > - > > Key: YARN-11241 > URL: https://issues.apache.org/jira/browse/YARN-11241 > Project: Hadoop YARN > Issue Type: New Feature > Components: log-aggregation >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Add uncleaning option for local app log file with log-aggregation enabled > This will be helpful for debugging purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597739#comment-17597739
]
ASF GitHub Bot commented on YARN-11241:
---
ashutoshcipher commented on code in PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#discussion_r958307863
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java:
##
@@ -472,7 +476,9 @@ public void run() {
// do post clean up of log directories on any other exception
LOG.error("Error occurred while aggregating the log for the application "
+ appId, e);
- doAppLogAggregationPostCleanUp();
+ if (enableLocalCleanup) {
+doAppLogAggregationPostCleanUp();
+ }
Review Comment:
Good idea, Addressed in my next commit.
> Add uncleaning option for local app log file with log-aggregation enabled
> -
>
> Key: YARN-11241
> URL: https://issues.apache.org/jira/browse/YARN-11241
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: log-aggregation
>Reporter: groot
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add uncleaning option for local app log file with log-aggregation enabled
> This will be helpful for debugging purpose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9708) Yarn Router Support DelegationToken
[
https://issues.apache.org/jira/browse/YARN-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597737#comment-17597737
]
ASF GitHub Bot commented on YARN-9708:
--
hadoop-yetus commented on PR #4746:
URL: https://github.com/apache/hadoop/pull/4746#issuecomment-1231457010
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 58s | | Maven dependency ordering for branch |
| -1 :x: | mvninstall | 25m 0s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4746/14/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 11m 1s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 3s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 6s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 55s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 7s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 32s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 39s | | the patch passed |
| +1 :green_heart: | compile | 9m 51s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 9m 51s | | the patch passed |
| -1 :x: | javac | 9m 51s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4746/14/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 1 new +
740 unchanged - 0 fixed = 741 total (was 740) |
| +1 :green_heart: | compile | 8m 54s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 54s | | the patch passed |
| -1 :x: | javac | 8m 54s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4746/14/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 649 unchanged - 2 fixed = 652 total (was 651) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 50s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 2
fixed = 26 total (was 28) |
| +1 :green_heart: | mvnsite | 4m 4s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 37s | | the patch passed
[jira] [Commented] (YARN-11273) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With SQL
[
https://issues.apache.org/jira/browse/YARN-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597722#comment-17597722
]
ASF GitHub Bot commented on YARN-11273:
---
hadoop-yetus commented on PR #4817:
URL: https://github.com/apache/hadoop/pull/4817#issuecomment-1231419615
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 21m 22s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 44s | | trunk passed |
| +1 :green_heart: | compile | 9m 54s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 2s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 11s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 55s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 4m 5s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 15m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 44s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 51s | | the patch passed |
| +1 :green_heart: | compile | 9m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 9m 28s | | the patch passed |
| +1 :green_heart: | compile | 8m 48s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 48s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 47s | | the patch passed |
| +1 :green_heart: | mvnsite | 6m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 22s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 43s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 15m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 237m 49s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 3m 29s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 5m 3s | |
hadoop-yarn-server-applicationhistoryservice in the patch passed. |
| +1 :green_heart: | asflicense | 1m 24s | | The patch does not
generate ASF License warnings. |
| | | 450m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4817/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4817 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient xmllint spotbugs checkstyle |
| uname | Linux f5b55e53f97a 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / abe9db31f1ce38ea49779c67925dc3d83dd89a3d |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job
[jira] [Commented] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597716#comment-17597716
]
ASF GitHub Bot commented on YARN-11287:
---
slfan1989 commented on PR #4828:
URL: https://github.com/apache/hadoop/pull/4828#issuecomment-1231407994
I think when upgrading junit test 4 to 5, we should consider global unit
tests.
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597715#comment-17597715
]
ASF GitHub Bot commented on YARN-11287:
---
slfan1989 commented on PR #4828:
URL: https://github.com/apache/hadoop/pull/4828#issuecomment-1231405949
@ashutoshcipher @aajisaka I found that after YARN-10793 Upgrade Junit from 4
to 5 in hadoop-yarn-server-applicationhistoryservice, Junit Test under the
entire module of Yarn will have an exception, java.lang.NoClassDefFoundError:
org/junit/platform/launcher/core/LauncherFactory :
org.junit.platform.launcher.core.LauncherFactory.
I import a dependency in the hadoop-yarn-server-applicationhistoryservice
module.
Please help to review the code. Thank you very much!
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-11287:
--
Labels: pull-request-available (was: )
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
[
https://issues.apache.org/jira/browse/YARN-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597711#comment-17597711
]
ASF GitHub Bot commented on YARN-11287:
---
slfan1989 opened a new pull request, #4828:
URL: https://github.com/apache/hadoop/pull/4828
JIRA: YARN-11287. Fix NoClassDefFoundError:
org/junit/platform/launcher/core/LauncherFactory After YARN-10793.
After executing the yarn-project global unit test, I found the following
error:
```
ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test) on
project hadoop-yarn-server-applicationhistoryservice: Execution default-test of
goal org.apache.maven.plugins:maven-surefire-
plugin:3.0.0-M1:test failed: java.lang.NoClassDefFoundError:
org/junit/platform/launcher/core/LauncherFactory:
org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice
```
> Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory
> After YARN-10793
> ---
>
> Key: YARN-11287
> URL: https://issues.apache.org/jira/browse/YARN-11287
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>
> After executing the yarn-project global unit test, I found the following
> error:
> {code:java}
> ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test)
> on project hadoop-yarn-server-applicationhistoryservice: Execution
> default-test of goal
> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
> java.lang.NoClassDefFoundError:
> org/junit/platform/launcher/core/LauncherFactory:
> org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11287) Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory After YARN-10793
fanshilun created YARN-11287:
Summary: Fix NoClassDefFoundError:
org/junit/platform/launcher/core/LauncherFactory After YARN-10793
Key: YARN-11287
URL: https://issues.apache.org/jira/browse/YARN-11287
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.4.0
Reporter: fanshilun
Assignee: fanshilun
After executing the yarn-project global unit test, I found the following error:
{code:java}
ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test (default-test) on
project hadoop-yarn-server-applicationhistoryservice: Execution default-test of
goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M1:test failed:
java.lang.NoClassDefFoundError:
org/junit/platform/launcher/core/LauncherFactory:
org.junit.platform.launcher.core.LauncherFactory -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hadoop-yarn-server-applicationhistoryservice {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6939) Upgrade JUnit from 4 to 5 in hadoop-yarn
[ https://issues.apache.org/jira/browse/YARN-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597689#comment-17597689 ] Akira Ajisaka commented on YARN-6939: - If you have some PRs for upgrading JUnit version, please ping or mention me in the PR. I can review later when I have time. > Upgrade JUnit from 4 to 5 in hadoop-yarn > > > Key: YARN-6939 > URL: https://issues.apache.org/jira/browse/YARN-6939 > Project: Hadoop YARN > Issue Type: Test >Reporter: Akira Ajisaka >Assignee: groot >Priority: Major > > Feel free to create sub-tasks for each module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597687#comment-17597687
]
ASF GitHub Bot commented on YARN-11241:
---
aajisaka commented on code in PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#discussion_r958178418
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java:
##
@@ -472,7 +476,9 @@ public void run() {
// do post clean up of log directories on any other exception
LOG.error("Error occurred while aggregating the log for the application "
+ appId, e);
- doAppLogAggregationPostCleanUp();
+ if (enableLocalCleanup) {
+doAppLogAggregationPostCleanUp();
+ }
Review Comment:
As `enableLocalCleanup` isn't local variable, can we skip cleanup in the
`doAppLogAggregationPostCleanUp()`?
```java
private void doAppLogAggregationPostCleanUp() {
if (!enableLocalCleanup) {
return;
}
```
That way we can change only one place.
> Add uncleaning option for local app log file with log-aggregation enabled
> -
>
> Key: YARN-11241
> URL: https://issues.apache.org/jira/browse/YARN-11241
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: log-aggregation
>Reporter: groot
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add uncleaning option for local app log file with log-aggregation enabled
> This will be helpful for debugging purpose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11241) Add uncleaning option for local app log file with log-aggregation enabled
[
https://issues.apache.org/jira/browse/YARN-11241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597686#comment-17597686
]
ASF GitHub Bot commented on YARN-11241:
---
aajisaka commented on code in PR #4703:
URL: https://github.com/apache/hadoop/pull/4703#discussion_r958182732
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java:
##
@@ -172,6 +173,9 @@ public AppLogAggregatorImpl(Dispatcher dispatcher,
this.logAggregationContext = logAggregationContext;
this.context = context;
this.nodeId = nodeId;
+this.enableLocalCleanup =
+conf.getBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP,
+YarnConfiguration.DEFAULT_LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP);
Review Comment:
If it is false, print warn message to show this configuration is only for
testing and not for any production system.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java:
##
@@ -234,31 +234,63 @@ private void verifyLocalFileDeletion(
// ensure filesystems were closed
verify(logAggregationService).closeFileSystems(
any(UserGroupInformation.class));
-List dirList = new ArrayList<>();
-dirList.add(new Path(app1LogDir.toURI()));
-verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
-delSrvc, user, null, dirList)));
-
-String containerIdStr = container11.toString();
-File containerLogDir = new File(app1LogDir, containerIdStr);
-int count = 0;
-int maxAttempts = 50;
-for (String fileType : new String[] { "stdout", "stderr", "syslog" }) {
- File f = new File(containerLogDir, fileType);
+boolean filesShouldBeDeleted =
+
this.conf.getBoolean(YarnConfiguration.LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP,
+YarnConfiguration.DEFAULT_LOG_AGGREGATION_ENABLE_LOCAL_CLEANUP);
+if (filesShouldBeDeleted) {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, times(2)).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
+ }
count = 0;
- while ((f.exists()) && (count < maxAttempts)) {
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
count++;
Thread.sleep(100);
}
- Assert.assertFalse("File [" + f + "] was not deleted", f.exists());
-}
-count = 0;
-while ((app1LogDir.exists()) && (count < maxAttempts)) {
- count++;
- Thread.sleep(100);
+ Assert.assertFalse("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
+} else {
+ List dirList = new ArrayList<>();
+ dirList.add(new Path(app1LogDir.toURI()));
+ verify(delSrvc, never()).delete(argThat(new FileDeletionMatcher(
+ delSrvc, user, null, dirList)));
+
+ String containerIdStr = container11.toString();
+ File containerLogDir = new File(app1LogDir, containerIdStr);
+ int count = 0;
+ int maxAttempts = 50;
+ for (String fileType : new String[]{"stdout", "stderr", "syslog"}) {
+File f = new File(containerLogDir, fileType);
+count = 0;
+while ((f.exists()) && (count < maxAttempts)) {
+ count++;
+ Thread.sleep(100);
+}
+Assert.assertTrue("File [" + f + "] was not deleted", f.exists());
+ }
+ count = 0;
+ while ((app1LogDir.exists()) && (count < maxAttempts)) {
+count++;
+Thread.sleep(100);
+ }
+ Assert.assertTrue("Directory [" + app1LogDir + "] was not deleted",
+ app1LogDir.exists());
Review Comment:
It takes at least 50 * 100ms * 2 = 10sec to run. Do we really need to wait
for 5 sec? Also, we just need to wait several seconds instead of running while
loop.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml:
##
@@ -1548,6 +1548,13 @@
60
+
+Whether to clean up nodemanager logs when log aggregation is
enabled
Review Comment:
We must warn the users. Could you document that setting to false dis
[jira] [Commented] (YARN-11273) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With SQL
[
https://issues.apache.org/jira/browse/YARN-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597677#comment-17597677
]
ASF GitHub Bot commented on YARN-11273:
---
slfan1989 commented on code in PR #4817:
URL: https://github.com/apache/hadoop/pull/4817#discussion_r957975795
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/SQLFederationStateStore.java:
##
@@ -1016,30 +1035,275 @@ private static byte[] getByteArray(ByteBuffer bb) {
@Override
public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
Review Comment:
Thank you for your suggestion.
I will explain this part of the test:
In YARN-11272, we implemented store Reservations in ZK, in this
pr(https://github.com/apache/hadoop/pull/4781) we defined the test of the
method in FederationStateStoreBaseTest.java
```
testAddReservationHomeSubCluster()
testAddReservationHomeSubClusterReservationAlreadyExists()
testAddReservationHomeSubClusterAppAlreadyExistsInTheSameSC()
testDeleteReservationHomeSubCluster()
testDeleteReservationHomeSubClusterUnknownApp()
testUpdateReservationHomeSubCluster()
```
Because TestSQLFederationStateStore extends FederationStateStoreBaseTest, we
can reuse YARN-11272 tests, and of course we will add some tests.
Can you help merge YARN-11272(#4781) into trunk branch? Thank you very much！
> [RESERVATION] Federation StateStore: Support storage/retrieval of
> Reservations With SQL
> ---
>
> Key: YARN-11273
> URL: https://issues.apache.org/jira/browse/YARN-11273
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11245) Upgrade JUnit from 4 to 5 in hadoop-yarn-csi
[ https://issues.apache.org/jira/browse/YARN-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597673#comment-17597673 ] ASF GitHub Bot commented on YARN-11245: --- aajisaka merged PR #4778: URL: https://github.com/apache/hadoop/pull/4778 > Upgrade JUnit from 4 to 5 in hadoop-yarn-csi > > > Key: YARN-11245 > URL: https://issues.apache.org/jira/browse/YARN-11245 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > > Upgrade JUnit from 4 to 5 in hadoop-yarn-csi -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11245) Upgrade JUnit from 4 to 5 in hadoop-yarn-csi
[ https://issues.apache.org/jira/browse/YARN-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597671#comment-17597671 ] ASF GitHub Bot commented on YARN-11245: --- aajisaka commented on PR #4778: URL: https://github.com/apache/hadoop/pull/4778#issuecomment-1231327492 @slfan1989 @ashutoshcipher Thank you for the discussion > I feel community is already busy with so many things. I am atleast personally not expecting from someone to sit and write down best practices for upgrading Junit 4 to Junit 5 for me. But if someone want/can do it, I am very much happy :) +1 to write down best practices. For the migration, now I like using AssertJ APIs as they are more fluent and their error messages are user-friendly, but I think we can continue to use JUnit 5 APIs because it's the similar experience as existing JUnit 4 APIs and it's more easier to use for the upgrade rather than replacing all the APIs with AssertJ. > Upgrade JUnit from 4 to 5 in hadoop-yarn-csi > > > Key: YARN-11245 > URL: https://issues.apache.org/jira/browse/YARN-11245 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > > Upgrade JUnit from 4 to 5 in hadoop-yarn-csi -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6946) Upgrade JUnit from 4 to 5 in hadoop-yarn-common
[ https://issues.apache.org/jira/browse/YARN-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597666#comment-17597666 ] ASF GitHub Bot commented on YARN-6946: -- ashutoshcipher commented on PR #4717: URL: https://github.com/apache/hadoop/pull/4717#issuecomment-1231322486 @aajisaka - Please help in reviewing the PR. Thanks. > Upgrade JUnit from 4 to 5 in hadoop-yarn-common > --- > > Key: YARN-6946 > URL: https://issues.apache.org/jira/browse/YARN-6946 > Project: Hadoop YARN > Issue Type: Sub-task > Components: test >Reporter: Akira Ajisaka >Assignee: groot >Priority: Major > Labels: pull-request-available > Attachments: YARN-6946.wip001.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11260) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice
[ https://issues.apache.org/jira/browse/YARN-11260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597660#comment-17597660 ] ASF GitHub Bot commented on YARN-11260: --- ashutoshcipher commented on PR #4775: URL: https://github.com/apache/hadoop/pull/4775#issuecomment-1231318894 @aajisaka - Please help in reviewing the PR. Thanks. > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice > --- > > Key: YARN-11260 > URL: https://issues.apache.org/jira/browse/YARN-11260 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11261) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy
[ https://issues.apache.org/jira/browse/YARN-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597658#comment-17597658 ] ASF GitHub Bot commented on YARN-11261: --- ashutoshcipher commented on PR #4777: URL: https://github.com/apache/hadoop/pull/4777#issuecomment-1231318644 @aajisaka - Please help in reviewing the PR. Thanks. > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy > - > > Key: YARN-11261 > URL: https://issues.apache.org/jira/browse/YARN-11261 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11264) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests
[ https://issues.apache.org/jira/browse/YARN-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597659#comment-17597659 ] ASF GitHub Bot commented on YARN-11264: --- ashutoshcipher commented on PR #4776: URL: https://github.com/apache/hadoop/pull/4776#issuecomment-1231318743 @aajisaka - Please help in reviewing the PR. Thanks. > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests > - > > Key: YARN-11264 > URL: https://issues.apache.org/jira/browse/YARN-11264 > Project: Hadoop YARN > Issue Type: Sub-task > Components: test, yarn >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11271) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-common
[ https://issues.apache.org/jira/browse/YARN-11271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597661#comment-17597661 ] ASF GitHub Bot commented on YARN-11271: --- ashutoshcipher commented on PR #4774: URL: https://github.com/apache/hadoop/pull/4774#issuecomment-1231319003 @aajisaka - Please help in reviewing the PR. Thanks. > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-common > > > Key: YARN-11271 > URL: https://issues.apache.org/jira/browse/YARN-11271 > Project: Hadoop YARN > Issue Type: Sub-task > Components: test, yarn >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11265) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-sharedcachemanager
[ https://issues.apache.org/jira/browse/YARN-11265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597663#comment-17597663 ] ASF GitHub Bot commented on YARN-11265: --- ashutoshcipher commented on PR #4772: URL: https://github.com/apache/hadoop/pull/4772#issuecomment-1231319296 @aajisaka - Please help in reviewing the PR. Thanks. > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-sharedcachemanager > -- > > Key: YARN-11265 > URL: https://issues.apache.org/jira/browse/YARN-11265 > Project: Hadoop YARN > Issue Type: Sub-task > Components: test, yarn >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11270) Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-client
[ https://issues.apache.org/jira/browse/YARN-11270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597662#comment-17597662 ] ASF GitHub Bot commented on YARN-11270: --- ashutoshcipher commented on PR #4773: URL: https://github.com/apache/hadoop/pull/4773#issuecomment-1231319138 @aajisaka - Please help in reviewing the PR. Thanks. > Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-client > > > Key: YARN-11270 > URL: https://issues.apache.org/jira/browse/YARN-11270 > Project: Hadoop YARN > Issue Type: Sub-task > Components: test, yarn >Affects Versions: 3.3.4 >Reporter: groot >Assignee: groot >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597636#comment-17597636
]
ASF GitHub Bot commented on YARN-11274:
---
hadoop-yetus commented on PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#issuecomment-1231277128
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 24s | | trunk passed |
| +1 :green_heart: | compile | 1m 45s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 58s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 33s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 2 new + 78 unchanged - 0 fixed = 80 total (was 78) |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 37s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 41s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 125m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4783 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux cbd16ee78ccf 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2723c22c7ace9001b69419e747ef685ce47adac8 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/2
