[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610236#comment-17610236 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1260032750 @goiri Thank you very much for your help reviewing the code! > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610225#comment-17610225 ] ASF GitHub Bot commented on YARN-11290: --- goiri merged PR #4846: URL: https://github.com/apache/hadoop/pull/4846 > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610217#comment-17610217 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1259987263 @goiri Can you help merge this pr into trunk branch? Thank you very much! after this pr is completed, I will submit a new pr to clean up the completed application in the router's StateStore. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17609842#comment-17609842
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1259032015
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 52s | | trunk passed |
| +1 :green_heart: | compile | 9m 56s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 44s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 49s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 6m 11s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 11s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 37s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 1m 3s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 13s | | the patch passed |
| +1 :green_heart: | compile | 9m 13s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 13s | | the patch passed |
| +1 :green_heart: | javac | 9m 13s | | the patch passed |
| +1 :green_heart: | compile | 8m 35s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 35s | | the patch passed |
| +1 :green_heart: | javac | 8m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 10s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 34s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 16s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 40s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 24s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 47s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 19s | | The patch does not
generate ASF License warnings. |
| | | 459m 6s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/30/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux 3eecc188ca4c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
|
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608895#comment-17608895
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1256603495
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 21s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 35s | | trunk passed |
| +1 :green_heart: | compile | 9m 58s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 50s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 43s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 2s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 5s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 32s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 57s | | the patch passed |
| +1 :green_heart: | compile | 9m 14s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 14s | | the patch passed |
| +1 :green_heart: | javac | 9m 14s | | the patch passed |
| +1 :green_heart: | compile | 8m 37s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 37s | | the patch passed |
| +1 :green_heart: | javac | 8m 37s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 46s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 56s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 51s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 17s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 14s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 28s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 22s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 35s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 457m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/29/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux bb6866d35151 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
|
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17608702#comment-17608702
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1256079979
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 48s | | trunk passed |
| +1 :green_heart: | compile | 11m 10s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 10m 24s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 21s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 34s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 52s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 16s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 27s | | the patch passed |
| +1 :green_heart: | compile | 10m 51s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 10m 51s | | the patch passed |
| +1 :green_heart: | javac | 10m 51s | | the patch passed |
| +1 :green_heart: | compile | 9m 45s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 45s | | the patch passed |
| +1 :green_heart: | javac | 9m 45s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/28/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 56s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/28/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 2 new + 165 unchanged
- 0 fixed = 167 total (was 165) |
| +1 :green_heart: | mvnsite | 8m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 46s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 56s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 58s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 22s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 41s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 34s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 45s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 477m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606175#comment-17606175
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1250132393
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 8s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 38s | | trunk passed |
| +1 :green_heart: | compile | 9m 51s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 48s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 11s | | trunk passed |
| +1 :green_heart: | mvnsite | 7m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 48s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 14s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 45s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 13s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 59s | | the patch passed |
| +1 :green_heart: | compile | 11m 0s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 11m 0s | | the patch passed |
| +1 :green_heart: | javac | 11m 0s | | the patch passed |
| +1 :green_heart: | compile | 9m 19s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 19s | | the patch passed |
| +1 :green_heart: | javac | 9m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 50s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 18s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 48s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 34s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 236m 35s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 28s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 31s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 28s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 455m 50s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/25/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux 432c09e8bf8e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
|
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605575#comment-17605575 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r972541463 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,24 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster] + WHERE [homeSubCluster] = @homeSubCluster OR @homeSubCluster = '') AS t Review Comment: Thank you very much for helping to review the code, I will modify the code. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605541#comment-17605541 ] ASF GitHub Bot commented on YARN-11290: --- goiri commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r972467705 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,24 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster] + WHERE [homeSubCluster] = @homeSubCluster OR @homeSubCluster = '') AS t Review Comment: In the other one we use AS applicationshomesubcluster; let's try to be consistent with the names as possible. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605523#comment-17605523 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1248591269 @goiri Please help to review the code again, thank you very much! Error in TestLogAggregationService in unit test, I have submitted pr separately to fix. (https://github.com/apache/hadoop/pull/4893) > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605230#comment-17605230
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1247843639
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 33s | | trunk passed |
| +1 :green_heart: | compile | 10m 42s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 17s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 31s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 19s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 16s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 28s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 52s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 39s | | the patch passed |
| +1 :green_heart: | compile | 9m 18s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 18s | | the patch passed |
| +1 :green_heart: | javac | 9m 18s | | the patch passed |
| +1 :green_heart: | compile | 8m 46s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 46s | | the patch passed |
| +1 :green_heart: | javac | 8m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 51s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 11s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 38s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 10s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 237m 42s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/24/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch passed. |
| +1 :green_heart: | unit | 1m 32s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 28s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 34s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 464m 13s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
|
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/24/artifact/out/Dockerfile
|
| GITHUB PR |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605028#comment-17605028
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r971434340
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,24 +263,54 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-long start = clock.getTime();
-List result = new ArrayList<>();
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
try {
- for (String child : zkManager.getChildren(appsZNode)) {
-ApplicationId appId = ApplicationId.fromString(child);
-SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
-result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List children = zkManager.getChildren(appsZNode);
+ List result =
+ children.stream().map(child -> generateAppHomeSC(child))
+
.sorted(Comparator.comparing(ApplicationHomeSubCluster::getCreateTime).reversed())
Review Comment:
Thanks for your suggestion, I will fix it.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17604928#comment-17604928
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1247192785
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 39s | | trunk passed |
| +1 :green_heart: | compile | 10m 43s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 1s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 20s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 0s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 7s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 4s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 57s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 21s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 23s | | the patch passed |
| +1 :green_heart: | compile | 9m 52s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 52s | | the patch passed |
| +1 :green_heart: | javac | 9m 52s | | the patch passed |
| +1 :green_heart: | compile | 9m 17s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 17s | | the patch passed |
| +1 :green_heart: | javac | 9m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 50s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 4s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 35s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 58s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 241m 32s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/23/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch passed. |
| +1 :green_heart: | unit | 1m 35s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 35s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 37s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 25s | | The patch does not
generate ASF License warnings. |
| | | 471m 2s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
|
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/23/artifact/out/Dockerfile
|
| GITHUB PR |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17604824#comment-17604824
]
ASF GitHub Bot commented on YARN-11290:
---
goiri commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r971038757
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,24 +263,54 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-long start = clock.getTime();
-List result = new ArrayList<>();
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
try {
- for (String child : zkManager.getChildren(appsZNode)) {
-ApplicationId appId = ApplicationId.fromString(child);
-SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
-result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List children = zkManager.getChildren(appsZNode);
+ List result =
+ children.stream().map(child -> generateAppHomeSC(child))
+
.sorted(Comparator.comparing(ApplicationHomeSubCluster::getCreateTime).reversed())
+ .filter(appHomeSC -> filterHomeSubCluster(requestSC,
appHomeSC.getHomeSubCluster()))
+ .limit(maxAppsInStateStore)
+ .collect(Collectors.toList());
+ long end = clock.getTime();
+ opDurations.addGetAppsHomeSubClusterDuration(start, end);
+ LOG.info("filterSubClusterId = {}, appCount = {}.", requestSC,
result.size());
+ return GetApplicationsHomeSubClusterResponse.newInstance(result);
} catch (Exception e) {
String errMsg = "Cannot get apps: " + e.getMessage();
FederationStateStoreUtils.logAndThrowStoreException(LOG, errMsg);
}
-long end = clock.getTime();
-opDurations.addGetAppsHomeSubClusterDuration(start, end);
-return GetApplicationsHomeSubClusterResponse.newInstance(result);
+
+throw new YarnException("Cannot get app by request");
+ }
+
+ private ApplicationHomeSubCluster generateAppHomeSC(String appId) {
+try {
+ ApplicationId applicationId = ApplicationId.fromString(appId);
+ SubClusterId homeSubCluster = getApp(applicationId);
+ ApplicationHomeSubCluster app =
+ ApplicationHomeSubCluster.newInstance(applicationId, Time.now(),
homeSubCluster);
+ return app;
+} catch (Exception ex) {
+ LOG.error("get homeSubCluster by appId = {}.", appId);
+}
+return null;
+ }
+
+ private boolean filterHomeSubCluster(SubClusterId filterSubCluster,
Review Comment:
We have this function defined three times in three places.
It should also be static.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,24 +263,54 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-long start = clock.getTime();
-List result = new ArrayList<>();
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
try {
- for (String child : zkManager.getChildren(appsZNode)) {
-ApplicationId appId = ApplicationId.fromString(child);
-SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
-result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List children = zkManager.getChildren(appsZNode);
+ List result =
+ children.stream().map(child -> generateAppHomeSC(child))
+
.sorted(Comparator.comparing(ApplicationHomeSubCluster::getCreateTime).reversed())
Review Comment:
I don't think indentation is correct.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions:
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603858#comment-17603858
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1246173571
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 24m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 46s | | trunk passed |
| +1 :green_heart: | compile | 9m 54s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 48s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 17s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 50s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 14s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 35s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 59s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 0s | | the patch passed |
| +1 :green_heart: | compile | 9m 12s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 12s | | the patch passed |
| +1 :green_heart: | javac | 9m 12s | | the patch passed |
| +1 :green_heart: | compile | 8m 43s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 43s | | the patch passed |
| +1 :green_heart: | javac | 8m 43s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 49s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 5s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 36s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 239m 38s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/21/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch passed. |
| +1 :green_heart: | unit | 1m 41s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 33s | | hadoop-yarn-common in the patch
passed. |
| -1 :x: | unit | 3m 39s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/21/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt)
| hadoop-yarn-server-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 19s | | The patch does not
generate ASF License warnings. |
| | | 466m 16s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.federation.store.impl.TestSQLFederationStateStore |
| |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603804#comment-17603804 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r970137569 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,37 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +IF @homeSubCluster = '' +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster]) AS t +WHERE app_rank <= @limit; +ELSE +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(partition by [homeSubCluster] order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster] + WHERE [homeSubCluster] = @homeSubCluster) AS t Review Comment: My code is a bit complicated, your approach is really good. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603802#comment-17603802 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r970137060 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,21 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + applicationId, + homeSubCluster, + createTime +FROM +(SELECT + *, + @rownum := 0, + IF(homeSubCluster_IN = '', 1, (homeSubCluster = homeSubCluster_IN)) AS filter_result + FROM applicationshomesubcluster + ORDER BY createTime DESC) AS app_home_sc +WHERE filter_result = 1 Review Comment: Thank you very much for your suggestion, I agree with you, I will modify the code. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603801#comment-17603801 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r970137060 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,21 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + applicationId, + homeSubCluster, + createTime +FROM +(SELECT + *, + @rownum := 0, + IF(homeSubCluster_IN = '', 1, (homeSubCluster = homeSubCluster_IN)) AS filter_result + FROM applicationshomesubcluster + ORDER BY createTime DESC) AS app_home_sc +WHERE filter_result = 1 Review Comment: Thank you very much for your suggestion, I agree with you, I will modify the code! > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603794#comment-17603794 ] ASF GitHub Bot commented on YARN-11290: --- goiri commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r970127749 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,21 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + applicationId, + homeSubCluster, + createTime +FROM +(SELECT + *, + @rownum := 0, + IF(homeSubCluster_IN = '', 1, (homeSubCluster = homeSubCluster_IN)) AS filter_result + FROM applicationshomesubcluster + ORDER BY createTime DESC) AS app_home_sc +WHERE filter_result = 1 Review Comment: Why you cannot do: ``` SELECT applicationId, homeSubCluster, createTime FROM applicationshomesubcluster WHERE homeSubCluster_IN = '' OR homeSubCluster = homeSubCluster_IN ``` ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,37 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +IF @homeSubCluster = '' +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster]) AS t +WHERE app_rank <= @limit; +ELSE +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM(SELECT + [applicationId], + [homeSubCluster], + [createTime], + row_number() over(partition by [homeSubCluster] order by [createTime] desc) AS app_rank + FROM [dbo].[applicationsHomeSubCluster] + WHERE [homeSubCluster] = @homeSubCluster) AS t Review Comment: Why not ``` WHERE [homeSubCluster] = @homeSubCluster OR @homeSubCluster = '' ``` > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603745#comment-17603745 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1245869268 @goiri Please help to review the code again, thank you very much! I have provided a test report, we can refer to the following link https://issues.apache.org/jira/secure/attachment/13049175/Improve%20Query%20Condition%20of%20FederationStateStore%23getApplicationsHomeSubCluster%20Test%20Report.pdf MySQL We need 5.7 and above - [ ] MySQL 5.7 - [ ] MySQL 8.0 SQLServer - [ ] SQL Server 2008 R2 Enterprise - [ ] SQL Server 2012 Enterprise - [ ] SQL Server 2016 Enterprise - [ ] SQL Server 2017 Enterprise - [ ] SQL Server 2019 Enterprise > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603740#comment-17603740 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r970008365 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,26 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + applicationId, + homeSubCluster, + createTime + FROM +(SELECT + applicationId, + homeSubCluster, + createTime, Review Comment: We can sort internally and get the sorted result directly externally. ``` CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) BEGIN SELECT applicationId, homeSubCluster, createTime > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603736#comment-17603736
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r97124
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/FederationStateStoreBaseTest.java:
##
@@ -410,6 +414,89 @@ public void testGetApplicationsHomeSubCluster() throws
Exception {
Assert.assertTrue(result.getAppsHomeSubClusters().contains(ahsc2));
}
+ @Test
+ public void testGetApplicationsHomeSubClusterEmpty() throws Exception {
+LambdaTestUtils.intercept(YarnException.class,
+"Missing getApplicationsHomeSubCluster request",
+() -> stateStore.getApplicationsHomeSubCluster(null));
+ }
+
+ @Test
+ public void testGetApplicationsHomeSubClusterFilter() throws Exception {
+// Add ApplicationHomeSC - SC1
+long now = Time.now();
+
+Set appHomeSubClusters = new HashSet<>();
+
+for (int i = 0; i < TEN_ROUNDS; i++) {
Review Comment:
Thanks for your suggestion, I will modify the code.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603734#comment-17603734
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r96747
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/records/ApplicationHomeSubCluster.java:
##
@@ -51,6 +51,18 @@ public static ApplicationHomeSubCluster
newInstance(ApplicationId appId,
return appMapping;
}
+ @Private
+ @Unstable
+ public static ApplicationHomeSubCluster newInstance(ApplicationId appId,
long createTime,
+ SubClusterId homeSubCluster) {
+ApplicationHomeSubCluster appMapping =
Review Comment:
I will fix it.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603733#comment-17603733 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r969998465 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,26 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + applicationId, + homeSubCluster, + createTime + FROM +(SELECT + applicationId, + homeSubCluster, + createTime, + @app_rank := IF(@home_sc = homeSubCluster, @app_rank + 1, 1) AS app_rank, + @home_sc := homeSubCluster +FROM applicationshomesubcluster +ORDER BY createTime DESC +) ranked + WHERE app_rank <= limit_IN + AND (CASE WHEN t.homeSubCluster_IN IS NULL THEN 1 = 1 Review Comment: For MySQL, I also tried to refactor this SQL using a simpler way of writing. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603731#comment-17603731 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r969997820 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,27 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM +(SELECT +[applicationId], +[homeSubCluster], +[createTime], +row_number() over(partition by [homeSubCluster] order by [createTime] desc) AS row_num +FROM [dbo].[applicationsHomeSubCluster]) AS t +WHERE row_num <= @limit + AND (CASE WHEN @homeSubCluster IS NULL THEN 1 Review Comment: I try to use `IF...ELSE` instead of `CASE...WHEN` > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603573#comment-17603573
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1245417267
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 18s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 35s | | trunk passed |
| +1 :green_heart: | compile | 9m 55s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 46s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 3s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 18s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 12s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 35s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 56s | | the patch passed |
| +1 :green_heart: | compile | 9m 4s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 4s | | the patch passed |
| +1 :green_heart: | javac | 9m 4s | | the patch passed |
| +1 :green_heart: | compile | 8m 36s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 36s | | the patch passed |
| +1 :green_heart: | javac | 8m 36s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 9s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 36s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 28s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 215m 36s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/20/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch failed. |
| +1 :green_heart: | unit | 1m 29s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 33s | | hadoop-yarn-common in the patch
passed. |
| -1 :x: | unit | 3m 28s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/20/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt)
| hadoop-yarn-server-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 35s | | The patch does not
generate ASF License warnings. |
| | | 435m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.federation.store.impl.TestSQLFederationStateStore |
| |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603292#comment-17603292 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r969064754 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,26 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + applicationId, + homeSubCluster, + createTime + FROM +(SELECT + applicationId, + homeSubCluster, + createTime, + @app_rank := IF(@home_sc = homeSubCluster, @app_rank + 1, 1) AS app_rank, + @home_sc := homeSubCluster +FROM applicationshomesubcluster Review Comment: Thanks for your help reviewing the code, I will fix it. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603234#comment-17603234
]
ASF GitHub Bot commented on YARN-11290:
---
goiri commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r968842977
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql:
##
@@ -122,10 +122,26 @@ BEGIN
WHERE applicationId = applicationID_IN;
END //
-CREATE PROCEDURE sp_getApplicationsHomeSubCluster()
-BEGIN
- SELECT applicationId, homeSubCluster
- FROM applicationsHomeSubCluster;
+CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN
homeSubCluster_IN varchar(256))
+BEGIN
+ SELECT
+ applicationId,
+ homeSubCluster,
+ createTime
+ FROM
+(SELECT
+ applicationId,
+ homeSubCluster,
+ createTime,
+ @app_rank := IF(@home_sc = homeSubCluster, @app_rank + 1, 1) AS
app_rank,
+ @home_sc := homeSubCluster
+FROM applicationshomesubcluster
Review Comment:
The tabs in line warnings are legit. Let's replace them with spaces.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/records/ApplicationHomeSubCluster.java:
##
@@ -51,6 +51,18 @@ public static ApplicationHomeSubCluster
newInstance(ApplicationId appId,
return appMapping;
}
+ @Private
+ @Unstable
+ public static ApplicationHomeSubCluster newInstance(ApplicationId appId,
long createTime,
+ SubClusterId homeSubCluster) {
+ApplicationHomeSubCluster appMapping =
Review Comment:
Single line
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/FederationStateStoreBaseTest.java:
##
@@ -410,6 +414,89 @@ public void testGetApplicationsHomeSubCluster() throws
Exception {
Assert.assertTrue(result.getAppsHomeSubClusters().contains(ahsc2));
}
+ @Test
+ public void testGetApplicationsHomeSubClusterEmpty() throws Exception {
+LambdaTestUtils.intercept(YarnException.class,
+"Missing getApplicationsHomeSubCluster request",
+() -> stateStore.getApplicationsHomeSubCluster(null));
+ }
+
+ @Test
+ public void testGetApplicationsHomeSubClusterFilter() throws Exception {
+// Add ApplicationHomeSC - SC1
+long now = Time.now();
+
+Set appHomeSubClusters = new HashSet<>();
+
+for (int i = 0; i < TEN_ROUNDS; i++) {
Review Comment:
TEN_ROUNDS is a weird variable.
I was initially thinking more about something like NUM_APPS_X and NUM_APSS_Y
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql:
##
@@ -122,10 +122,26 @@ BEGIN
WHERE applicationId = applicationID_IN;
END //
-CREATE PROCEDURE sp_getApplicationsHomeSubCluster()
-BEGIN
- SELECT applicationId, homeSubCluster
- FROM applicationsHomeSubCluster;
+CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN
homeSubCluster_IN varchar(256))
+BEGIN
+ SELECT
+ applicationId,
+ homeSubCluster,
+ createTime
+ FROM
+(SELECT
+ applicationId,
+ homeSubCluster,
+ createTime,
+ @app_rank := IF(@home_sc = homeSubCluster, @app_rank + 1, 1) AS
app_rank,
+ @home_sc := homeSubCluster
+FROM applicationshomesubcluster
+ORDER BY createTime DESC
+) ranked
+ WHERE app_rank <= limit_IN
+ AND (CASE WHEN t.homeSubCluster_IN IS NULL THEN 1 = 1
Review Comment:
Can we make this a regular boolean condition with no CASE?
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql:
##
@@ -122,10 +122,26 @@ BEGIN
WHERE applicationId = applicationID_IN;
END //
-CREATE PROCEDURE sp_getApplicationsHomeSubCluster()
-BEGIN
- SELECT applicationId, homeSubCluster
- FROM applicationsHomeSubCluster;
+CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN
homeSubCluster_IN varchar(256))
+BEGIN
+ SELECT
+ applicationId,
+ homeSubCluster,
+ createTime
+ FROM
+(SELECT
+ applicationId,
+ homeSubCluster,
+ createTime,
Review Comment:
Shouldn't we sort this?
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql:
##
@@ -111,12 +111,27 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P'
) IS NOT NULL
GO
CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster]
+@limit int,
+@homeSubCluster VARCHAR(256)
AS BEGIN
DECLARE @errorMessage nvarchar(4000)
BEGIN TRY
-SELECT [applicationId], [homeSubCluster], [createTime]
-FROM [dbo].[applicationsHomeSubCluster]
+SELECT
+
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603162#comment-17603162
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1243947248
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 8s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 38s | | trunk passed |
| +1 :green_heart: | compile | 10m 33s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 14s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 13s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 16s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 53s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 16s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 28s | | the patch passed |
| +1 :green_heart: | compile | 10m 5s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 10m 5s | | the patch passed |
| +1 :green_heart: | javac | 10m 5s | | the patch passed |
| +1 :green_heart: | compile | 9m 18s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 18s | | the patch passed |
| +1 :green_heart: | javac | 9m 18s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/19/artifact/out/blanks-tabs.txt)
| The patch 3 line(s) with tabs. |
| +1 :green_heart: | checkstyle | 1m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 6s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 42s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 50s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 19s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 212m 43s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/19/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch failed. |
| +1 :green_heart: | unit | 1m 30s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 31s | | hadoop-yarn-common in the patch
passed. |
| -1 :x: | unit | 3m 29s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/19/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt)
| hadoop-yarn-server-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not
generate ASF License warnings. |
| | | 439m 21s | | |
| Reason | Tests |
|---:|:--|
|
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603160#comment-17603160
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1243941828
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 58s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 35s | | trunk passed |
| +1 :green_heart: | compile | 10m 22s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 19s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 6s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 14s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 3s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 25s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 16s | | the patch passed |
| +1 :green_heart: | compile | 10m 2s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 10m 2s | | the patch passed |
| +1 :green_heart: | javac | 10m 2s | | the patch passed |
| +1 :green_heart: | compile | 9m 13s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 13s | | the patch passed |
| +1 :green_heart: | javac | 9m 13s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/18/artifact/out/blanks-tabs.txt)
| The patch 10 line(s) with tabs. |
| +1 :green_heart: | checkstyle | 1m 47s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 48s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 40s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 51s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 212m 34s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/18/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch failed. |
| +1 :green_heart: | unit | 1m 34s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 22s | | hadoop-yarn-common in the patch
passed. |
| -1 :x: | unit | 3m 39s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/18/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.txt)
| hadoop-yarn-server-common in the patch passed. |
| +1 :green_heart: | asflicense | 1m 19s | | The patch does not
generate ASF License warnings. |
| | | 438m 10s | | |
| Reason | Tests |
|---:|:--|
|
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602894#comment-17602894
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1243013741
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 24s | | trunk passed |
| +1 :green_heart: | compile | 9m 54s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 45s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 26s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 40s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 13s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 37s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 35s | | the patch passed |
| +1 :green_heart: | compile | 9m 16s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 16s | | the patch passed |
| +1 :green_heart: | javac | 9m 16s | | the patch passed |
| +1 :green_heart: | compile | 8m 31s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 31s | | the patch passed |
| +1 :green_heart: | javac | 8m 31s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/17/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/17/artifact/out/blanks-tabs.txt)
| The patch 10 line(s) with tabs. |
| +1 :green_heart: | checkstyle | 1m 48s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 37s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 59s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 9s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 212m 10s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/17/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch failed. |
| +1 :green_heart: | unit | 1m 41s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 21s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 27s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 4s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 35s | | The patch does not
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602564#comment-17602564
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1242502110
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 44s | | trunk passed |
| +1 :green_heart: | compile | 10m 38s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 9s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 10m 15s | | trunk passed |
| +1 :green_heart: | javadoc | 7m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 6m 14s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 9s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 31s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 30s | | the patch passed |
| +1 :green_heart: | compile | 9m 56s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 56s | | the patch passed |
| +1 :green_heart: | javac | 9m 56s | | the patch passed |
| +1 :green_heart: | compile | 8m 59s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 59s | | the patch passed |
| +1 :green_heart: | javac | 8m 59s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/16/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/16/artifact/out/blanks-tabs.txt)
| The patch 10 line(s) with tabs. |
| +1 :green_heart: | checkstyle | 1m 56s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 36s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 55s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 261m 23s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 21s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 7s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 24s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 5s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 13s | | The patch does not
generate ASF License warnings. |
| | | 503m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602553#comment-17602553
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1242477787
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 55s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 22s | | trunk passed |
| +1 :green_heart: | compile | 10m 47s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 25s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 32s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 37s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 43s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 8s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 35s | | the patch passed |
| +1 :green_heart: | compile | 9m 51s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 51s | | the patch passed |
| +1 :green_heart: | javac | 9m 51s | | the patch passed |
| +1 :green_heart: | compile | 9m 18s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 18s | | the patch passed |
| +1 :green_heart: | javac | 9m 18s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/15/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/15/artifact/out/blanks-tabs.txt)
| The patch 10 line(s) with tabs. |
| +1 :green_heart: | checkstyle | 1m 54s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 17s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 237m 30s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 27s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 21s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 33s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 9s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 473m 54s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602510#comment-17602510
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1242351273
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 46s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 41s | | trunk passed |
| +1 :green_heart: | compile | 9m 57s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 47s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 40s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 59s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 3s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 27s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 35s | | the patch passed |
| +1 :green_heart: | compile | 9m 21s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 21s | | the patch passed |
| +1 :green_heart: | javac | 9m 21s | | the patch passed |
| +1 :green_heart: | compile | 9m 4s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 4s | | the patch passed |
| +1 :green_heart: | javac | 9m 4s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/14/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 52s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 38s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 57s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 20m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 14s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 30s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 29s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 32s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 13s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 19s | | The patch does not
generate ASF License warnings. |
| | | 469m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602056#comment-17602056 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r966545040 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,29 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + t4.applicationId, + t4.createTime, + t4.homeSubCluster, + t4.row_num +FROM ( + SELECT + t2.applicationId, + t2.createTime, + t2.homeSubCluster, + @row_num := @row_num + 1 AS row_num + FROM ( + SELECT + t.applicationId, + t.homeSubCluster, + t.createTime + FROM applicationshomesubcluster AS t + WHERE ( CASE WHEN t.homeSubCluster_IN IS NULL THEN 1 = 1 WHEN t.homeSubCluster_IN IS NOT NULL THEN homeSubCluster = homeSubCluster_IN END ) Review Comment: I will continue to optimize the code and find an easy way. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602055#comment-17602055 ] ASF GitHub Bot commented on YARN-11290: --- goiri commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r966535217 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,29 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SELECT + t4.applicationId, + t4.createTime, + t4.homeSubCluster, + t4.row_num +FROM ( + SELECT + t2.applicationId, + t2.createTime, + t2.homeSubCluster, + @row_num := @row_num + 1 AS row_num + FROM ( + SELECT + t.applicationId, + t.homeSubCluster, + t.createTime + FROM applicationshomesubcluster AS t + WHERE ( CASE WHEN t.homeSubCluster_IN IS NULL THEN 1 = 1 WHEN t.homeSubCluster_IN IS NOT NULL THEN homeSubCluster = homeSubCluster_IN END ) Review Comment: Wow! Is this the cleanest way to return the top N records? > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602041#comment-17602041
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1241339614
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 20s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 34s | | trunk passed |
| +1 :green_heart: | compile | 9m 47s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 48s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 28s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 53s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 31s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 20s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 35s | | the patch passed |
| +1 :green_heart: | compile | 9m 15s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 15s | | the patch passed |
| +1 :green_heart: | javac | 9m 15s | | the patch passed |
| +1 :green_heart: | compile | 8m 39s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 39s | | the patch passed |
| +1 :green_heart: | javac | 8m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 49s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 36s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 9s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 58s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 240m 49s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 31s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 27s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 21s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 3m 57s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 469m 37s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux 39ecd88331c4 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601521#comment-17601521
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r965333105
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,17 +261,45 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
}
-GetApplicationsHomeSubClusterResponse.newInstance(result);
+SubClusterId requestSC = request.getSubClusterId();
+List result = applications.keySet().stream()
Review Comment:
I will add sorting logic.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,24 +261,53 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-long start = clock.getTime();
-List result = new ArrayList<>();
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
try {
- for (String child : zkManager.getChildren(appsZNode)) {
-ApplicationId appId = ApplicationId.fromString(child);
-SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
-result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List children = zkManager.getChildren(appsZNode);
+ List result =
Review Comment:
I will fix it.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601511#comment-17601511 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r965316896 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml: ## @@ -4963,4 +4963,14 @@ + +yarn.federation.state-store.max-applications +1000 + + yarn federation state-store supports Review Comment: Thanks for your suggestion, I will fix it. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601510#comment-17601510 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r965313989 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql: ## @@ -111,12 +111,27 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P' ) IS NOT NULL GO CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster] +@limit int, +@homeSubCluster VARCHAR(256) AS BEGIN DECLARE @errorMessage nvarchar(4000) BEGIN TRY -SELECT [applicationId], [homeSubCluster], [createTime] -FROM [dbo].[applicationsHomeSubCluster] +SELECT +[applicationId], +[homeSubCluster], +[createTime] +FROM +(SELECT +[applicationId], +[homeSubCluster], +[createTime], +row_number() over(partition by [homeSubCluster] order by [createTime] desc) as row_num Review Comment: Yes, we should get TopN's app, I will modify the code. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601509#comment-17601509 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r965313622 ## hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql: ## @@ -122,10 +122,17 @@ BEGIN WHERE applicationId = applicationID_IN; END // -CREATE PROCEDURE sp_getApplicationsHomeSubCluster() -BEGIN - SELECT applicationId, homeSubCluster - FROM applicationsHomeSubCluster; +CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN homeSubCluster_IN varchar(256)) +BEGIN + SET @row_num = 0; + SELECT + applicationId, homeSubCluster + FROM (SELECT + applicationId, homeSubCluster, (@rownum := @rownum + 1) AS row_num +FROM applicationsHomeSubCluster) AS t +WHERE row_num <= limit_IN Review Comment: Thank you very much for your suggestion, we should sort the apps and get the topN apps, I will modify the code. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601410#comment-17601410
]
ASF GitHub Bot commented on YARN-11290:
---
goiri commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r965083185
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreStoreProcs.sql:
##
@@ -111,12 +111,27 @@ IF OBJECT_ID ( '[sp_getApplicationsHomeSubCluster]', 'P'
) IS NOT NULL
GO
CREATE PROCEDURE [dbo].[sp_getApplicationsHomeSubCluster]
+@limit int,
+@homeSubCluster VARCHAR(256)
AS BEGIN
DECLARE @errorMessage nvarchar(4000)
BEGIN TRY
-SELECT [applicationId], [homeSubCluster], [createTime]
-FROM [dbo].[applicationsHomeSubCluster]
+SELECT
+[applicationId],
+[homeSubCluster],
+[createTime]
+FROM
+(SELECT
+[applicationId],
+[homeSubCluster],
+[createTime],
+row_number() over(partition by [homeSubCluster] order by
[createTime] desc) as row_num
Review Comment:
We are getting the top N newest reservations right?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,17 +261,45 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
}
-GetApplicationsHomeSubClusterResponse.newInstance(result);
+SubClusterId requestSC = request.getSubClusterId();
+List result = applications.keySet().stream()
Review Comment:
Do we have any order here?
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreStoredProcs.sql:
##
@@ -122,10 +122,17 @@ BEGIN
WHERE applicationId = applicationID_IN;
END //
-CREATE PROCEDURE sp_getApplicationsHomeSubCluster()
-BEGIN
- SELECT applicationId, homeSubCluster
- FROM applicationsHomeSubCluster;
+CREATE PROCEDURE sp_getApplicationsHomeSubCluster(IN limit_IN int, IN
homeSubCluster_IN varchar(256))
+BEGIN
+ SET @row_num = 0;
+ SELECT
+ applicationId, homeSubCluster
+ FROM (SELECT
+ applicationId, homeSubCluster, (@rownum := @rownum + 1) AS row_num
+FROM applicationsHomeSubCluster) AS t
+WHERE row_num <= limit_IN
Review Comment:
Given that we don't order, aren't we getting random apps?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,24 +261,53 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-long start = clock.getTime();
-List result = new ArrayList<>();
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
try {
- for (String child : zkManager.getChildren(appsZNode)) {
-ApplicationId appId = ApplicationId.fromString(child);
-SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
-result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List children = zkManager.getChildren(appsZNode);
+ List result =
Review Comment:
Order?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml:
##
@@ -4963,4 +4963,14 @@
+
+yarn.federation.state-store.max-applications
+1000
+
+ yarn federation state-store supports
Review Comment:
Fix the capitalization while making it one line and add periods at the end.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601212#comment-17601212
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1239142291
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 38s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 8s | | trunk passed |
| +1 :green_heart: | compile | 10m 53s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 30s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 9m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 45s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 52s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 51s | | the patch passed |
| +1 :green_heart: | compile | 10m 10s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 10m 10s | | the patch passed |
| +1 :green_heart: | javac | 10m 10s | | the patch passed |
| +1 :green_heart: | compile | 9m 16s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 16s | | the patch passed |
| +1 :green_heart: | javac | 9m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 56s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 6m 3s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 20s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 240m 25s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 42s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 22s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 42s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 27s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 21s | | The patch does not
generate ASF License warnings. |
| | | 479m 28s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux 36ea07a14f0b 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601208#comment-17601208
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1239121352
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 52s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 30s | | trunk passed |
| +1 :green_heart: | compile | 10m 30s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 57s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 9m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 50s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 55s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 20m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 20s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 14s | | the patch passed |
| +1 :green_heart: | compile | 9m 30s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 30s | | the patch passed |
| +1 :green_heart: | javac | 9m 30s | | the patch passed |
| +1 :green_heart: | compile | 9m 27s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 27s | | the patch passed |
| +1 :green_heart: | javac | 9m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 56s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 6m 8s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 14s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 20m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 239m 15s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 39s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 26s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 31s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 18s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 23s | | The patch does not
generate ASF License warnings. |
| | | 479m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux a2a64fc272a9 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601055#comment-17601055 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r964257610 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/TestableFederationClientInterceptor.java: ## @@ -90,6 +95,7 @@ protected ApplicationClientProtocol getClientRMProxyForSubCluster( mockRMs.put(subClusterId, mockRM); } initNodeAttributes(subClusterId, mockRM); + initReservationSystem(mockRM); Review Comment: The new pr will optimize the repetitive submission of reservations and refactor some test code. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601053#comment-17601053 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on code in PR #4846: URL: https://github.com/apache/hadoop/pull/4846#discussion_r964256887 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/TestableFederationClientInterceptor.java: ## @@ -90,6 +95,7 @@ protected ApplicationClientProtocol getClientRMProxyForSubCluster( mockRMs.put(subClusterId, mockRM); } initNodeAttributes(subClusterId, mockRM); + initReservationSystem(mockRM); Review Comment: I will submit pr separately to improve this code. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601052#comment-17601052
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r964252561
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java:
##
@@ -949,7 +949,7 @@ public ReservationSubmissionResponse submitReservation(
// Second, determine whether the current ReservationId has a
corresponding subCluster.
// If it does not exist, add it. If it exists, update it.
Boolean exists = existsReservationHomeSubCluster(reservationId);
-if (!exists) {
+if (!exists || i == 0) {
Review Comment:
Reservation may be repeatedly submitted (the same reservationId is submitted
repeatedly). At this time, we should try to use the first reservation result.
If the first reservation encounters some abnormalities, find another RM for
reservation.
If a reservationId(**reservationId_1**) has successfully applied for
reserved resources in SC_1, and the **reservationId_1** is submitted
repeatedly, we should try to use the allocation result of SC_1
For the case of **reservationId_1**, the following code will return true,
```
Boolean exists = existsReservationHomeSubCluster(reservationId);
```
Then the first subcluster(SC_1) will be used instead of SC2
```
updateReservationHomeSubCluster(subClusterId, reservationId,
reservationHomeSubCluster);
```
But at this time we still want to go
`addReservationHomeSubCluster(reservationId, reservationHomeSubCluster)`
`addReservationHomeSubCluster`, will not be added repeatedly for the
existing reservationId.
I added a condition, if i=0, continue to use `addReservationHomeSubCluster`
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601048#comment-17601048
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r964252561
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java:
##
@@ -949,7 +949,7 @@ public ReservationSubmissionResponse submitReservation(
// Second, determine whether the current ReservationId has a
corresponding subCluster.
// If it does not exist, add it. If it exists, update it.
Boolean exists = existsReservationHomeSubCluster(reservationId);
-if (!exists) {
+if (!exists || i == 0) {
Review Comment:
Reservation may be repeatedly submitted (the same reservationId is submitted
repeatedly). At this time, we should try to use the first reservation result.
If the first reservation encounters some abnormalities, find another RM for
reservation.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601047#comment-17601047
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r964250673
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/records/GetApplicationsHomeSubClusterRequest.java:
##
@@ -37,4 +38,33 @@ public static GetApplicationsHomeSubClusterRequest
newInstance() {
return request;
}
+ @Private
+ @Unstable
+ public static GetApplicationsHomeSubClusterRequest
+ newInstance(SubClusterId subClusterId) {
+GetApplicationsHomeSubClusterRequest request =
+Records.newRecord(GetApplicationsHomeSubClusterRequest.class);
+request.setSubClusterId(subClusterId);
+return request;
+ }
+
+ /**
+ * Get the {@link SubClusterId} representing the unique identifier of the
+ * subcluster.
+ *
+ * @return the subcluster identifier
+ */
+ @InterfaceAudience.Public
Review Comment:
I will fix it.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601046#comment-17601046
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r964250594
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,24 +261,52 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-long start = clock.getTime();
-List result = new ArrayList<>();
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
try {
- for (String child : zkManager.getChildren(appsZNode)) {
-ApplicationId appId = ApplicationId.fromString(child);
-SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
-result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List children = zkManager.getChildren(appsZNode);
+ List result =
+ children.stream().map(child -> generateAppHomeSC(child))
+ .filter(appHomeSC -> judgeAdd(requestSC,
appHomeSC.getHomeSubCluster()))
+ .limit(maxAppsInStateStore)
+ .collect(Collectors.toList());
+ long end = clock.getTime();
+ opDurations.addGetAppsHomeSubClusterDuration(start, end);
+ LOG.info("filterSubClusterId = {}, appCount = {}.", requestSC,
result.size());
+ return GetApplicationsHomeSubClusterResponse.newInstance(result);
} catch (Exception e) {
String errMsg = "Cannot get apps: " + e.getMessage();
FederationStateStoreUtils.logAndThrowStoreException(LOG, errMsg);
}
-long end = clock.getTime();
-opDurations.addGetAppsHomeSubClusterDuration(start, end);
-return GetApplicationsHomeSubClusterResponse.newInstance(result);
+
+throw new YarnException("Cannot get app by request");
+ }
+
+ private ApplicationHomeSubCluster generateAppHomeSC(String appId) {
+try {
+ ApplicationId applicationId = ApplicationId.fromString(appId);
+ SubClusterId homeSubCluster = getApp(applicationId);
+ ApplicationHomeSubCluster app =
+ ApplicationHomeSubCluster.newInstance(applicationId, homeSubCluster);
+ return app;
+} catch (Exception ex) {
+ LOG.error("get homeSubCluster by appId = {}.", appId);
+}
+return null;
+ }
+
+ private boolean judgeAdd(SubClusterId filterSubCluster, SubClusterId
homeSubCluster) {
Review Comment:
I will re-select an appropriate method name.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601045#comment-17601045
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r964250393
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,17 +261,36 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
}
-GetApplicationsHomeSubClusterResponse.newInstance(result);
+SubClusterId requestSC = request.getSubClusterId();
+List result = applications.keySet().stream()
+.map(applicationId -> generateAppHomeSC(applicationId))
+.filter(appHomeSC -> judgeAdd(requestSC,
appHomeSC.getHomeSubCluster()))
+.limit(maxAppsInStateStore)
+.collect(Collectors.toList());
+
+LOG.info("filterSubClusterId = {}, appCount = {}.", requestSC,
result.size());
return GetApplicationsHomeSubClusterResponse.newInstance(result);
}
+ private ApplicationHomeSubCluster generateAppHomeSC(ApplicationId
applicationId) {
+SubClusterId subClusterId = applications.get(applicationId);
+return ApplicationHomeSubCluster.newInstance(applicationId, subClusterId);
+ }
+
+ private boolean judgeAdd(SubClusterId filterSubCluster, SubClusterId
homeSubCluster) {
+if (filterSubCluster == null) {
+ return true;
+} else if (filterSubCluster.equals(homeSubCluster)) {
Review Comment:
Thanks for your help reviewing the code, I will modify the code.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Attachments: SqlServer-StoredProc.png
>
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601041#comment-17601041
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1238739439
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 22s | | trunk passed |
| +1 :green_heart: | compile | 10m 23s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 7s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 9m 14s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 47s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 56s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 12s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 24s | | the patch passed |
| +1 :green_heart: | compile | 9m 16s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 16s | | the patch passed |
| +1 :green_heart: | javac | 9m 16s | | the patch passed |
| +1 :green_heart: | compile | 8m 35s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 35s | | the patch passed |
| +1 :green_heart: | javac | 8m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 55s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 46s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 9s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 58s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 34s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 22s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 29s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 27s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 17s | | The patch does not
generate ASF License warnings. |
| | | 469m 17s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux d6d029dc8f61 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600963#comment-17600963
]
ASF GitHub Bot commented on YARN-11290:
---
goiri commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r964122414
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,24 +261,52 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-long start = clock.getTime();
-List result = new ArrayList<>();
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
try {
- for (String child : zkManager.getChildren(appsZNode)) {
-ApplicationId appId = ApplicationId.fromString(child);
-SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
-result.add(app);
- }
+ long start = clock.getTime();
+ SubClusterId requestSC = request.getSubClusterId();
+ List children = zkManager.getChildren(appsZNode);
+ List result =
+ children.stream().map(child -> generateAppHomeSC(child))
+ .filter(appHomeSC -> judgeAdd(requestSC,
appHomeSC.getHomeSubCluster()))
+ .limit(maxAppsInStateStore)
+ .collect(Collectors.toList());
+ long end = clock.getTime();
+ opDurations.addGetAppsHomeSubClusterDuration(start, end);
+ LOG.info("filterSubClusterId = {}, appCount = {}.", requestSC,
result.size());
+ return GetApplicationsHomeSubClusterResponse.newInstance(result);
} catch (Exception e) {
String errMsg = "Cannot get apps: " + e.getMessage();
FederationStateStoreUtils.logAndThrowStoreException(LOG, errMsg);
}
-long end = clock.getTime();
-opDurations.addGetAppsHomeSubClusterDuration(start, end);
-return GetApplicationsHomeSubClusterResponse.newInstance(result);
+
+throw new YarnException("Cannot get app by request");
+ }
+
+ private ApplicationHomeSubCluster generateAppHomeSC(String appId) {
+try {
+ ApplicationId applicationId = ApplicationId.fromString(appId);
+ SubClusterId homeSubCluster = getApp(applicationId);
+ ApplicationHomeSubCluster app =
+ ApplicationHomeSubCluster.newInstance(applicationId, homeSubCluster);
+ return app;
+} catch (Exception ex) {
+ LOG.error("get homeSubCluster by appId = {}.", appId);
+}
+return null;
+ }
+
+ private boolean judgeAdd(SubClusterId filterSubCluster, SubClusterId
homeSubCluster) {
Review Comment:
`judgeAdd` is a weird name fot this function.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/FederationStateStoreBaseTest.java:
##
@@ -410,6 +412,89 @@ public void testGetApplicationsHomeSubCluster() throws
Exception {
Assert.assertTrue(result.getAppsHomeSubClusters().contains(ahsc2));
}
+ @Test
+ public void testGetApplicationsHomeSubClusterEmpty() throws Exception {
+LambdaTestUtils.intercept(YarnException.class,
+"Missing getApplicationsHomeSubCluster request",
+() -> stateStore.getApplicationsHomeSubCluster(null));
+ }
+
+ @Test
+ public void testGetApplicationsHomeSubClusterFilter() throws Exception {
+// Add ApplicationHomeSC - SC1
+long now = Time.now();
+
+Set appHomeSubClusters = new HashSet<>();
+
+for (int i = 0; i < 10; i++) {
Review Comment:
Make 10 and 20 constants.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/clientrm/TestableFederationClientInterceptor.java:
##
@@ -90,6 +95,7 @@ protected ApplicationClientProtocol
getClientRMProxyForSubCluster(
mockRMs.put(subClusterId, mockRM);
}
initNodeAttributes(subClusterId, mockRM);
+ initReservationSystem(mockRM);
Review Comment:
This seems out of scope.
Should we do it in a separate PR?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,17 +261,36 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600886#comment-17600886
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1238405786
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 50s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 25s | | trunk passed |
| +1 :green_heart: | compile | 10m 36s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 46s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 9m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 35s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 40s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 57s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 44s | | the patch passed |
| +1 :green_heart: | compile | 10m 19s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 10m 19s | | the patch passed |
| +1 :green_heart: | javac | 10m 19s | | the patch passed |
| +1 :green_heart: | compile | 9m 22s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 22s | | the patch passed |
| +1 :green_heart: | javac | 9m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 54s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 19s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 37s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 239m 38s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 34s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 33s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 32s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 12s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 21s | | The patch does not
generate ASF License warnings. |
| | | 474m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux d11f227539ba 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600835#comment-17600835 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1238308788 @goiri Please help to review the code again, thank you very much! This PR changes the execution script of the stored procedure, I will check the script against the major version of the database. **SQLServer** The change script of the stored procedure has been verified on SQL Server 2012 Enterprise And SQL Server 2008 R2 Enterprise, I will verify it on the following version as soon as possible - [ ] SQL Server 2008 R2 Enterprise - [ ] SQL Server 2012 Enterprise SQL Server 2016 Enterprise SQL Server 2017 Enterprise SQL Server 2019 Enterprise **MySQL** MySQL 5.7 and above must be required, MySQL 5.5 and MySQL 5.6 cannot be supported. MySQL-5.5 and MySQL-5.6 Can't Create Table membership and policies, for the following reasons: 1.The primary key of the membership table is subClusterId varchar(256) , Mysql will prompt the following error: Specified key was too long; max key length is 767 bytes. 2.The primary key of the policies table is subClusterId varchar(256) , Mysql will prompt the following error: Specified key was too long; max key length is 767 bytes. - [ ] MySQL 5.7 - [ ] MySQL 8.0 > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600789#comment-17600789
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1238215896
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 40s | | trunk passed |
| +1 :green_heart: | compile | 10m 12s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 3s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 9m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 43s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 39s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 55s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 8s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 24s | | the patch passed |
| +1 :green_heart: | compile | 9m 16s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 16s | | the patch passed |
| +1 :green_heart: | javac | 9m 16s | | the patch passed |
| +1 :green_heart: | compile | 8m 54s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 54s | | the patch passed |
| +1 :green_heart: | javac | 8m 54s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 54s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/8/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 165 unchanged
- 0 fixed = 168 total (was 165) |
| +1 :green_heart: | mvnsite | 8m 23s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 49s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 20s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 6s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 30s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 33s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 28s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 35s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 24s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 23s | | The patch does not
generate ASF License warnings. |
| | | 472m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600655#comment-17600655
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1237793763
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 50s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 27s | | trunk passed |
| +1 :green_heart: | compile | 9m 53s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 39s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 29s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 42s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 59s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 24s | | the patch passed |
| +1 :green_heart: | compile | 9m 15s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 15s | | the patch passed |
| +1 :green_heart: | javac | 9m 15s | | the patch passed |
| +1 :green_heart: | compile | 8m 36s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 36s | | the patch passed |
| +1 :green_heart: | javac | 8m 36s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 50s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/7/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 165 unchanged
- 0 fixed = 168 total (was 165) |
| +1 :green_heart: | mvnsite | 8m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 50s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 9s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 41s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 31s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 40s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 35s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 10s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 22s | | The patch does not
generate ASF License warnings. |
| | | 468m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600527#comment-17600527
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1237512502
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 38s | | trunk passed |
| +1 :green_heart: | compile | 9m 56s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 42s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 15s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 43s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 9s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 40s | | the patch passed |
| +1 :green_heart: | compile | 9m 7s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 7s | | the patch passed |
| +1 :green_heart: | javac | 9m 7s | | the patch passed |
| +1 :green_heart: | compile | 8m 38s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 38s | | the patch passed |
| +1 :green_heart: | javac | 8m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 47s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/6/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 165 unchanged
- 0 fixed = 168 total (was 165) |
| +1 :green_heart: | mvnsite | 8m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 44s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 14s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 237m 48s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/6/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch passed. |
| +1 :green_heart: | unit | 1m 32s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 30s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 39s | | hadoop-yarn-server-common in
the patch passed. |
| -1 :x: | unit | 4m 27s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/6/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
| hadoop-yarn-server-router in the patch passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not
generate ASF License warnings. |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600463#comment-17600463 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1237189620 @goiri Please help to review the code again, thank you very much! > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600331#comment-17600331
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1236867158
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 21s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 42s | | trunk passed |
| +1 :green_heart: | compile | 10m 16s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 13s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 9m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 56s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 14s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 39s | | the patch passed |
| +1 :green_heart: | compile | 9m 31s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 31s | | the patch passed |
| +1 :green_heart: | javac | 9m 31s | | the patch passed |
| +1 :green_heart: | compile | 9m 1s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 1s | | the patch passed |
| +1 :green_heart: | javac | 9m 1s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 54s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/5/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 165 unchanged
- 0 fixed = 168 total (was 165) |
| +1 :green_heart: | mvnsite | 8m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 48s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 10s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 237m 14s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 35s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 21s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 28s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 13s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 37s | | The patch does not
generate ASF License warnings. |
| | | 471m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600223#comment-17600223
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1236593039
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 26s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 40s | | trunk passed |
| +1 :green_heart: | compile | 9m 53s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 49s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 54s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 4s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 46s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 46s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 3s | | the patch passed |
| +1 :green_heart: | compile | 9m 14s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 14s | | the patch passed |
| +1 :green_heart: | javac | 9m 14s | | the patch passed |
| +1 :green_heart: | compile | 8m 38s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 38s | | the patch passed |
| +1 :green_heart: | javac | 8m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 50s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 10s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 32s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 42s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 38s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 29s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 36s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 456m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
| uname | Linux 28b5bc65f642 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 354b8b3ba130fe90e92cfddaddce8e9ebfa3fa44 |
| Default Java | Private
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600169#comment-17600169 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 commented on PR #4846: URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1236511195 This PR changes the execution script of the stored procedure, I will check the script against the major version of the database. SQL Server The change script of the stored procedure has been verified on SQL Server 2012 Enterprise, I will verify it on the following version as soon as possible - SQL Server 2008 R2 Enterprise - SQL Server 2016 Enterprise - SQL Server 2017 Enterprise - SQL Server 2019 Enterprise MySQL - MySQL 5.5 - MySQL 5.6 - MySQL 5.7 - MySQL 8.0 > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: SqlServer-StoredProc.png > > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600116#comment-17600116
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1236421286
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 12s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 4s | | trunk passed |
| +1 :green_heart: | compile | 10m 26s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 0s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 3s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 48s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 0s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 4s | | the patch passed |
| +1 :green_heart: | compile | 9m 15s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 15s | | the patch passed |
| +1 :green_heart: | javac | 9m 15s | | the patch passed |
| +1 :green_heart: | compile | 8m 35s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 35s | | the patch passed |
| +1 :green_heart: | javac | 8m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 47s | | the patch passed |
| +1 :green_heart: | mvnsite | 7m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 16s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 35s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 54s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 238m 7s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/3/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch passed. |
| +1 :green_heart: | unit | 1m 28s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 26s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 42s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 26s | | The patch does not
generate ASF License warnings. |
| | | 458m 39s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.router.clientrm.TestFederationClientInterceptor |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint cc
buflint bufcompat |
|
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599955#comment-17599955
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1236140992
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 32s | | trunk passed |
| +1 :green_heart: | compile | 9m 51s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 40s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 52s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 11s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 17m 26s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 53s | | the patch passed |
| +1 :green_heart: | compile | 9m 13s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 13s | | the patch passed |
| +1 :green_heart: | javac | 9m 13s | | the patch passed |
| +1 :green_heart: | compile | 8m 38s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 38s | | the patch passed |
| +1 :green_heart: | javac | 8m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 49s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 165 unchanged
- 0 fixed = 168 total (was 165) |
| +1 :green_heart: | mvnsite | 7m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 13s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 42s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | spotbugs | 12m 2s |
[/new-spotbugs-hadoop-yarn-project_hadoop-yarn.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/2/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn.html)
| hadoop-yarn-project/hadoop-yarn generated 2 new + 0 unchanged - 0 fixed = 2
total (was 0) |
| -1 :x: | spotbugs | 1m 34s |
[/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/2/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common.html)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shadedclient | 20m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 237m 32s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 38s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 31s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: |
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599821#comment-17599821
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r962114313
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,14 +261,33 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
+List result = new ArrayList<>();
+List applicationIdList =
+applications.keySet().stream().collect(Collectors.toList());
+
+SubClusterId requestSubClusterId = request.getSubClusterId();
+int appCount = 0;
+for (int i = 0; i < applicationIdList.size(); i++) {
+ if (appCount >= maxAppsInStateStore) {
+break;
+ }
+ ApplicationId applicationId = applicationIdList.get(i);
+ SubClusterId subClusterId = applications.get(applicationId);
+ // If the requestSubClusterId that needs to be filtered in the request
+ // is inconsistent with the SubClusterId in the data, continue to the
next round
+ if (requestSubClusterId != null &&
!requestSubClusterId.equals(subClusterId)){
+continue;
+ }
+ result.add(ApplicationHomeSubCluster.newInstance(applicationId,
subClusterId));
Review Comment:
I will fix it.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599812#comment-17599812
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r962108067
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,14 +261,33 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
+List result = new ArrayList<>();
+List applicationIdList =
+applications.keySet().stream().collect(Collectors.toList());
+
+SubClusterId requestSubClusterId = request.getSubClusterId();
+int appCount = 0;
+for (int i = 0; i < applicationIdList.size(); i++) {
+ if (appCount >= maxAppsInStateStore) {
Review Comment:
Thanks for your suggestion, I will modify the code!
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599811#comment-17599811
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r962108019
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,14 +261,33 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
+List result = new ArrayList<>();
+List applicationIdList =
+applications.keySet().stream().collect(Collectors.toList());
+
+SubClusterId requestSubClusterId = request.getSubClusterId();
+int appCount = 0;
+for (int i = 0; i < applicationIdList.size(); i++) {
+ if (appCount >= maxAppsInStateStore) {
+break;
+ }
+ ApplicationId applicationId = applicationIdList.get(i);
+ SubClusterId subClusterId = applications.get(applicationId);
+ // If the requestSubClusterId that needs to be filtered in the request
+ // is inconsistent with the SubClusterId in the data, continue to the
next round
+ if (requestSubClusterId != null &&
!requestSubClusterId.equals(subClusterId)){
+continue;
+ }
+ result.add(ApplicationHomeSubCluster.newInstance(applicationId,
subClusterId));
+ appCount++;
}
-GetApplicationsHomeSubClusterResponse.newInstance(result);
Review Comment:
This line of code should be an extra line of code, which has no practical
significance. The following is directly constructed and returned.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599736#comment-17599736
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r962057944
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,23 +260,41 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
long start = clock.getTime();
List result = new ArrayList<>();
+SubClusterId requestSubClusterId = request.getSubClusterId();
+int appCount = 0;
try {
- for (String child : zkManager.getChildren(appsZNode)) {
+ List childrens = zkManager.getChildren(appsZNode);
+ for (String child : childrens) {
+if (appCount >= maxAppsInStateStore) {
+ break;
+}
ApplicationId appId = ApplicationId.fromString(child);
SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
+// If the requestSubClusterId that needs to be filtered in the request
+// is inconsistent with the SubClusterId in the data, continue to the
next round
+if (requestSubClusterId != null &&
!requestSubClusterId.equals(homeSubCluster)) {
+ continue;
Review Comment:
I will fix it.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599735#comment-17599735
]
ASF GitHub Bot commented on YARN-11290:
---
slfan1989 commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r962057774
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/SQLFederationStateStore.java:
##
@@ -726,13 +731,23 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
CallableStatement cstmt = null;
ResultSet rs = null;
-List appsHomeSubClusters =
-new ArrayList();
+List appsHomeSubClusters = new ArrayList<>();
try {
cstmt = getCallableStatement(CALL_SP_GET_APPLICATIONS_HOME_SUBCLUSTER);
+ cstmt.setInt("limit_IN", maxAppsInStateStore);
+ String homeSubClusterIN = null;;
Review Comment:
I will fix it.
> Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
> -
>
> Key: YARN-11290
> URL: https://issues.apache.org/jira/browse/YARN-11290
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1. Currently this interface returns the number of apps in all sub-clusters,
> increasing the limit on the number of query apps, limited to 1000 apps.
> 2. Allows to query the App based on the specified HomeSubCluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599706#comment-17599706
]
ASF GitHub Bot commented on YARN-11290:
---
goiri commented on code in PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#discussion_r961990524
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,14 +261,33 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
+List result = new ArrayList<>();
+List applicationIdList =
+applications.keySet().stream().collect(Collectors.toList());
+
+SubClusterId requestSubClusterId = request.getSubClusterId();
+int appCount = 0;
+for (int i = 0; i < applicationIdList.size(); i++) {
+ if (appCount >= maxAppsInStateStore) {
Review Comment:
Move this to the for condition.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -255,23 +260,41 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
long start = clock.getTime();
List result = new ArrayList<>();
+SubClusterId requestSubClusterId = request.getSubClusterId();
+int appCount = 0;
try {
- for (String child : zkManager.getChildren(appsZNode)) {
+ List childrens = zkManager.getChildren(appsZNode);
+ for (String child : childrens) {
+if (appCount >= maxAppsInStateStore) {
+ break;
+}
ApplicationId appId = ApplicationId.fromString(child);
SubClusterId homeSubCluster = getApp(appId);
-ApplicationHomeSubCluster app =
-ApplicationHomeSubCluster.newInstance(appId, homeSubCluster);
+// If the requestSubClusterId that needs to be filtered in the request
+// is inconsistent with the SubClusterId in the data, continue to the
next round
+if (requestSubClusterId != null &&
!requestSubClusterId.equals(homeSubCluster)) {
+ continue;
Review Comment:
reverse the if
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/SQLFederationStateStore.java:
##
@@ -726,13 +731,23 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
CallableStatement cstmt = null;
ResultSet rs = null;
-List appsHomeSubClusters =
-new ArrayList();
+List appsHomeSubClusters = new ArrayList<>();
try {
cstmt = getCallableStatement(CALL_SP_GET_APPLICATIONS_HOME_SUBCLUSTER);
+ cstmt.setInt("limit_IN", maxAppsInStateStore);
+ String homeSubClusterIN = null;;
Review Comment:
Extra ;
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -255,14 +261,33 @@ public GetApplicationHomeSubClusterResponse
getApplicationHomeSubCluster(
@Override
public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(
GetApplicationsHomeSubClusterRequest request) throws YarnException {
-List result =
-new ArrayList();
-for (Entry e : applications.entrySet()) {
- result
- .add(ApplicationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
+
+if (request == null) {
+ throw new YarnException("Missing getApplicationsHomeSubCluster request");
+}
+
+List result = new ArrayList<>();
+List applicationIdList =
+applications.keySet().stream().collect(Collectors.toList());
+
+SubClusterId requestSubClusterId =
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599675#comment-17599675
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1235812702
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 29s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 35s | | trunk passed |
| +1 :green_heart: | compile | 9m 51s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 38s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 7m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 50s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 17s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 15m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 41s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 6s | | the patch passed |
| +1 :green_heart: | compile | 9m 9s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 9s | | the patch passed |
| +1 :green_heart: | javac | 9m 9s | | the patch passed |
| +1 :green_heart: | compile | 8m 30s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 30s | | the patch passed |
| +1 :green_heart: | javac | 8m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 46s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 6 new + 165 unchanged
- 0 fixed = 171 total (was 165) |
| +1 :green_heart: | mvnsite | 6m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 19s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 45s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 16m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 16s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 238m 45s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn in the patch passed. |
| -1 :x: | unit | 1m 33s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt)
| hadoop-yarn-api in the patch passed. |
| +1 :green_heart: | unit | 3m 28s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 15s | | The patch does not
generate ASF License warnings. |
| | | 442m 18s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.yarn.conf.TestYarnConfigurationFields |
| | hadoop.yarn.conf.TestYarnConfigurationFields |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[ https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599450#comment-17599450 ] ASF GitHub Bot commented on YARN-11290: --- slfan1989 opened a new pull request, #4846: URL: https://github.com/apache/hadoop/pull/4846 JIRA: YARN-11290. Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster. 1. Currently this interface returns the number of apps in all sub-clusters, increasing the limit on the number of query apps, limited to 1000 apps. 2. Allows to query the App based on the specified HomeSubCluster. > Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster > - > > Key: YARN-11290 > URL: https://issues.apache.org/jira/browse/YARN-11290 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > > 1. Currently this interface returns the number of apps in all sub-clusters, > increasing the limit on the number of query apps, limited to 1000 apps. > 2. Allows to query the App based on the specified HomeSubCluster. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
