[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579334#comment-17579334 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1214296791 @goiri Thank you very much for your help reviewing the code! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579265#comment-17579265 ] ASF GitHub Bot commented on YARN-11236: --- goiri merged PR #4711: URL: https://github.com/apache/hadoop/pull/4711 > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579222#comment-17579222
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1214049418
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 29s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 45s | | trunk passed |
| +1 :green_heart: | compile | 4m 5s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 54s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 35s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 39s | | the patch passed |
| +1 :green_heart: | compile | 3m 50s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 50s | | the patch passed |
| -1 :x: | javac | 3m 50s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/9/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 21s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 21s | | the patch passed |
| -1 :x: | javac | 3m 21s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/9/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 10s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 50s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 33s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 2s | |
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579211#comment-17579211
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1213696635
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 24s | | trunk passed |
| +1 :green_heart: | compile | 4m 5s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 55s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 40s | | the patch passed |
| +1 :green_heart: | compile | 3m 47s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 47s | | the patch passed |
| -1 :x: | javac | 3m 47s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/8/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 17s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 17s | | the patch passed |
| -1 :x: | javac | 3m 17s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/8/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 12s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 51s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 3s | |
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579145#comment-17579145 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1213529914 > Can we get a clean build? @goiri The warning of proto seems to be unavoidable, this part of the code is automatically generated. ``` hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/generated-sources/java/org/apache/hadoop/yarn/federation/proto/YarnServerFederationProtos.java:15099:107:[deprecation] PARSER in ReservationIdProto has been deprecated hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/generated-sources/java/org/apache/hadoop/yarn/federation/proto/YarnServerFederationProtos.java:18160:107:[deprecation] PARSER in ReservationIdProto has been deprecated hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/generated-sources/java/org/apache/hadoop/yarn/federation/proto/YarnServerFederationProtos.java:20584:107:[deprecation] PARSER in ReservationIdProto has been deprecated ``` > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579036#comment-17579036 ] ASF GitHub Bot commented on YARN-11236: --- goiri commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1213280257 Can we get a clean build? > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579029#comment-17579029 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1213264291 @goiri Can you help to merge this pr into trunk branch? I will follow up with YARN-11252, Thank you very much! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578113#comment-17578113
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1211131931
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 19s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 21s | | trunk passed |
| +1 :green_heart: | compile | 4m 4s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 17s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 56s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 41s | | the patch passed |
| +1 :green_heart: | compile | 3m 48s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 48s | | the patch passed |
| -1 :x: | javac | 3m 48s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/7/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 16s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 16s | | the patch passed |
| -1 :x: | javac | 3m 16s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/7/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 11s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 52s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 2s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart:
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578112#comment-17578112
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1211127930
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 12s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 19s | | trunk passed |
| +1 :green_heart: | compile | 4m 17s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 25s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 48s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 57s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 12s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 37s | | the patch passed |
| +1 :green_heart: | compile | 3m 59s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 59s | | the patch passed |
| -1 :x: | javac | 3m 59s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/6/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
444 unchanged - 0 fixed = 447 total (was 444) |
| +1 :green_heart: | compile | 3m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 18s | | the patch passed |
| -1 :x: | javac | 3m 18s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/6/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 368 unchanged - 0 fixed = 371 total (was 368) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 8s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 16s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 5s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart:
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578108#comment-17578108
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1211103628
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 11s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 11s | | trunk passed |
| +1 :green_heart: | compile | 4m 26s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 44s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 51s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 32s | | the patch passed |
| +1 :green_heart: | compile | 3m 55s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 55s | | the patch passed |
| -1 :x: | javac | 3m 55s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 15s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 15s | | the patch passed |
| -1 :x: | javac | 3m 15s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 4s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 52s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart:
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578032#comment-17578032 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1210819769 @goiri Please help to review the code again, Thank you very much! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578030#comment-17578030
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r942573705
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/policies/router/BaseRouterPoliciesTest.java:
##
@@ -126,4 +136,62 @@ public void testNullReservationContext() throws Exception {
() -> policy.getReservationHomeSubcluster(null));
}
+ @Test
+ public void testUnknownReservation() throws Exception {
+
+long now = Time.now();
+ReservationSubmissionRequest resReq = getReservationSubmissionRequest();
+ReservationId reservationId = ReservationId.newInstance(now, 1);
+when(resReq.getQueue()).thenReturn("queue1");
+when(resReq.getReservationId()).thenReturn(reservationId);
+
+// route an application that uses this app
+ApplicationSubmissionContext applicationSubmissionContext =
+ApplicationSubmissionContext.newInstance(
+ApplicationId.newInstance(now, 1), "app1", "queue1",
Priority.newInstance(1),
+null, false, false, 1, null, null, false);
+
+applicationSubmissionContext.setReservationID(resReq.getReservationId());
+FederationRouterPolicy policy = (FederationRouterPolicy) getPolicy();
+
+LambdaTestUtils.intercept(YarnException.class,
+"Reservation " + reservationId + " does not exist",
+() -> policy.getHomeSubcluster(applicationSubmissionContext, new
ArrayList<>()));
+ }
+
+ @Test
+ public void testFollowReservation() throws YarnException {
+
+long now = Time.now();
+ReservationSubmissionRequest resReq = getReservationSubmissionRequest();
+when(resReq.getQueue()).thenReturn("queue1");
+when(resReq.getReservationId())
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578028#comment-17578028
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r942573421
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/records/impl/pb/ReservationHomeSubClusterPBImpl.java:
##
@@ -0,0 +1,167 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.records.impl.pb;
+
+import org.apache.hadoop.classification.InterfaceAudience.Private;
+import org.apache.hadoop.classification.InterfaceStability.Unstable;
+import org.apache.hadoop.yarn.api.records.ReservationId;
+import org.apache.hadoop.yarn.api.records.impl.pb.ReservationIdPBImpl;
+import
org.apache.hadoop.yarn.federation.proto.YarnServerFederationProtos.ReservationHomeSubClusterProto;
+import
org.apache.hadoop.yarn.federation.proto.YarnServerFederationProtos.ReservationHomeSubClusterProtoOrBuilder;
+import
org.apache.hadoop.yarn.federation.proto.YarnServerFederationProtos.SubClusterIdProto;
+import org.apache.hadoop.yarn.proto.YarnProtos.ReservationIdProto;
+import
org.apache.hadoop.yarn.server.federation.store.records.ReservationHomeSubCluster;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterId;
+
+import org.apache.hadoop.thirdparty.protobuf.TextFormat;
+
+/**
+ * Protocol buffer based implementation of {@link ReservationHomeSubCluster}.
+ */
+@Private
+@Unstable
+public class ReservationHomeSubClusterPBImpl extends ReservationHomeSubCluster
{
+
+ private ReservationHomeSubClusterProto proto =
+ ReservationHomeSubClusterProto.getDefaultInstance();
+ private ReservationHomeSubClusterProto.Builder builder = null;
+ private boolean viaProto = false;
+
+ private ReservationId reservationId = null;
+ private SubClusterId homeSubCluster = null;
+
+ public ReservationHomeSubClusterPBImpl() {
+builder = ReservationHomeSubClusterProto.newBuilder();
+ }
+
+ public ReservationHomeSubClusterPBImpl(ReservationHomeSubClusterProto proto)
{
+this.proto = proto;
+viaProto = true;
+ }
+
+ public ReservationHomeSubClusterProto getProto() {
+mergeLocalToProto();
+proto = viaProto ? proto : builder.build();
+viaProto = true;
+return proto;
+ }
+
+ private void mergeLocalToProto() {
+if (viaProto) {
+ maybeInitBuilder();
+}
+mergeLocalToBuilder();
+proto = builder.build();
+viaProto = true;
+ }
+
+ private void maybeInitBuilder() {
+if (viaProto || builder == null) {
+ builder = ReservationHomeSubClusterProto.newBuilder(proto);
+}
+viaProto = false;
+ }
+
+ private void mergeLocalToBuilder() {
+if (this.reservationId != null) {
+ builder.setReservationId(convertToProtoFormat(this.reservationId));
+}
+if (this.homeSubCluster != null) {
+ builder.setHomeSubCluster(convertToProtoFormat(this.homeSubCluster));
+}
+ }
+
+ @Override
+ public int hashCode() {
+return getProto().hashCode();
+ }
+
+ @Override
+ public boolean equals(Object other) {
+if (other == null) {
+ return false;
+}
+if (other.getClass().isAssignableFrom(this.getClass())) {
+ return this.getProto().equals(this.getClass().cast(other).getProto());
+}
+return false;
+ }
+
+ @Override
+ public String toString() {
+return TextFormat.shortDebugString(getProto());
+ }
+
+ @Override
+ public ReservationId getReservationId() {
+ReservationHomeSubClusterProtoOrBuilder p = viaProto ? proto : builder;
+if (!p.hasReservationId()) {
+ return null;
+}
+this.reservationId = convertFromProtoFormat(p.getReservationId());
+return reservationId;
+ }
+
+ @Override
+ public void setReservationId(ReservationId reservationId) {
+maybeInitBuilder();
+if (reservationId == null) {
Review Comment:
Thanks for your suggestion, I will modify the code.
##
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578019#comment-17578019
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r942540670
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new YarnException("Reservation " + reservationId + " does not
exist");
+}
+SubClusterId subClusterId = reservations.get(reservationId);
+return GetReservationHomeSubClusterResponse.newInstance(
+ReservationHomeSubCluster.newInstance(reservationId, subClusterId));
+ }
+
+ @Override
+ public GetReservationsHomeSubClusterResponse getReservationsHomeSubCluster(
+ GetReservationsHomeSubClusterRequest request) throws YarnException {
+List result = new ArrayList<>();
+
+for (Entry entry : reservations.entrySet()) {
+ ReservationId key = entry.getKey();
Review Comment:
I will fix it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/records/ReservationHomeSubCluster.java:
##
@@ -0,0 +1,130 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.records;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.commons.lang3.builder.HashCodeBuilder;
+import org.apache.hadoop.classification.InterfaceAudience.Private;
+import org.apache.hadoop.classification.InterfaceAudience.Public;
+import org.apache.hadoop.classification.InterfaceStability.Unstable;
+import org.apache.hadoop.yarn.api.records.ReservationId;
+import org.apache.hadoop.yarn.util.Records;
+
+/**
+ *
+ * ReservationHomeSubCluster is a report of the runtime information of the
+ * reservation that is running in the federated cluster.
+ *
+ *
+ * It includes information such as:
+ *
+ * {@link ReservationId}
+ * {@link SubClusterId}
+ *
+ *
+ */
+@Private
+@Unstable
+public abstract class ReservationHomeSubCluster {
+
+ @Private
+ @Unstable
+ public static ReservationHomeSubCluster newInstance(ReservationId appId,
+ SubClusterId homeSubCluster) {
+ReservationHomeSubCluster appMapping =
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578016#comment-17578016
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r942539432
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
Review Comment:
I will fix it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new YarnException("Reservation " + reservationId + " does not
exist");
+}
+SubClusterId subClusterId = reservations.get(reservationId);
+return GetReservationHomeSubClusterResponse.newInstance(
+ReservationHomeSubCluster.newInstance(reservationId, subClusterId));
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17578017#comment-17578017
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r942539715
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new YarnException("Reservation " + reservationId + " does not
exist");
+}
+SubClusterId subClusterId = reservations.get(reservationId);
+return GetReservationHomeSubClusterResponse.newInstance(
+ReservationHomeSubCluster.newInstance(reservationId, subClusterId));
+ }
+
+ @Override
+ public GetReservationsHomeSubClusterResponse getReservationsHomeSubCluster(
+ GetReservationsHomeSubClusterRequest request) throws YarnException {
+List result = new ArrayList<>();
+
+for (Entry entry : reservations.entrySet()) {
+ ReservationId key = entry.getKey();
+ SubClusterId value = entry.getValue();
+ ReservationHomeSubCluster homeSubCluster =
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577663#comment-17577663
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r941870965
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577640#comment-17577640
]
ASF GitHub Bot commented on YARN-11236:
---
goiri commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r941830630
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new YarnException("Reservation " + reservationId + " does not
exist");
+}
+SubClusterId subClusterId = reservations.get(reservationId);
+return GetReservationHomeSubClusterResponse.newInstance(
+ReservationHomeSubCluster.newInstance(reservationId, subClusterId));
+ }
+
+ @Override
+ public GetReservationsHomeSubClusterResponse getReservationsHomeSubCluster(
+ GetReservationsHomeSubClusterRequest request) throws YarnException {
+List result = new ArrayList<>();
+
+for (Entry entry : reservations.entrySet()) {
+ ReservationId key = entry.getKey();
+ SubClusterId value = entry.getValue();
+ ReservationHomeSubCluster homeSubCluster =
Review Comment:
Single line
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new YarnException("Reservation " + reservationId + " does not
exist");
+}
+SubClusterId subClusterId = reservations.get(reservationId);
+return GetReservationHomeSubClusterResponse.newInstance(
+ReservationHomeSubCluster.newInstance(reservationId, subClusterId));
+ }
+
+ @Override
+ public GetReservationsHomeSubClusterResponse getReservationsHomeSubCluster(
+ GetReservationsHomeSubClusterRequest request) throws YarnException {
+List result = new ArrayList<>();
+
+for (Entry entry : reservations.entrySet()) {
+ ReservationId key = entry.getKey();
Review Comment:
better names than key/value; reservationId, subclusterId
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,45 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
Review Comment:
Single line
##
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577502#comment-17577502 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1209574795 @goiri Please help to review the code again, Thank you very much! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576817#comment-17576817 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1208215384 @goiri Please help to review the code again, Thank you very much! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576364#comment-17576364
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1207383885
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 2s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 20s | | trunk passed |
| +1 :green_heart: | compile | 4m 2s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 9s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 54s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 54s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 39s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 40s | | the patch passed |
| +1 :green_heart: | compile | 3m 50s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 50s | | the patch passed |
| -1 :x: | javac | 3m 50s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/4/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 16s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 16s | | the patch passed |
| -1 :x: | javac | 3m 16s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/4/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 14s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 50s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 2s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart:
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576333#comment-17576333
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939620389
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/records/ReservationHomeSubCluster.java:
##
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.records;
+
+import org.apache.hadoop.classification.InterfaceAudience.Private;
+import org.apache.hadoop.classification.InterfaceAudience.Public;
+import org.apache.hadoop.classification.InterfaceStability.Unstable;
+import org.apache.hadoop.yarn.api.records.ReservationId;
+import org.apache.hadoop.yarn.util.Records;
+
+/**
+ *
+ * ReservationHomeSubCluster is a report of the runtime information of the
+ * reservation that is running in the federated cluster.
+ *
+ *
+ * It includes information such as:
+ *
+ * {@link ReservationId}
+ * {@link SubClusterId}
+ *
+ *
+ */
+@Private
+@Unstable
+public abstract class ReservationHomeSubCluster {
+
+ @Private
+ @Unstable
+ public static ReservationHomeSubCluster newInstance(ReservationId appId,
+ SubClusterId homeSubCluster) {
+ReservationHomeSubCluster appMapping =
+Records.newRecord(ReservationHomeSubCluster.class);
+appMapping.setReservationId(appId);
+appMapping.setHomeSubCluster(homeSubCluster);
+return appMapping;
+ }
+
+ /**
+ * Get the {@link ReservationId} representing the unique identifier of the
+ * Reservation.
+ *
+ * @return the reservation identifier
+ */
+ @Public
+ @Unstable
+ public abstract ReservationId getReservationId();
+
+ /**
+ * Set the {@link ReservationId} representing the unique identifier of the
+ * Reservation.
+ *
+ * @param reservationId the reservation identifier
+ */
+ @Private
+ @Unstable
+ public abstract void setReservationId(ReservationId reservationId);
+
+ /**
+ * Get the {@link SubClusterId} representing the unique identifier of the
home
+ * subcluster in which the reservation is mapped to.
+ *
+ * @return the home subcluster identifier
+ */
+ @Public
+ @Unstable
+ public abstract SubClusterId getHomeSubCluster();
+
+ /**
+ * Set the {@link SubClusterId} representing the unique identifier of the
home
+ * subcluster in which the ReservationMaster of the reservation is running.
+ *
+ * @param homeSubCluster the home subcluster identifier
+ */
+ @Private
+ @Unstable
+ public abstract void setHomeSubCluster(SubClusterId homeSubCluster);
+
+ @Override
+ public boolean equals(Object obj) {
+if (this == obj) {
Review Comment:
I will refactor the code using EqualsBuilder and HashBuilder.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576328#comment-17576328
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939617042
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/policies/router/BaseRouterPoliciesTest.java:
##
@@ -126,4 +134,55 @@ public void testNullReservationContext() throws Exception {
() -> policy.getReservationHomeSubcluster(null));
}
+ @Test
+ public void testUnknownReservation() throws Exception {
+ReservationSubmissionRequest resReq = getReservationSubmissionRequest();
+ReservationId reservationId =
ReservationId.newInstance(System.currentTimeMillis(), 1);
+when(resReq.getQueue()).thenReturn("queue1");
+when(resReq.getReservationId()).thenReturn(reservationId);
+
+// route an application that uses this app
+ApplicationSubmissionContext applicationSubmissionContext =
+ApplicationSubmissionContext.newInstance(
+ApplicationId.newInstance(System.currentTimeMillis(), 1), "app1",
+"queue1", Priority.newInstance(1), null, false, false, 1, null,
null, false);
+
+applicationSubmissionContext.setReservationID(resReq.getReservationId());
+FederationRouterPolicy policy = (FederationRouterPolicy) getPolicy();
+
+LambdaTestUtils.intercept(YarnException.class,
+"Reservation " + reservationId + " does not exist",
+() -> policy.getHomeSubcluster(applicationSubmissionContext, new
ArrayList<>()));
+ }
+
+ @Test
+ public void testFollowReservation() throws YarnException {
+ReservationSubmissionRequest resReq = getReservationSubmissionRequest();
+when(resReq.getQueue()).thenReturn("queue1");
+when(resReq.getReservationId())
+.thenReturn(ReservationId.newInstance(System.currentTimeMillis(), 1));
+
+// first we invoke a reservation placement
+SubClusterId chosen = ((FederationRouterPolicy) getPolicy())
+.getReservationHomeSubcluster(resReq);
+
+// add this to the store
+this.getFederationPolicyContext().getFederationStateStoreFacade()
+.addReservationHomeSubCluster(ReservationHomeSubCluster.newInstance(
+ resReq.getReservationId(), chosen));
+
+// route an application that uses this app
+ApplicationSubmissionContext applicationSubmissionContext =
+ApplicationSubmissionContext.newInstance(
+ApplicationId.newInstance(System.currentTimeMillis(), 1), "app1",
+"queue1", Priority.newInstance(1), null, false, false, 1, null,
+null, false);
+applicationSubmissionContext.setReservationID(resReq.getReservationId());
+SubClusterId chosen2 = ((FederationRouterPolicy) getPolicy())
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576327#comment-17576327
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939616561
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/utils/FederationStateStoreFacade.java:
##
@@ -400,6 +406,40 @@ public Configuration getConf() {
return this.conf;
}
+
+
+ /**
+ * Adds the home {@link SubClusterId} for the specified {@link
ReservationId}.
+ *
+ * @param appHomeSubCluster the mapping of the reservation to it's home
+ * sub-cluster
+ * @return the stored Subcluster from StateStore
+ * @throws YarnException if the call to the state store is unsuccessful
+ */
+ public SubClusterId addReservationHomeSubCluster(
+ ReservationHomeSubCluster appHomeSubCluster) throws YarnException {
+AddReservationHomeSubClusterResponse response =
+stateStore.addReservationHomeSubCluster(
+AddReservationHomeSubClusterRequest.newInstance(appHomeSubCluster));
+return response.getHomeSubCluster();
+ }
+
+ /**
+ * Returns the home {@link SubClusterId} for the specified
+ * {@link ReservationId}.
+ *
+ * @param reservationId the identifier of the reservation
+ * @return the home sub cluster identifier
+ * @throws YarnException if the call to the state store is unsuccessful
+ */
+ public SubClusterId getReservationHomeSubCluster(ReservationId reservationId)
+ throws YarnException {
+GetReservationHomeSubClusterResponse response =
+stateStore.getReservationHomeSubCluster(
Review Comment:
I will fix it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/policies/router/BaseRouterPoliciesTest.java:
##
@@ -126,4 +134,55 @@ public void testNullReservationContext() throws Exception {
() -> policy.getReservationHomeSubcluster(null));
}
+ @Test
+ public void testUnknownReservation() throws Exception {
+ReservationSubmissionRequest resReq = getReservationSubmissionRequest();
+ReservationId reservationId =
ReservationId.newInstance(System.currentTimeMillis(), 1);
+when(resReq.getQueue()).thenReturn("queue1");
+when(resReq.getReservationId()).thenReturn(reservationId);
+
+// route an application that uses this app
+ApplicationSubmissionContext applicationSubmissionContext =
+ApplicationSubmissionContext.newInstance(
+ApplicationId.newInstance(System.currentTimeMillis(), 1), "app1",
+"queue1", Priority.newInstance(1), null, false, false, 1, null,
null, false);
+
+applicationSubmissionContext.setReservationID(resReq.getReservationId());
+FederationRouterPolicy policy = (FederationRouterPolicy) getPolicy();
+
+LambdaTestUtils.intercept(YarnException.class,
+"Reservation " + reservationId + " does not exist",
+() -> policy.getHomeSubcluster(applicationSubmissionContext, new
ArrayList<>()));
+ }
+
+ @Test
+ public void testFollowReservation() throws YarnException {
+ReservationSubmissionRequest resReq = getReservationSubmissionRequest();
+when(resReq.getQueue()).thenReturn("queue1");
+when(resReq.getReservationId())
+.thenReturn(ReservationId.newInstance(System.currentTimeMillis(), 1));
+
+// first we invoke a reservation placement
+SubClusterId chosen = ((FederationRouterPolicy) getPolicy())
+.getReservationHomeSubcluster(resReq);
+
+// add this to the store
+this.getFederationPolicyContext().getFederationStateStoreFacade()
+.addReservationHomeSubCluster(ReservationHomeSubCluster.newInstance(
+ resReq.getReservationId(), chosen));
+
+// route an application that uses this app
+ApplicationSubmissionContext applicationSubmissionContext =
+ApplicationSubmissionContext.newInstance(
+ApplicationId.newInstance(System.currentTimeMillis(), 1), "app1",
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576326#comment-17576326
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939616164
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,39 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new YarnException("Reservation " + reservationId + " does not
exist");
+}
+return GetReservationHomeSubClusterResponse.newInstance(
Review Comment:
Thanks for your help reviewing the code, I will fix it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,39 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new YarnException("Reservation " + reservationId + " does not
exist");
+}
+return GetReservationHomeSubClusterResponse.newInstance(
+ReservationHomeSubCluster.newInstance(reservationId,
reservations.get(reservationId)));
+ }
+
+ @Override
+ public GetReservationsHomeSubClusterResponse getReservationsHomeSubCluster(
+ GetReservationsHomeSubClusterRequest request) throws YarnException {
+List result = new ArrayList<>();
+for (Entry e : reservations.entrySet()) {
+ result.add(ReservationHomeSubCluster.newInstance(e.getKey(),
e.getValue()));
Review Comment:
I will fix it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.java:
##
@@ -637,4 +643,22 @@ private static long getCurrentTime() {
Calendar cal = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
return cal.getTimeInMillis();
}
+
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+return null;
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+return null;
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576263#comment-17576263
]
ASF GitHub Bot commented on YARN-11236:
---
goiri commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939549105
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/records/ReservationHomeSubCluster.java:
##
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.records;
+
+import org.apache.hadoop.classification.InterfaceAudience.Private;
+import org.apache.hadoop.classification.InterfaceAudience.Public;
+import org.apache.hadoop.classification.InterfaceStability.Unstable;
+import org.apache.hadoop.yarn.api.records.ReservationId;
+import org.apache.hadoop.yarn.util.Records;
+
+/**
+ *
+ * ReservationHomeSubCluster is a report of the runtime information of the
+ * reservation that is running in the federated cluster.
+ *
+ *
+ * It includes information such as:
+ *
+ * {@link ReservationId}
+ * {@link SubClusterId}
+ *
+ *
+ */
+@Private
+@Unstable
+public abstract class ReservationHomeSubCluster {
+
+ @Private
+ @Unstable
+ public static ReservationHomeSubCluster newInstance(ReservationId appId,
+ SubClusterId homeSubCluster) {
+ReservationHomeSubCluster appMapping =
+Records.newRecord(ReservationHomeSubCluster.class);
+appMapping.setReservationId(appId);
+appMapping.setHomeSubCluster(homeSubCluster);
+return appMapping;
+ }
+
+ /**
+ * Get the {@link ReservationId} representing the unique identifier of the
+ * Reservation.
+ *
+ * @return the reservation identifier
+ */
+ @Public
+ @Unstable
+ public abstract ReservationId getReservationId();
+
+ /**
+ * Set the {@link ReservationId} representing the unique identifier of the
+ * Reservation.
+ *
+ * @param reservationId the reservation identifier
+ */
+ @Private
+ @Unstable
+ public abstract void setReservationId(ReservationId reservationId);
+
+ /**
+ * Get the {@link SubClusterId} representing the unique identifier of the
home
+ * subcluster in which the reservation is mapped to.
+ *
+ * @return the home subcluster identifier
+ */
+ @Public
+ @Unstable
+ public abstract SubClusterId getHomeSubCluster();
+
+ /**
+ * Set the {@link SubClusterId} representing the unique identifier of the
home
+ * subcluster in which the ReservationMaster of the reservation is running.
+ *
+ * @param homeSubCluster the home subcluster identifier
+ */
+ @Private
+ @Unstable
+ public abstract void setHomeSubCluster(SubClusterId homeSubCluster);
+
+ @Override
+ public boolean equals(Object obj) {
+if (this == obj) {
Review Comment:
EqualsBuilder and HashBuilder?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,39 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId =
+request.getReservationHomeSubCluster().getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ reservations.put(reservationId,
+ request.getReservationHomeSubCluster().getHomeSubCluster());
+}
+return
AddReservationHomeSubClusterResponse.newInstance(reservations.get(reservationId));
+ }
+
+ @Override
+ public GetReservationHomeSubClusterResponse getReservationHomeSubCluster(
+ GetReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator.validate(request);
+ReservationId reservationId = request.getReservationId();
+if (!reservations.containsKey(reservationId)) {
+ throw new
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576237#comment-17576237 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1207221017 @goiri Please help to review the code again, Thank you very much! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576233#comment-17576233
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1207220538
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 6s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 36s | | trunk passed |
| +1 :green_heart: | compile | 4m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 21s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 44s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 56s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 41s | | the patch passed |
| +1 :green_heart: | compile | 3m 52s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 52s | | the patch passed |
| -1 :x: | javac | 3m 52s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/3/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 17s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 17s | | the patch passed |
| -1 :x: | javac | 3m 17s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/3/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 13s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 48s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 33s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 3s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart:
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576227#comment-17576227
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939531365
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/utils/FederationReservationHomeSubClusterStoreInputValidator.java:
##
@@ -0,0 +1,134 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.utils;
+
+import org.apache.hadoop.yarn.api.records.ReservationId;
+import
org.apache.hadoop.yarn.server.federation.store.exception.FederationStateStoreInvalidInputException;
+import
org.apache.hadoop.yarn.server.federation.store.records.AddReservationHomeSubClusterRequest;
+import
org.apache.hadoop.yarn.server.federation.store.records.ReservationHomeSubCluster;
+import
org.apache.hadoop.yarn.server.federation.store.records.GetReservationHomeSubClusterRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility class to validate the inputs to
+ * {@code FederationReservationHomeSubClusterStore}, allows a fail fast
+ * mechanism for invalid user inputs.
+ *
+ */
+public final class FederationReservationHomeSubClusterStoreInputValidator {
+
+ private static final Logger LOG = LoggerFactory
+ .getLogger(FederationReservationHomeSubClusterStoreInputValidator.class);
+
+ private FederationReservationHomeSubClusterStoreInputValidator() {
+ }
+
+ /**
+ * Quick validation on the input to check some obvious fail conditions (fail
+ * fast). Check if the provided {@link AddReservationHomeSubClusterRequest}
+ * for adding a new reservation is valid or not.
+ *
+ * @param request the {@link AddReservationHomeSubClusterRequest} to validate
+ * against
+ * @throws FederationStateStoreInvalidInputException if the request is
invalid
+ */
+ public static void validateAddReservationHomeSubClusterRequest(
+ AddReservationHomeSubClusterRequest request)
+ throws FederationStateStoreInvalidInputException {
+if (request == null) {
+ String message = "Missing AddReservationHomeSubCluster Request."
+ + " Please try again by specifying"
+ + " an AddReservationHomeSubCluster information.";
+ LOG.warn(message);
+ throw new FederationStateStoreInvalidInputException(message);
+}
+
+// validate ReservationHomeSubCluster info
+checkReservationHomeSubCluster(request.getReservationHomeSubCluster());
+ }
+
+ /**
+ * Quick validation on the input to check some obvious fail conditions (fail
+ * fast). Check if the provided {@link GetReservationHomeSubClusterRequest}
+ * for querying reservation's information is valid or not.
+ *
+ * @param request the {@link GetReservationHomeSubClusterRequest} to validate
+ * against
+ * @throws FederationStateStoreInvalidInputException if the request is
invalid
+ */
+ public static void validateGetReservationHomeSubClusterRequest(
+ GetReservationHomeSubClusterRequest request)
+ throws FederationStateStoreInvalidInputException {
+if (request == null) {
+ String message = "Missing GetReservationHomeSubCluster Request."
+ + " Please try again by specifying an Reservation Id information.";
+ LOG.warn(message);
+ throw new FederationStateStoreInvalidInputException(message);
+}
+
+// validate Reservation Id
+checkReservationId(request.getReservationId());
+ }
+
+ /**
+ * Validate if the ReservationHomeSubCluster info are present or not.
+ *
+ * @param reservationHomeSubCluster the information of the Reservation to be
+ * verified
+ * @throws FederationStateStoreInvalidInputException if the SubCluster Info
+ * are invalid
+ */
+ private static void checkReservationHomeSubCluster(
+ ReservationHomeSubCluster reservationHomeSubCluster)
+
+ throws FederationStateStoreInvalidInputException {
+if
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576226#comment-17576226
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939531330
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/utils/FederationReservationHomeSubClusterStoreInputValidator.java:
##
@@ -0,0 +1,134 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.utils;
+
+import org.apache.hadoop.yarn.api.records.ReservationId;
+import
org.apache.hadoop.yarn.server.federation.store.exception.FederationStateStoreInvalidInputException;
+import
org.apache.hadoop.yarn.server.federation.store.records.AddReservationHomeSubClusterRequest;
+import
org.apache.hadoop.yarn.server.federation.store.records.ReservationHomeSubCluster;
+import
org.apache.hadoop.yarn.server.federation.store.records.GetReservationHomeSubClusterRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility class to validate the inputs to
+ * {@code FederationReservationHomeSubClusterStore}, allows a fail fast
+ * mechanism for invalid user inputs.
+ *
+ */
+public final class FederationReservationHomeSubClusterStoreInputValidator {
+
+ private static final Logger LOG = LoggerFactory
+ .getLogger(FederationReservationHomeSubClusterStoreInputValidator.class);
+
+ private FederationReservationHomeSubClusterStoreInputValidator() {
+ }
+
+ /**
+ * Quick validation on the input to check some obvious fail conditions (fail
+ * fast). Check if the provided {@link AddReservationHomeSubClusterRequest}
+ * for adding a new reservation is valid or not.
+ *
+ * @param request the {@link AddReservationHomeSubClusterRequest} to validate
+ * against
+ * @throws FederationStateStoreInvalidInputException if the request is
invalid
+ */
+ public static void validateAddReservationHomeSubClusterRequest(
+ AddReservationHomeSubClusterRequest request)
+ throws FederationStateStoreInvalidInputException {
+if (request == null) {
+ String message = "Missing AddReservationHomeSubCluster Request."
+ + " Please try again by specifying"
+ + " an AddReservationHomeSubCluster information.";
+ LOG.warn(message);
+ throw new FederationStateStoreInvalidInputException(message);
+}
+
+// validate ReservationHomeSubCluster info
+checkReservationHomeSubCluster(request.getReservationHomeSubCluster());
+ }
+
+ /**
+ * Quick validation on the input to check some obvious fail conditions (fail
+ * fast). Check if the provided {@link GetReservationHomeSubClusterRequest}
+ * for querying reservation's information is valid or not.
+ *
+ * @param request the {@link GetReservationHomeSubClusterRequest} to validate
+ * against
+ * @throws FederationStateStoreInvalidInputException if the request is
invalid
+ */
+ public static void validateGetReservationHomeSubClusterRequest(
Review Comment:
I have modified the code, the method is called `validate()`
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,41 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator
+.validateAddReservationHomeSubClusterRequest(request);
Review Comment:
Thanks for your suggestion, I have modified the code, the method is called
`validate()`
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL:
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576224#comment-17576224
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939531303
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,41 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator
+.validateAddReservationHomeSubClusterRequest(request);
Review Comment:
Thanks for your suggestion, I have modified the code, the method is called
`validate`
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576222#comment-17576222 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1207218561 @goiri Please help to review the code again, Thank you very much! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576145#comment-17576145
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1207149152
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 49s | | trunk passed |
| +1 :green_heart: | compile | 4m 1s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 6s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 56s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 40s | | the patch passed |
| +1 :green_heart: | compile | 3m 49s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 3m 49s | | the patch passed |
| -1 :x: | javac | 3m 49s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/2/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 16s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 16s | | the patch passed |
| -1 :x: | javac | 3m 16s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/2/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 11s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 0 new +
11 unchanged - 1 fixed = 11 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 49s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 5s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart:
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576087#comment-17576087
]
ASF GitHub Bot commented on YARN-11236:
---
slfan1989 commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939449057
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/policies/router/AbstractRouterPolicy.java:
##
@@ -98,7 +99,16 @@ protected Map
prefilterSubClusters(
// if a reservation exists limit scope to the sub-cluster this
// reservation is mapped to
-// TODO: Implemented in YARN-11236
+if (reservationId != null) {
+ // note this might throw YarnException if the reservation is
+ // unknown. This is to be expected, and should be handled by
+ // policy invoker.
+ SubClusterId resSubCluster =
getPolicyContext().getFederationStateStoreFacade().
Review Comment:
Thank you very much for your help reviewing the code, I will fix it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/policies/router/AbstractRouterPolicy.java:
##
@@ -98,7 +99,16 @@ protected Map
prefilterSubClusters(
// if a reservation exists limit scope to the sub-cluster this
// reservation is mapped to
-// TODO: Implemented in YARN-11236
+if (reservationId != null) {
+ // note this might throw YarnException if the reservation is
+ // unknown. This is to be expected, and should be handled by
+ // policy invoker.
+ SubClusterId resSubCluster =
getPolicyContext().getFederationStateStoreFacade().
+ getReservationHomeSubCluster(reservationId);
+
+ return Collections.singletonMap(resSubCluster,
activeSubClusters.get(resSubCluster));
Review Comment:
I will fix it.
> Implement FederationReservationHomeSubClusterStore With MemoryStore
> ---
>
> Key: YARN-11236
> URL: https://issues.apache.org/jira/browse/YARN-11236
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576068#comment-17576068
]
ASF GitHub Bot commented on YARN-11236:
---
goiri commented on code in PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#discussion_r939396343
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/MemoryFederationStateStore.java:
##
@@ -312,4 +324,41 @@ public Version loadVersion() {
return null;
}
+ @Override
+ public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(
+ AddReservationHomeSubClusterRequest request) throws YarnException {
+FederationReservationHomeSubClusterStoreInputValidator
+.validateAddReservationHomeSubClusterRequest(request);
Review Comment:
These names are pretty verose.
If we are in FederationReservationHomeSubClusterStoreInputValidator isn't
enough to say validateRequest? or validate as the type is in the arg?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/utils/FederationReservationHomeSubClusterStoreInputValidator.java:
##
@@ -0,0 +1,134 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.utils;
+
+import org.apache.hadoop.yarn.api.records.ReservationId;
+import
org.apache.hadoop.yarn.server.federation.store.exception.FederationStateStoreInvalidInputException;
+import
org.apache.hadoop.yarn.server.federation.store.records.AddReservationHomeSubClusterRequest;
+import
org.apache.hadoop.yarn.server.federation.store.records.ReservationHomeSubCluster;
+import
org.apache.hadoop.yarn.server.federation.store.records.GetReservationHomeSubClusterRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility class to validate the inputs to
+ * {@code FederationReservationHomeSubClusterStore}, allows a fail fast
+ * mechanism for invalid user inputs.
+ *
+ */
+public final class FederationReservationHomeSubClusterStoreInputValidator {
+
+ private static final Logger LOG = LoggerFactory
+ .getLogger(FederationReservationHomeSubClusterStoreInputValidator.class);
+
+ private FederationReservationHomeSubClusterStoreInputValidator() {
+ }
+
+ /**
+ * Quick validation on the input to check some obvious fail conditions (fail
+ * fast). Check if the provided {@link AddReservationHomeSubClusterRequest}
+ * for adding a new reservation is valid or not.
+ *
+ * @param request the {@link AddReservationHomeSubClusterRequest} to validate
+ * against
+ * @throws FederationStateStoreInvalidInputException if the request is
invalid
+ */
+ public static void validateAddReservationHomeSubClusterRequest(
+ AddReservationHomeSubClusterRequest request)
+ throws FederationStateStoreInvalidInputException {
+if (request == null) {
+ String message = "Missing AddReservationHomeSubCluster Request."
+ + " Please try again by specifying"
+ + " an AddReservationHomeSubCluster information.";
+ LOG.warn(message);
+ throw new FederationStateStoreInvalidInputException(message);
+}
+
+// validate ReservationHomeSubCluster info
+checkReservationHomeSubCluster(request.getReservationHomeSubCluster());
+ }
+
+ /**
+ * Quick validation on the input to check some obvious fail conditions (fail
+ * fast). Check if the provided {@link GetReservationHomeSubClusterRequest}
+ * for querying reservation's information is valid or not.
+ *
+ * @param request the {@link GetReservationHomeSubClusterRequest} to validate
+ * against
+ * @throws FederationStateStoreInvalidInputException if the request is
invalid
+ */
+ public static void validateGetReservationHomeSubClusterRequest(
Review Comment:
validate() should be enough as the type is there.
##
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576060#comment-17576060 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 commented on PR #4711: URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1206930345 @goiri Could you please help to review the code? Thank you very much! > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[
https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17576040#comment-17576040
]
ASF GitHub Bot commented on YARN-11236:
---
hadoop-yetus commented on PR #4711:
URL: https://github.com/apache/hadoop/pull/4711#issuecomment-1206819328
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 14s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 1s | | trunk passed |
| +1 :green_heart: | compile | 4m 25s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 38s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 2s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 52s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 44s | | the patch passed |
| +1 :green_heart: | compile | 4m 13s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 4m 13s | | the patch passed |
| -1 :x: | javac | 4m 13s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 3 new +
446 unchanged - 0 fixed = 449 total (was 446) |
| +1 :green_heart: | compile | 3m 36s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 3m 36s | | the patch passed |
| -1 :x: | javac | 3m 36s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 370 unchanged - 0 fixed = 373 total (was 370) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 14s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4711/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1
new + 11 unchanged - 1 fixed = 12 total (was 12) |
| +1 :green_heart: | mvnsite | 1m 52s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 56s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m
[jira] [Commented] (YARN-11236) Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575926#comment-17575926 ] ASF GitHub Bot commented on YARN-11236: --- slfan1989 opened a new pull request, #4711: URL: https://github.com/apache/hadoop/pull/4711 JIRA:YARN-11236. Implement FederationReservationHomeSubClusterStore With MemoryStore. > Implement FederationReservationHomeSubClusterStore With MemoryStore > --- > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
