[jira] [Commented] (YARN-10846) Add dispatcher metrics to NM
[ https://issues.apache.org/jira/browse/YARN-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614498#comment-17614498 ] ASF GitHub Bot commented on YARN-10846: --- slfan1989 commented on PR #4687: URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1272338733 @9uapaw Thank you very much for your suggestion, I will modify the code. > Add dispatcher metrics to NM > > > Key: YARN-10846 > URL: https://issues.apache.org/jira/browse/YARN-10846 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: chaosju >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Attachments: YARN-10846.001.patch, screenshot-1.png > > Time Spent: 1h 20m > Remaining Estimate: 0h > > base [YARN-9615|https://issues.apache.org/jira/browse/YARN-9615], add > dispatcher metrics to NM -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11332) [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration.
[ https://issues.apache.org/jira/browse/YARN-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614497#comment-17614497 ] ASF GitHub Bot commented on YARN-11332: --- slfan1989 commented on PR #4982: URL: https://github.com/apache/hadoop/pull/4982#issuecomment-1272337601 @goiri Can you help review the code again? Thank you very much! > [Federation] Improve FederationClientInterceptor#ThreadPool thread pool > configuration. > -- > > Key: YARN-11332 > URL: https://issues.apache.org/jira/browse/YARN-11332 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, router >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > [Federation] Improve FederationClientInterceptor#ThreadPool thread pool > configuration. > 1. Increase the thread pool core thread configuration and maximum thread > configuration > 2. Abandon the original thread pool configuration parameters, the original > parameters control the maximum number of threads and core threads at the same > time > 3. Increase the thread KeepAliveTime configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10680) Revisit try blocks without catch blocks but having finally blocks
[
https://issues.apache.org/jira/browse/YARN-10680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614492#comment-17614492
]
ASF GitHub Bot commented on YARN-10680:
---
hadoop-yetus commented on PR #4941:
URL: https://github.com/apache/hadoop/pull/4941#issuecomment-1272329041
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 32m 19s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4941/5/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 36s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 57s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | compile | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 0m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 56s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 12m 11s | | hadoop-sls in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 102m 24s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4941/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4941 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e312f6c01c95 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f551d16007f2d8bb55a776880afae0353e4c6446 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4941/5/testReport/ |
| Max. proce
[jira] [Updated] (YARN-9708) Yarn Router Support DelegationToken
[ https://issues.apache.org/jira/browse/YARN-9708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-9708: Priority: Major (was: Minor) > Yarn Router Support DelegationToken > --- > > Key: YARN-9708 > URL: https://issues.apache.org/jira/browse/YARN-9708 > Project: Hadoop YARN > Issue Type: New Feature > Components: router >Affects Versions: 3.4.0 >Reporter: Xie YiFan >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Attachments: Add_getDelegationToken_and_SecureLogin_in_router.patch, > RMDelegationTokenSecretManager_storeNewMasterKey.svg, > RouterDelegationTokenSecretManager_storeNewMasterKey.svg > > > 1.we use router as proxy to manage multiple cluster which be independent of > each other in order to apply unified client. Thus, we implement our > customized AMRMProxyPolicy that doesn't broadcast ResourceRequest to other > cluster. > 2.Our production environment need kerberos. But router doesn't support > SecureLogin for now. > https://issues.apache.org/jira/browse/YARN-6539 desn't work. So we > improvement it. > 3.Some framework like oozie would get Token via yarnclient#getDelegationToken > which router doesn't support. Our solution is that adding homeCluster to > ApplicationSubmissionContextProto & GetDelegationTokenRequestProto. Job would > be submitted with specified clusterid so that router knows which cluster to > submit this job. Router would get Token from one RM according to specified > clusterid when client call getDelegation meanwhile apply some mechanism to > save this token in memory. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11315) [Federation] YARN Federation Router Supports Cross-Origin.
[ https://issues.apache.org/jira/browse/YARN-11315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11315: - Parent: YARN-9708 Issue Type: Sub-task (was: Improvement) > [Federation] YARN Federation Router Supports Cross-Origin. > -- > > Key: YARN-11315 > URL: https://issues.apache.org/jira/browse/YARN-11315 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > 1.The Rest interface of Router should support Cross-Origin like RM and NM. > 2.The Yarn-UI2 module requires the Router to support Cross-Origin. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10846) Add dispatcher metrics to NM
[ https://issues.apache.org/jira/browse/YARN-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614483#comment-17614483 ] ASF GitHub Bot commented on YARN-10846: --- 9uapaw commented on PR #4687: URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1272312993 Hi @slfan1989, thanks for the patch! I think this should be hidden behind a config flag, and should be opt-in (false by default). Metrics aggregation often causes performance degradation. > Add dispatcher metrics to NM > > > Key: YARN-10846 > URL: https://issues.apache.org/jira/browse/YARN-10846 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: chaosju >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Attachments: YARN-10846.001.patch, screenshot-1.png > > Time Spent: 1h 20m > Remaining Estimate: 0h > > base [YARN-9615|https://issues.apache.org/jira/browse/YARN-9615], add > dispatcher metrics to NM -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11317) [Federation] Refactoring Yarn Router's About Web Page.
[ https://issues.apache.org/jira/browse/YARN-11317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11317: - Parent: YARN-11333 Issue Type: Sub-task (was: Improvement) > [Federation] Refactoring Yarn Router's About Web Page. > -- > > Key: YARN-11317 > URL: https://issues.apache.org/jira/browse/YARN-11317 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Attachments: image-2022-09-25-12-33-58-099.png, > image-2022-09-28-23-12-37-821.png, image-2022-09-28-23-42-15-475.png, > image-2022-09-29-12-35-59-534.png, image-2022-09-29-12-46-59-592.png > > > Optimize Yarn Federation Router About page > 1. Make the page more similar to the Yarn RM page style > 2. Add more friendly tips. > > Before page optimization: > !image-2022-09-25-12-33-58-099.png|width=982,height=417! > After page optimization: > Enable Yarn Federation ： > !image-2022-09-29-12-46-59-592.png|width=947,height=431! > Not Enable Yarn Federation： > !image-2022-09-29-12-35-59-534.png|width=977,height=414! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11310) [Yarn Federation] Refactoring Router's Federation Web Page
[ https://issues.apache.org/jira/browse/YARN-11310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11310: - Parent: YARN-11333 Issue Type: Sub-task (was: Improvement) > [Yarn Federation] Refactoring Router's Federation Web Page > -- > > Key: YARN-11310 > URL: https://issues.apache.org/jira/browse/YARN-11310 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: Federation_Web_Page.gif, > image-2022-09-22-11-43-27-954.png, image-2022-09-22-11-49-10-974.png, > image-2022-09-22-11-52-50-188.png, image-2022-09-22-11-55-08-113.png > > > The Yarn Federation page before modification is as follows: > enable federation: > !image-2022-09-22-11-43-27-954.png|width=952,height=232! > federation is not enabled > !image-2022-09-22-11-49-10-974.png|width=958,height=209! > > The page needs to be optimized as follows: > 1. Title is inaccurate, should be "About The Federation" > 2.If the user does not configure federation.enable, the page should prompt > information instead of showing blank. > 3.For cluster information, we care more about the current sub-cluster status, > registration time, heartbeat time, etc., rather than capacity information. > > enable federation: > !image-2022-09-22-11-52-50-188.png|width=736,height=320! > federation is not enabled > !image-2022-09-22-11-55-08-113.png|width=733,height=211! > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11333) Federation: Improve Yarn Router Web Page
fanshilun created YARN-11333: Summary: Federation: Improve Yarn Router Web Page Key: YARN-11333 URL: https://issues.apache.org/jira/browse/YARN-11333 Project: Hadoop YARN Issue Type: Improvement Components: federation, router Affects Versions: 3.4.0 Reporter: fanshilun Assignee: fanshilun Improve the Yarn Router page, involving pages such as About, Federation, Node, Application, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11327) [Federation] Refactoring Yarn Router's Node Web Page.
[ https://issues.apache.org/jira/browse/YARN-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11327: - Parent: YARN-11333 Issue Type: Sub-task (was: Improvement) > [Federation] Refactoring Yarn Router's Node Web Page. > - > > Key: YARN-11327 > URL: https://issues.apache.org/jira/browse/YARN-11327 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, router >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Attachments: image-2022-10-05-17-31-01-780.png, > image-2022-10-05-17-35-37-762.png, image-2022-10-05-17-37-20-718.png, > image-2022-10-05-17-45-16-500.png > > > [Federation] Refactoring Yarn Router's Node Web Page. > The modified content is as follows: > 1.Router's Node Web Page is more similar to RM's Node Web Page, showing > Cluster Metrics, Cluster Nodes Metrics, Scheduler Metrics > 2.The information of the cluster is displayed under the list > 3. Display the relevant node information of the sub-cluster respectively, and > display the information in the header > *The style before the page change is as follows:* > *!image-2022-10-05-17-31-01-780.png|width=765,height=280!* > *The style of the page after the change is as follows:* > *Enable Federation:* > *!image-2022-10-05-17-35-37-762.png|width=765,height=394!* > *!image-2022-10-05-17-37-20-718.png|width=765,height=320!* > *Not Enable Federation:* > *!image-2022-10-05-17-45-16-500.png|width=764,height=344!* -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7615) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations
[ https://issues.apache.org/jira/browse/YARN-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-7615: Summary: [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations (was: [RESERVATION] Federation StateStore: support storage/retrieval of reservations) > [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations > -- > > Key: YARN-7615 > URL: https://issues.apache.org/jira/browse/YARN-7615 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: Carlo Curino >Assignee: fanshilun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11252) [RESERVATION] Yarn Federation Router Supports Update / Delete Reservation in MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11252: - Summary: [RESERVATION] Yarn Federation Router Supports Update / Delete Reservation in MemoryStore (was: [RESERVATION] Yarn Federation Router Supports Update /Delete Reservation in MemoryStore) > [RESERVATION] Yarn Federation Router Supports Update / Delete Reservation in > MemoryStore > > > Key: YARN-11252 > URL: https://issues.apache.org/jira/browse/YARN-11252 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10680) Revisit try blocks without catch blocks but having finally blocks
[
https://issues.apache.org/jira/browse/YARN-10680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614477#comment-17614477
]
ASF GitHub Bot commented on YARN-10680:
---
susheel-gupta commented on code in PR #4941:
URL: https://github.com/apache/hadoop/pull/4941#discussion_r990636469
##
hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java:
##
@@ -105,7 +108,11 @@ public boolean tryCommit(Resource cluster,
ResourceCommitRequest r,
@Override
public void handle(SchedulerEvent schedulerEvent) {
-schedulerCommons.handle(schedulerEvent);
+try{
+ schedulerCommons.handle(schedulerEvent);
Review Comment:
There are more than 100 instances where this handle method is being called,
I have checked fewof them and updated the code.
> Revisit try blocks without catch blocks but having finally blocks
> -
>
> Key: YARN-10680
> URL: https://issues.apache.org/jira/browse/YARN-10680
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Szilard Nemeth
>Assignee: Susheel Gupta
>Priority: Minor
> Labels: newbie, pull-request-available, trivial
> Attachments: YARN-10860.001.patch
>
>
> This jira is to revisit all try blocks without catch blocks but having
> finally blocks in SLS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11252) [RESERVATION] Yarn Federation Router Supports Update /Delete Reservation in MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11252: - Summary: [RESERVATION] Yarn Federation Router Supports Update /Delete Reservation in MemoryStore (was: Yarn Federation Router Supports Update /Delete Reservation in MemoryStore) > [RESERVATION] Yarn Federation Router Supports Update /Delete Reservation in > MemoryStore > --- > > Key: YARN-11252 > URL: https://issues.apache.org/jira/browse/YARN-11252 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11235) [RESERVATION] Refactor Policy Code and Define getReservationHomeSubcluster
[ https://issues.apache.org/jira/browse/YARN-11235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11235: - Summary: [RESERVATION] Refactor Policy Code and Define getReservationHomeSubcluster (was: Refactor Policy Code and Define getReservationHomeSubcluster) > [RESERVATION] Refactor Policy Code and Define getReservationHomeSubcluster > -- > > Key: YARN-11235 > URL: https://issues.apache.org/jira/browse/YARN-11235 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: [RESERVATION] Add support for reservation-based > routing.pdf > > Time Spent: 8h 20m > Remaining Estimate: 0h > > Refer to 2.1 Router Policy, which describes the changes to be made. The > documentation will continue to improve, the current version is V1. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11236) [RESERVATION] Implement FederationReservationHomeSubClusterStore With MemoryStore
[ https://issues.apache.org/jira/browse/YARN-11236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11236: - Summary: [RESERVATION] Implement FederationReservationHomeSubClusterStore With MemoryStore (was: Implement FederationReservationHomeSubClusterStore With MemoryStore) > [RESERVATION] Implement FederationReservationHomeSubClusterStore With > MemoryStore > - > > Key: YARN-11236 > URL: https://issues.apache.org/jira/browse/YARN-11236 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7615) [RESERVATION] Federation StateStore: support storage/retrieval of reservations
[ https://issues.apache.org/jira/browse/YARN-7615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-7615: Parent: YARN-5871 Issue Type: Sub-task (was: New Feature) > [RESERVATION] Federation StateStore: support storage/retrieval of reservations > -- > > Key: YARN-7615 > URL: https://issues.apache.org/jira/browse/YARN-7615 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: Carlo Curino >Assignee: fanshilun >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11273) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With SQL
[ https://issues.apache.org/jira/browse/YARN-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11273: - Parent Issue: YARN-5871 (was: YARN-7615) > [RESERVATION] Federation StateStore: Support storage/retrieval of > Reservations With SQL > --- > > Key: YARN-11273 > URL: https://issues.apache.org/jira/browse/YARN-11273 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11272) [RESERVATION] Federation StateStore: Support storage/retrieval of Reservations With Zk
[ https://issues.apache.org/jira/browse/YARN-11272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11272: - Parent Issue: YARN-5871 (was: YARN-7615) > [RESERVATION] Federation StateStore: Support storage/retrieval of > Reservations With Zk > -- > > Key: YARN-11272 > URL: https://issues.apache.org/jira/browse/YARN-11272 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, reservation system >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11216) Avoid unnecessary reconstruction of ConfigurationProperties
[
https://issues.apache.org/jira/browse/YARN-11216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614475#comment-17614475
]
ASF GitHub Bot commented on YARN-11216:
---
hadoop-yetus commented on PR #4655:
URL: https://github.com/apache/hadoop/pull/4655#issuecomment-1272305412
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 6s | | trunk passed |
| +1 :green_heart: | compile | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 11s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 57s | | the patch passed |
| +1 :green_heart: | compile | 1m 3s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 3s | | the patch passed |
| +1 :green_heart: | compile | 0m 54s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 54s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 45s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 59s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 40s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 106m 17s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 219m 21s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4655/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4655 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5817927f14d3 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 573dd3f965623b027a4c9ace981ea85d2357d225 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4655/7/testReport/ |
| Max. process+thread count | 955 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4655/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Y
[jira] [Updated] (YARN-11297) Improve Yarn Router Reservation Submission Code
[ https://issues.apache.org/jira/browse/YARN-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] fanshilun updated YARN-11297: - Parent: YARN-10144 Issue Type: Sub-task (was: Improvement) > Improve Yarn Router Reservation Submission Code > --- > > Key: YARN-11297 > URL: https://issues.apache.org/jira/browse/YARN-11297 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > The same reservation may be submitted repeatedly. At this time, we should use > the reserved results first. If the reserved results are not available, > consider applying from other RMs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8262) get_executable in container-executor should provide meaningful error codes
[ https://issues.apache.org/jira/browse/YARN-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614420#comment-17614420 ] ASF GitHub Bot commented on YARN-8262: -- K0K0V0K commented on PR #4949: URL: https://github.com/apache/hadoop/pull/4949#issuecomment-1272264656 I think there should be at least one test in the TestLinuxContainerExecutor, what will check in case of an error the app will return with a not -1 error code. > get_executable in container-executor should provide meaningful error codes > -- > > Key: YARN-8262 > URL: https://issues.apache.org/jira/browse/YARN-8262 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Miklos Szegedi >Assignee: Susheel Gupta >Priority: Minor > Labels: newbie, pull-request-available, trivial > > Currently it calls exit(-1) that makes it difficult to debug without stderr. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
