date:20221020

[jira] [Commented] (YARN-11359) [Federation] Routing admin invocations transparently to multiple RMs.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621502#comment-17621502
 ] 

ASF GitHub Bot commented on YARN-11359:
---

slfan1989 opened a new pull request, #5057:
URL: https://github.com/apache/hadoop/pull/5057

   JIRA: YARN-11359. [Federation] Routing admin invocations transparently to 
multiple RMs.




> [Federation] Routing admin invocations transparently to multiple RMs.
> -
>
> Key: YARN-11359
> URL: https://issues.apache.org/jira/browse/YARN-11359
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>
> [Federation] Routing admin invocations transparently to multiple RMs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11359) [Federation] Routing admin invocations transparently to multiple RMs.

2022-10-20 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated YARN-11359:
--
Labels: pull-request-available  (was: )

> [Federation] Routing admin invocations transparently to multiple RMs.
> -
>
> Key: YARN-11359
> URL: https://issues.apache.org/jira/browse/YARN-11359
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>
> [Federation] Routing admin invocations transparently to multiple RMs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11358) [Federation] Add Strict Mode Configuration Parameters.

2022-10-20 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated YARN-11358:
--
Labels: pull-request-available  (was: )

> [Federation] Add Strict Mode Configuration Parameters.
> --
>
> Key: YARN-11358
> URL: https://issues.apache.org/jira/browse/YARN-11358
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>
> When we use Federation, we will have multiple sub-clusters, and some 
> interfaces need to call all clusters to get results, 
> such as getNodes, getNodeToLabels, etc.
> But if the sub-cluster is unavailable when we call the interface, should the 
> interface return the result?
> The current code is to return the result.If the result is returned, the 
> result information is inaccurate; 
> if the result is not returned, it is not user-friendly
> We'd better add a parameter control, the user chooses whether to report an 
> error or return the result.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11358) [Federation] Add Strict Mode Configuration Parameters.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621500#comment-17621500
 ] 

ASF GitHub Bot commented on YARN-11358:
---

slfan1989 opened a new pull request, #5056:
URL: https://github.com/apache/hadoop/pull/5056

   JIRA: YARN-11358. [Federation] Add Strict Mode Configuration Parameters.




> [Federation] Add Strict Mode Configuration Parameters.
> --
>
> Key: YARN-11358
> URL: https://issues.apache.org/jira/browse/YARN-11358
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>
> When we use Federation, we will have multiple sub-clusters, and some 
> interfaces need to call all clusters to get results, 
> such as getNodes, getNodeToLabels, etc.
> But if the sub-cluster is unavailable when we call the interface, should the 
> interface return the result?
> The current code is to return the result.If the result is returned, the 
> result information is inaccurate; 
> if the result is not returned, it is not user-friendly
> We'd better add a parameter control, the user chooses whether to report an 
> error or return the result.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11332) [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621480#comment-17621480
 ] 

ASF GitHub Bot commented on YARN-11332:
---

hadoop-yetus commented on PR #4982:
URL: https://github.com/apache/hadoop/pull/4982#issuecomment-1286453733

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 47s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  xmllint  |   0m  0s |  |  xmllint was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  18m 34s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 18s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  10m  6s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |  10m 16s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   2m 10s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 25s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   3m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   3m  8s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   5m 56s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  22m 59s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 30s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 57s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  10m 15s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |  10m 15s |  |  
hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 
with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 0 new + 731 
unchanged - 2 fixed = 731 total (was 733)  |
   | +1 :green_heart: |  compile  |   9m 16s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   9m 16s |  |  
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
 with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 generated 0 new 
+ 641 unchanged - 2 fixed = 641 total (was 643)  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 57s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   3m 19s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 59s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   2m 43s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   6m 17s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  22m 56s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 33s |  |  hadoop-yarn-api in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   5m 22s |  |  hadoop-yarn-common in the patch 
passed.  |
   | -1 :x: |  unit  |   5m 50s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4982/11/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
 |  hadoop-yarn-server-router in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 16s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 189m 52s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | 
hadoop.yarn.server.router.clientrm.TestFederationClientInterceptorRetry |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4982/11/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4982 |
   | Optional Tests | dupname asflicense compile

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621476#comment-17621476
 ] 

ASF GitHub Bot commented on YARN-11345:
---

hadoop-yetus commented on PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#issuecomment-1286438510

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 41s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  1s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  18m 31s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  27m 51s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  10m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   9m 17s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   2m  4s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   2m 10s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 22s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   2m 11s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 38s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  22m 14s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 15s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   9m 51s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |   9m 51s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   9m  4s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   9m  4s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   1m 47s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5030/5/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
 |  hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 3 unchanged - 
0 fixed = 4 total (was 3)  |
   | +1 :green_heart: |  mvnsite  |   1m 59s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 59s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 53s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 35s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  24m 31s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |   5m 21s |  |  hadoop-yarn-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   5m 49s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 14s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 174m 24s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5030/5/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5030 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 89d75cd3feb7 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / d2b820682398e8f8f449b089a7227c99c26e1c1e |
   | Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   |  Test Results |

[jira] [Commented] (YARN-11357) Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621473#comment-17621473
 ] 

ASF GitHub Bot commented on YARN-11357:
---

hadoop-yetus commented on PR #5055:
URL: https://github.com/apache/hadoop/pull/5055#issuecomment-1286427217

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 55s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  43m  1s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 38s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 35s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   0m 35s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 43s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 43s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   1m 10s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  23m 49s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 26s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 27s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 27s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 24s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   0m 24s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 17s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 26s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 23s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 22s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   0m 57s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 18s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |   5m 15s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 40s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 107m  6s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5055/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5055 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 66db3e38c4de 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / a403687d5fbe0cc4fcc67f24bb0a3a714fa77bee |
   | Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5055/1/testReport/ |
   | Max. process+thread count | 732 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5055/1/console |

[jira] [Created] (YARN-11359) [Federation] Routing admin invocations transparently to multiple RMs.

2022-10-20 Thread fanshilun (Jira)

fanshilun created YARN-11359:


 Summary: [Federation] Routing admin invocations transparently to 
multiple RMs.
 Key: YARN-11359
 URL: https://issues.apache.org/jira/browse/YARN-11359
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: federation, router
Affects Versions: 3.4.0
Reporter: fanshilun
Assignee: fanshilun


[Federation] Routing admin invocations transparently to multiple RMs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Created] (YARN-11358) [Federation] Add Strict Mode Configuration Parameters.

2022-10-20 Thread fanshilun (Jira)

fanshilun created YARN-11358:


 Summary: [Federation] Add Strict Mode Configuration Parameters.
 Key: YARN-11358
 URL: https://issues.apache.org/jira/browse/YARN-11358
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: federation, router
Affects Versions: 3.4.0
Reporter: fanshilun
Assignee: fanshilun


When we use Federation, we will have multiple sub-clusters, and some interfaces 
need to call all clusters to get results, 
such as getNodes, getNodeToLabels, etc.

But if the sub-cluster is unavailable when we call the interface, should the 
interface return the result?

The current code is to return the result.If the result is returned, the result 
information is inaccurate; 
if the result is not returned, it is not user-friendly

We'd better add a parameter control, the user chooses whether to report an 
error or return the result.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11357) Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621442#comment-17621442
 ] 

ASF GitHub Bot commented on YARN-11357:
---

slfan1989 opened a new pull request, #5055:
URL: https://github.com/apache/hadoop/pull/5055

   JIRA: YARN-11357. Fix FederationClientInterceptor#submitApplication Can't 
Update SubClusterId.
   
   In [YARN-11342](https://issues.apache.org/jira/browse/YARN-11342), I 
refactored the submitApplication method, but in the process of implementation, 
the judgment condition of if was written incorrectly.
   
   ```
   ...
   // Step2. Query homeSubCluster according to ApplicationId.
   Boolean exists = existsApplicationHomeSubCluster(applicationId);
   
   ApplicationHomeSubCluster appHomeSubCluster =
   ApplicationHomeSubCluster.newInstance(applicationId, subClusterId);
   
   // should be !exists
   if (exists || retryCount == 0){ 
   addApplicationHomeSubCluster(applicationId, appHomeSubCluster); 
   }
   else{ 
  updateApplicationHomeSubCluster(subClusterId, applicationId, 
appHomeSubCluster); 
   }
   ```




> Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId
> ---
>
> Key: YARN-11357
> URL: https://issues.apache.org/jira/browse/YARN-11357
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>
> In YARN-11342, I refactored the submitApplication method, but in the process 
> of implementation, the judgment condition of if was written incorrectly.
>  
> ```
> // Step2. Query homeSubCluster according to ApplicationId.
> Boolean exists = existsApplicationHomeSubCluster(applicationId);
> ApplicationHomeSubCluster appHomeSubCluster =
> ApplicationHomeSubCluster.newInstance(applicationId, subClusterId);
> // should be !exists
> if (exists || retryCount == 0) {
> addApplicationHomeSubCluster(applicationId, appHomeSubCluster);
> } else {
> updateApplicationHomeSubCluster(subClusterId, applicationId, 
> appHomeSubCluster);
> }
> ```



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11357) Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId

2022-10-20 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated YARN-11357:
--
Labels: pull-request-available  (was: )

> Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId
> ---
>
> Key: YARN-11357
> URL: https://issues.apache.org/jira/browse/YARN-11357
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>
> In YARN-11342, I refactored the submitApplication method, but in the process 
> of implementation, the judgment condition of if was written incorrectly.
>  
> ```
> // Step2. Query homeSubCluster according to ApplicationId.
> Boolean exists = existsApplicationHomeSubCluster(applicationId);
> ApplicationHomeSubCluster appHomeSubCluster =
> ApplicationHomeSubCluster.newInstance(applicationId, subClusterId);
> // should be !exists
> if (exists || retryCount == 0) {
> addApplicationHomeSubCluster(applicationId, appHomeSubCluster);
> } else {
> updateApplicationHomeSubCluster(subClusterId, applicationId, 
> appHomeSubCluster);
> }
> ```



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Created] (YARN-11357) Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId

2022-10-20 Thread fanshilun (Jira)

fanshilun created YARN-11357:


 Summary: Fix FederationClientInterceptor#submitApplication Can't 
Update SubClusterId
 Key: YARN-11357
 URL: https://issues.apache.org/jira/browse/YARN-11357
 Project: Hadoop YARN
  Issue Type: Bug
  Components: federation
Affects Versions: 3.4.0
Reporter: fanshilun
Assignee: fanshilun


In YARN-11342, I refactored the submitApplication method, but in the process of 
implementation, the judgment condition of if was written incorrectly.

 

```

// Step2. Query homeSubCluster according to ApplicationId.
Boolean exists = existsApplicationHomeSubCluster(applicationId);

ApplicationHomeSubCluster appHomeSubCluster =
ApplicationHomeSubCluster.newInstance(applicationId, subClusterId);

// should be !exists
if (exists || retryCount == 0) {
addApplicationHomeSubCluster(applicationId, appHomeSubCluster);
} else {
updateApplicationHomeSubCluster(subClusterId, applicationId, appHomeSubCluster);
}

```



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621434#comment-17621434
 ] 

ASF GitHub Bot commented on YARN-11345:
---

slfan1989 commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1001269099


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +155,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null) {
+  Collection apps = appsInfo.getApps();
+  if (CollectionUtils.isNotEmpty(apps)) {
+int numApps = apps.size();
+int i = 0;
+for (AppInfo app : apps) {
+  try {
+String percent = String.format("%.1f", app.getProgress() * 100.0F);
+String trackingURL =
+app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
+
+// AppID numerical value parsed by parseHadoopID in 
yarn.dt.plugins.js
+appsTableData.append("[\"")
+.append("")
+.append(app.getAppId()).append("\",\"")
+.append(escape(app.getUser())).append("\",\"")
+.append(escape(app.getName())).append("\",\"")
+.append(escape(app.getApplicationType())).append("\",\"")
+.append(escape(app.getQueue())).append("\",\"")
+.append(app.getPriority()).append("\",\"")
+.append(app.getStartTime()).append("\",\"")
+.append(app.getFinishTime()).append("\",\"")
+.append(app.getState()).append("\",\"")
+.append(app.getFinalStatus()).append("\",\"")
+// Progress bar
+.append("  ").append("  
")
+// History link
+.append("\",\"")
+.append("History").append("");
+appsTableData.append("\"]\n");
+
+if (i < numApps - 1) {
+  appsTableData.append(",");
+}
+  } catch (Exception e) {
+LOG.info("Cannot add application {}: {}", app.getAppId(), 
e.getMessage());
+  }
+  i++;

Review Comment:
   I read this part of the code carefully, the original code is reasonable, we 
should remove the extra comma outside the loop because we can't tell where the 
last comma is
   
   For example:
   We have 4 apps, A, B, C, D, and we expect the result to be [A, B, C, D], but 
when traversing the app list, the last app D fails, and we end up with [A 
,B,C,], the last app D fails, we can't handle this situation inside the loop.
   
   But the readability of the original logic is not good, I refactored this 
part of the code.





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail:

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621433#comment-17621433
 ] 

ASF GitHub Bot commented on YARN-11345:
---

slfan1989 commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1001269099


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +155,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null) {
+  Collection apps = appsInfo.getApps();
+  if (CollectionUtils.isNotEmpty(apps)) {
+int numApps = apps.size();
+int i = 0;
+for (AppInfo app : apps) {
+  try {
+String percent = String.format("%.1f", app.getProgress() * 100.0F);
+String trackingURL =
+app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
+
+// AppID numerical value parsed by parseHadoopID in 
yarn.dt.plugins.js
+appsTableData.append("[\"")
+.append("")
+.append(app.getAppId()).append("\",\"")
+.append(escape(app.getUser())).append("\",\"")
+.append(escape(app.getName())).append("\",\"")
+.append(escape(app.getApplicationType())).append("\",\"")
+.append(escape(app.getQueue())).append("\",\"")
+.append(app.getPriority()).append("\",\"")
+.append(app.getStartTime()).append("\",\"")
+.append(app.getFinishTime()).append("\",\"")
+.append(app.getState()).append("\",\"")
+.append(app.getFinalStatus()).append("\",\"")
+// Progress bar
+.append("  ").append("  
")
+// History link
+.append("\",\"")
+.append("History").append("");
+appsTableData.append("\"]\n");
+
+if (i < numApps - 1) {
+  appsTableData.append(",");
+}
+  } catch (Exception e) {
+LOG.info("Cannot add application {}: {}", app.getAppId(), 
e.getMessage());
+  }
+  i++;

Review Comment:
   I read this part of the code carefully, the original code is reasonable, we 
should remove the extra comma outside the loop because we can't tell where the 
last comma is
   
   For example:
   We have 4 apps, A, B, C, D, and we expect the result to be [A, B, C, D], but 
when traversing the app list, the last app D fails, and we end up with [A 
,B,C,], the last app D fails, we can't handle this situation inside the loop
   
   But the readability of the original logic is not good, I refactored this 
part of the code.





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail:

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621408#comment-17621408
 ] 

ASF GitHub Bot commented on YARN-11345:
---

slfan1989 commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1001214547


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +155,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null) {
+  Collection apps = appsInfo.getApps();
+  if (CollectionUtils.isNotEmpty(apps)) {
+int numApps = apps.size();
+int i = 0;
+for (AppInfo app : apps) {
+  try {
+String percent = String.format("%.1f", app.getProgress() * 100.0F);
+String trackingURL =
+app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
+
+// AppID numerical value parsed by parseHadoopID in 
yarn.dt.plugins.js
+appsTableData.append("[\"")
+.append("")
+.append(app.getAppId()).append("\",\"")
+.append(escape(app.getUser())).append("\",\"")
+.append(escape(app.getName())).append("\",\"")
+.append(escape(app.getApplicationType())).append("\",\"")
+.append(escape(app.getQueue())).append("\",\"")
+.append(app.getPriority()).append("\",\"")
+.append(app.getStartTime()).append("\",\"")
+.append(app.getFinishTime()).append("\",\"")
+.append(app.getState()).append("\",\"")
+.append(app.getFinalStatus()).append("\",\"")
+// Progress bar
+.append("  ").append("  
")
+// History link
+.append("\",\"")
+.append("History").append("");
+appsTableData.append("\"]\n");
+
+if (i < numApps - 1) {
+  appsTableData.append(",");
+}
+  } catch (Exception e) {
+LOG.info("Cannot add application {}: {}", app.getAppId(), 
e.getMessage());
+  }
+  i++;

Review Comment:
   Thanks for your suggestion, I will refactor this part of the code to make it 
more readable.





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11330) Use secure XML parser utils in YARN

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621397#comment-17621397
 ] 

ASF GitHub Bot commented on YARN-11330:
---

hadoop-yetus commented on PR #5017:
URL: https://github.com/apache/hadoop/pull/5017#issuecomment-1286243493

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 49s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 15 new or modified test files.  |
    _ branch-3.3 Compile Tests _ |
   | +0 :ok: |  mvndep  |  16m  1s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  24m 10s |  |  branch-3.3 passed  |
   | +1 :green_heart: |  compile  |  18m 24s |  |  branch-3.3 passed  |
   | +1 :green_heart: |  checkstyle  |   3m  3s |  |  branch-3.3 passed  |
   | +1 :green_heart: |  mvnsite  |   6m  0s |  |  branch-3.3 passed  |
   | +1 :green_heart: |  javadoc  |   4m 50s |  |  branch-3.3 passed  |
   | +1 :green_heart: |  spotbugs  |   8m 48s |  |  branch-3.3 passed  |
   | +1 :green_heart: |  shadedclient  |  25m 40s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 35s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m 30s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  17m 35s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |  17m 35s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   2m 59s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   5m 48s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   4m 20s |  |  the patch passed  |
   | +1 :green_heart: |  spotbugs  |   9m  3s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  25m 37s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  18m 28s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5017/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  unit  |  23m 22s |  |  hadoop-yarn-server-nodemanager 
in the patch passed.  |
   | +1 :green_heart: |  unit  |  90m 38s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |  27m 44s |  |  hadoop-yarn-client in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   1m 29s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 343m  2s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.security.ssl.TestReloadingX509TrustManager |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5017/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5017 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 48ddbcab3355 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | branch-3.3 / 7ba468df1f8725b3f2230171773b515540c13b92 |
   | Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~18.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5017/3/testReport/ |
   | Max. process+thread count | 1860 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5017/3/console |
   | versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   




> Use secure XML parser utils in YARN
>

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621396#comment-17621396
 ] 

ASF GitHub Bot commented on YARN-11345:
---

slfan1989 commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1001179187


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +155,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null) {
+  Collection apps = appsInfo.getApps();
+  if (CollectionUtils.isNotEmpty(apps)) {
+int numApps = apps.size();
+int i = 0;

Review Comment:
   I put i++ after try...catch and the loop can end even if the application has 
errors.





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621394#comment-17621394
 ] 

ASF GitHub Bot commented on YARN-11345:
---

slfan1989 commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1001179567


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +155,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null) {
+  Collection apps = appsInfo.getApps();
+  if (CollectionUtils.isNotEmpty(apps)) {
+int numApps = apps.size();
+int i = 0;
+for (AppInfo app : apps) {
+  try {
+String percent = String.format("%.1f", app.getProgress() * 100.0F);
+String trackingURL =
+app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
+
+// AppID numerical value parsed by parseHadoopID in 
yarn.dt.plugins.js
+appsTableData.append("[\"")
+.append("")
+.append(app.getAppId()).append("\",\"")
+.append(escape(app.getUser())).append("\",\"")
+.append(escape(app.getName())).append("\",\"")
+.append(escape(app.getApplicationType())).append("\",\"")
+.append(escape(app.getQueue())).append("\",\"")
+.append(app.getPriority()).append("\",\"")
+.append(app.getStartTime()).append("\",\"")
+.append(app.getFinishTime()).append("\",\"")
+.append(app.getState()).append("\",\"")
+.append(app.getFinalStatus()).append("\",\"")
+// Progress bar
+.append("  ").append("  
")
+// History link
+.append("\",\"")
+.append("History").append("");
+appsTableData.append("\"]\n");
+
+if (i < numApps - 1) {
+  appsTableData.append(",");
+}
+  } catch (Exception e) {
+LOG.info("Cannot add application {}: {}", app.getAppId(), 
e.getMessage());
+  }
+  i++;

Review Comment:
   i++ is placed after try...catch.





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621392#comment-17621392
 ] 

ASF GitHub Bot commented on YARN-11345:
---

slfan1989 commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1001179187


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +155,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null) {
+  Collection apps = appsInfo.getApps();
+  if (CollectionUtils.isNotEmpty(apps)) {
+int numApps = apps.size();
+int i = 0;

Review Comment:
   I put i++ after try...catch, because if any app is affected, the loop is 
guaranteed to end.





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11295) [Federation] Router Support DelegationToken in MemoryStore mode

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621387#comment-17621387
 ] 

ASF GitHub Bot commented on YARN-11295:
---

slfan1989 commented on PR #5032:
URL: https://github.com/apache/hadoop/pull/5032#issuecomment-1286226835

   @goiri Thank you very much for helping to review the code!




> [Federation] Router Support DelegationToken in MemoryStore mode
> ---
>
> Key: YARN-11295
> URL: https://issues.apache.org/jira/browse/YARN-11295
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11229) [Federation] Add checkUserAccessToQueue REST APIs for Router

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621357#comment-17621357
 ] 

ASF GitHub Bot commented on YARN-11229:
---

hadoop-yetus commented on PR #4929:
URL: https://github.com/apache/hadoop/pull/4929#issuecomment-1286100860

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 38s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 4 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  16m  6s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  26m  3s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   4m  0s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   3m 28s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 30s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   2m  2s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 37s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 20s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 47s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  21m 15s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 29s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 29s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 54s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | -1 :x: |  javac  |   3m 54s | 
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/13/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt)
 |  
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
 with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 1 new + 437 
unchanged - 1 fixed = 438 total (was 438)  |
   | +1 :green_heart: |  compile  |   3m 19s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   3m 19s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   1m  8s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/13/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
 |  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 
new + 0 unchanged - 3 fixed = 1 total (was 3)  |
   | +1 :green_heart: |  mvnsite  |   1m 30s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 24s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 11s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  2s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m 32s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  99m 14s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |   5m 13s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 43s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 226m 57s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/13/artifact/out/Dockerfile
 |
   |

[jira] [Commented] (YARN-11295) [Federation] Router Support DelegationToken in MemoryStore mode

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621351#comment-17621351
 ] 

ASF GitHub Bot commented on YARN-11295:
---

goiri merged PR #5032:
URL: https://github.com/apache/hadoop/pull/5032




> [Federation] Router Support DelegationToken in MemoryStore mode
> ---
>
> Key: YARN-11295
> URL: https://issues.apache.org/jira/browse/YARN-11295
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Resolved] (YARN-11295) [Federation] Router Support DelegationToken in MemoryStore mode

2022-10-20 Thread Jira



 [ 
https://issues.apache.org/jira/browse/YARN-11295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Íñigo Goiri resolved YARN-11295.

Fix Version/s: 3.4.0
 Hadoop Flags: Reviewed
   Resolution: Fixed

> [Federation] Router Support DelegationToken in MemoryStore mode
> ---
>
> Key: YARN-11295
> URL: https://issues.apache.org/jira/browse/YARN-11295
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621350#comment-17621350
 ] 

ASF GitHub Bot commented on YARN-11345:
---

goiri commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1001069884


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +155,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null) {
+  Collection apps = appsInfo.getApps();
+  if (CollectionUtils.isNotEmpty(apps)) {
+int numApps = apps.size();
+int i = 0;

Review Comment:
   i++ missing?





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11332) [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621348#comment-17621348
 ] 

ASF GitHub Bot commented on YARN-11332:
---

hadoop-yetus commented on PR #4982:
URL: https://github.com/apache/hadoop/pull/4982#issuecomment-1286083147

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 46s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  xmllint  |   0m  1s |  |  xmllint was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m 44s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  27m 21s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  10m 13s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   9m 22s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   2m 13s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 30s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   3m 25s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   3m 16s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   6m 14s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m  5s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 31s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 55s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  10m 23s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |  10m 23s |  |  
hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 
with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 0 new + 730 
unchanged - 2 fixed = 730 total (was 732)  |
   | +1 :green_heart: |  compile  |  10m 31s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |  10m 31s |  |  
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
 with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 generated 0 new 
+ 640 unchanged - 2 fixed = 640 total (was 642)  |
   | +1 :green_heart: |  blanks  |   0m  1s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   2m  2s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   3m 18s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   3m 17s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   3m  5s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   6m 30s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 40s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 30s |  |  hadoop-yarn-api in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   5m 30s |  |  hadoop-yarn-common in the patch 
passed.  |
   | -1 :x: |  unit  |   5m 35s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4982/10/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt)
 |  hadoop-yarn-server-router in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 12s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 190m 33s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | 
hadoop.yarn.server.router.clientrm.TestFederationClientInterceptorRetry |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4982/10/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4982 |
   | Optional Tests | dupname asflicense compile

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621335#comment-17621335
 ] 

ASF GitHub Bot commented on YARN-11345:
---

hadoop-yetus commented on PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#issuecomment-1286042680

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 52s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m 28s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  28m 33s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  10m 23s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   8m 49s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 46s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 32s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 35s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 23s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 58s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  23m 25s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 24s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   0m 59s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   9m 47s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |   9m 47s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   8m 45s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   8m 45s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 36s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 24s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 18s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 18s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   2m 59s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 35s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |   4m 45s |  |  hadoop-yarn-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   5m 21s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 48s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 162m 43s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5030/4/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5030 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 5262eacd5120 4.15.0-192-generic #203-Ubuntu SMP Wed Aug 10 
17:40:03 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / ca3270139a999c571b191194487a2fc118a19152 |
   | Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5030/4/testReport/ |
   | Max. process+thread count | 753 (vs. ulimit of 5500) |
   | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:

[jira] [Commented] (YARN-11037) Add configurable logic to split resource request to least loaded SC

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621284#comment-17621284
 ] 

ASF GitHub Bot commented on YARN-11037:
---

minni31 commented on PR #3760:
URL: https://github.com/apache/hadoop/pull/3760#issuecomment-1285924514

   @bibinchundatt Can you please review this PR




> Add configurable logic to split resource request to least loaded SC
> ---
>
> Key: YARN-11037
> URL: https://issues.apache.org/jira/browse/YARN-11037
> Project: Hadoop YARN
>  Issue Type: Task
>Reporter: Minni Mittal
>Assignee: Minni Mittal
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Add configurable logic to split resource request to least loaded subcluster.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11342) [Federation] Refactor FederationClientInterceptor#submitApplication Use FederationActionRetry

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621247#comment-17621247
 ] 

ASF GitHub Bot commented on YARN-11342:
---

slfan1989 commented on PR #5005:
URL: https://github.com/apache/hadoop/pull/5005#issuecomment-1285877687

   @goiri Thank you very much for helping to review the code!




> [Federation] Refactor FederationClientInterceptor#submitApplication Use 
> FederationActionRetry 
> --
>
> Key: YARN-11342
> URL: https://issues.apache.org/jira/browse/YARN-11342
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>
> The retry logic of FederationClientInterceptor#submitApplication is more 
> complicated, so refactor this part of the code.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11229) [Federation] Add checkUserAccessToQueue REST APIs for Router

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621246#comment-17621246
 ] 

ASF GitHub Bot commented on YARN-11229:
---

slfan1989 commented on code in PR #4929:
URL: https://github.com/apache/hadoop/pull/4929#discussion_r1000889253


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestFederationInterceptorRESTRetry.java:
##
@@ -377,48 +378,37 @@ public void testGetNodeOneBadOneGood()
* composed of only 1 bad SubCluster.
*/
   @Test
-  public void testGetNodesOneBadSC()
-  throws YarnException, IOException, InterruptedException {
+  public void testGetNodesOneBadSC() throws Exception {
 
 setupCluster(Arrays.asList(bad2));
 
-NodesInfo response = interceptor.getNodes(null);
-Assert.assertNotNull(response);
-Assert.assertEquals(0, response.getNodes().size());
-// The remove duplicate operations is tested in TestRouterWebServiceUtil
+LambdaTestUtils.intercept(YarnRuntimeException.class, "RM is stopped",
+() -> interceptor.getNodes(null));
   }
 
   /**
* This test validates the correctness of GetNodes in case the cluster is
* composed of only 2 bad SubClusters.
*/
   @Test
-  public void testGetNodesTwoBadSCs()
-  throws YarnException, IOException, InterruptedException {
+  public void testGetNodesTwoBadSCs() throws Exception {
+
 setupCluster(Arrays.asList(bad1, bad2));
 
-NodesInfo response = interceptor.getNodes(null);
-Assert.assertNotNull(response);

Review Comment:
   In Federation mode, if there is a problem with some sub-clusters, we should 
throw an exception directly to tell the user a clear error message, and should 
not return a normal response with empty node list.





> [Federation] Add checkUserAccessToQueue REST APIs for Router
> 
>
> Key: YARN-11229
> URL: https://issues.apache.org/jira/browse/YARN-11229
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation
>Affects Versions: 3.4.0, 3.3.4
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621243#comment-17621243
 ] 

ASF GitHub Bot commented on YARN-11345:
---

slfan1989 commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1000883075


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +154,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null && CollectionUtils.isNotEmpty(appsInfo.getApps())) {
+  for (AppInfo app : appsInfo.getApps()) {
+try {
+
+  String percent = String.format("%.1f", app.getProgress() * 100.0F);
+  String trackingURL =
+  app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
+
+  // AppID numerical value parsed by parseHadoopID in 
yarn.dt.plugins.js
+  appsTableData.append("[\"")
+  .append("")
+  .append(app.getAppId()).append("\",\"")
+  .append(escape(app.getUser())).append("\",\"")
+  .append(escape(app.getName())).append("\",\"")
+  .append(escape(app.getApplicationType())).append("\",\"")
+  .append(escape(app.getQueue())).append("\",\"")
+  .append(app.getPriority()).append("\",\"")
+  .append(app.getStartTime()).append("\",\"")
+  .append(app.getFinishTime()).append("\",\"")
+  .append(app.getState()).append("\",\"")
+  .append(app.getFinalStatus()).append("\",\"")
+  // Progress bar
+  .append("  ").append("  ")
+  // History link
+  .append("\",\"")
+  .append("History").append("");
+  appsTableData.append("\"],\n");
+
+} catch (Exception e) {
+  LOG.info("Cannot add application {}: {}", app.getAppId(), 
e.getMessage());
+}
+  }
+
+  // The purpose of this part of the code is to remove redundant commas.

Review Comment:
   Thanks for your suggestion, I will refactor this part of the code.





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621235#comment-17621235
 ] 

ASF GitHub Bot commented on YARN-11356:
---

bteke commented on code in PR #5052:
URL: https://github.com/apache/hadoop/pull/5052#discussion_r1000875531


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.11.5/css/jquery.dataTables.css:
##
@@ -1,21 +1,3 @@
-/**

Review Comment:
   We shouldn't remove it, however I'm not sure about the second part of the 
sentence, what do you mean by "if the JS pair of this file this"?





> Upgrade DataTables to 1.11.5 to fix CVEs
> 
>
> Key: YARN-11356
> URL: https://issues.apache.org/jira/browse/YARN-11356
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 3.3.4
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Major
>  Labels: pull-request-available
>
> This ticket is intended to fix the following CVEs in the *DataTables.net* 
> lib, by upgrading the lib to 1.11.5 
> *CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
> vulnerable to Prototype Pollution due to an incomplete fix for 
> [https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].
> [https://nvd.nist.gov/vuln/detail/CVE-2020-28458]
> *CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
> before 1.11.3. If an array is passed to the HTML escape entities function it 
> would not have its contents escaped.
> [https://nvd.nist.gov/vuln/detail/CVE-2021-23445]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Resolved] (YARN-11342) [Federation] Refactor FederationClientInterceptor#submitApplication Use FederationActionRetry

2022-10-20 Thread Jira



 [ 
https://issues.apache.org/jira/browse/YARN-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Íñigo Goiri resolved YARN-11342.

Fix Version/s: 3.4.0
 Hadoop Flags: Reviewed
   Resolution: Fixed

> [Federation] Refactor FederationClientInterceptor#submitApplication Use 
> FederationActionRetry 
> --
>
> Key: YARN-11342
> URL: https://issues.apache.org/jira/browse/YARN-11342
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>
> The retry logic of FederationClientInterceptor#submitApplication is more 
> complicated, so refactor this part of the code.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11342) [Federation] Refactor FederationClientInterceptor#submitApplication Use FederationActionRetry

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621224#comment-17621224
 ] 

ASF GitHub Bot commented on YARN-11342:
---

goiri merged PR #5005:
URL: https://github.com/apache/hadoop/pull/5005




> [Federation] Refactor FederationClientInterceptor#submitApplication Use 
> FederationActionRetry 
> --
>
> Key: YARN-11342
> URL: https://issues.apache.org/jira/browse/YARN-11342
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>
> The retry logic of FederationClientInterceptor#submitApplication is more 
> complicated, so refactor this part of the code.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11345) [Federation] Refactoring Yarn Router's Application Web Page.

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621221#comment-17621221
 ] 

ASF GitHub Bot commented on YARN-11345:
---

goiri commented on code in PR #5030:
URL: https://github.com/apache/hadoop/pull/5030#discussion_r1000850995


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/AppsBlock.java:
##
@@ -81,53 +154,57 @@ protected void render(Block html) {
 
 // Render the applications
 StringBuilder appsTableData = new StringBuilder("[\n");
-for (AppInfo app : apps.getApps()) {
-  try {
-
-String percent = String.format("%.1f", app.getProgress() * 100.0F);
-String trackingURL =
-app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
-// AppID numerical value parsed by parseHadoopID in yarn.dt.plugins.js
-appsTableData.append("[\"")
-.append("")
-.append(app.getAppId()).append("\",\"")
-.append(escape(app.getUser())).append("\",\"")
-.append(escape(app.getName())).append("\",\"")
-.append(escape(app.getApplicationType())).append("\",\"")
-.append(escape(app.getQueue())).append("\",\"")
-.append(String.valueOf(app.getPriority())).append("\",\"")
-.append(app.getStartTime()).append("\",\"")
-.append(app.getFinishTime()).append("\",\"")
-.append(app.getState()).append("\",\"")
-.append(app.getFinalStatus()).append("\",\"")
-// Progress bar
-.append("  ").append("  ")
-// History link
-.append("\",\"")
-.append("History").append("");
-appsTableData.append("\"],\n");
-
-  } catch (Exception e) {
-LOG.info(
-"Cannot add application {}: {}", app.getAppId(), e.getMessage());
+
+if (appsInfo != null && CollectionUtils.isNotEmpty(appsInfo.getApps())) {
+  for (AppInfo app : appsInfo.getApps()) {
+try {
+
+  String percent = String.format("%.1f", app.getProgress() * 100.0F);
+  String trackingURL =
+  app.getTrackingUrl() == null ? "#" : app.getTrackingUrl();
+
+  // AppID numerical value parsed by parseHadoopID in 
yarn.dt.plugins.js
+  appsTableData.append("[\"")
+  .append("")
+  .append(app.getAppId()).append("\",\"")
+  .append(escape(app.getUser())).append("\",\"")
+  .append(escape(app.getName())).append("\",\"")
+  .append(escape(app.getApplicationType())).append("\",\"")
+  .append(escape(app.getQueue())).append("\",\"")
+  .append(app.getPriority()).append("\",\"")
+  .append(app.getStartTime()).append("\",\"")
+  .append(app.getFinishTime()).append("\",\"")
+  .append(app.getState()).append("\",\"")
+  .append(app.getFinalStatus()).append("\",\"")
+  // Progress bar
+  .append("  ").append("  ")
+  // History link
+  .append("\",\"")
+  .append("History").append("");
+  appsTableData.append("\"],\n");
+
+} catch (Exception e) {
+  LOG.info("Cannot add application {}: {}", app.getAppId(), 
e.getMessage());
+}
+  }
+
+  // The purpose of this part of the code is to remove redundant commas.

Review Comment:
   We know how many apps there will be, we should only add it if not the last:
   ```
   if (appsInfo != null) {
 Collection apps = appsInfo.getApps();
 if (CollectionUtils.isNotEmpty(apps)) {
   int numApps = apps.size();
   for (AppInfo app: apps) {
 ...
 if (i < numApps - 1) {
   appsTableData.append(",");
 }
   }
 }
   }
   ```





> [Federation] Refactoring Yarn Router's Application Web Page.
> 
>
> Key: YARN-11345
> URL: https://issues.apache.org/jira/browse/YARN-11345
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: federation, router
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11229) [Federation] Add checkUserAccessToQueue REST APIs for Router

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621164#comment-17621164
 ] 

ASF GitHub Bot commented on YARN-11229:
---

hadoop-yetus commented on PR #4929:
URL: https://github.com/apache/hadoop/pull/4929#issuecomment-1285672095

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 40s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 4 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m 48s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  25m 38s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   3m 59s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   3m 26s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 22s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 53s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 49s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 33s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 16s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 16s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  20m 40s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 25s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 53s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | -1 :x: |  javac  |   3m 53s | 
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/12/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt)
 |  
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
 with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 1 new + 437 
unchanged - 1 fixed = 438 total (was 438)  |
   | +1 :green_heart: |  compile  |   3m 20s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   3m 20s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   1m  8s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/12/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
 |  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 
new + 0 unchanged - 3 fixed = 1 total (was 3)  |
   | +1 :green_heart: |  mvnsite  |   1m 35s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 18s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m  8s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  1s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m 15s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  99m  2s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |   5m  5s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 47s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 224m 55s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/12/artifact/out/Dockerfile
 |
   |

[jira] [Commented] (YARN-11229) [Federation] Add checkUserAccessToQueue REST APIs for Router

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621165#comment-17621165
 ] 

ASF GitHub Bot commented on YARN-11229:
---

hadoop-yetus commented on PR #4929:
URL: https://github.com/apache/hadoop/pull/4929#issuecomment-1285674115

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 37s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 4 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  15m 47s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  25m 54s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   4m  2s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   3m 30s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 25s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 53s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 42s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 29s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 27s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 50s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  21m 15s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 31s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 27s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   4m 12s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | -1 :x: |  javac  |   4m 12s | 
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/11/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt)
 |  
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
 with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 1 new + 437 
unchanged - 1 fixed = 438 total (was 438)  |
   | +1 :green_heart: |  compile  |   3m 29s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   3m 29s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   1m  9s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/11/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
 |  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 
new + 0 unchanged - 3 fixed = 1 total (was 3)  |
   | +1 :green_heart: |  mvnsite  |   1m 35s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 11s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m  6s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  6s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m 33s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  99m 29s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |   5m 15s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 43s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 227m 45s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/11/artifact/out/Dockerfile
 |
   |

[jira] [Commented] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621060#comment-17621060
 ] 

ASF GitHub Bot commented on YARN-11356:
---

K0K0V0K commented on code in PR #5052:
URL: https://github.com/apache/hadoop/pull/5052#discussion_r1000546604


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.11.5/css/jquery.dataTables.css:
##
@@ -1,21 +1,3 @@
-/**

Review Comment:
   Do we need this? I mean is it OK to add "The ASF licenses this file to you 
under the Apache License, Version 2.0 (the "License")" if the JS pair of this 
file this
   `opyright 2008-2021 SpryMedia Ltd.
   
This source file is free software, available under the following license:
  MIT license - http://datatables.net/license
   `



##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.11.5/css/jquery.dataTables.css:
##
@@ -1,21 +1,3 @@
-/**

Review Comment:
   Do we need this? I mean is it OK to add "The ASF licenses this file to you 
under the Apache License, Version 2.0 (the "License")" if the JS pair of this 
file this
   
   `Copyright 2008-2021 SpryMedia Ltd.
   
This source file is free software, available under the following license:
  MIT license - http://datatables.net/license
   `





> Upgrade DataTables to 1.11.5 to fix CVEs
> 
>
> Key: YARN-11356
> URL: https://issues.apache.org/jira/browse/YARN-11356
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 3.3.4
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Major
>  Labels: pull-request-available
>
> This ticket is intended to fix the following CVEs in the *DataTables.net* 
> lib, by upgrading the lib to 1.11.5 
> *CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
> vulnerable to Prototype Pollution due to an incomplete fix for 
> [https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].
> [https://nvd.nist.gov/vuln/detail/CVE-2020-28458]
> *CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
> before 1.11.3. If an array is passed to the HTML escape entities function it 
> would not have its contents escaped.
> [https://nvd.nist.gov/vuln/detail/CVE-2021-23445]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621061#comment-17621061
 ] 

ASF GitHub Bot commented on YARN-11356:
---

K0K0V0K commented on code in PR #5052:
URL: https://github.com/apache/hadoop/pull/5052#discussion_r1000546604


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.11.5/css/jquery.dataTables.css:
##
@@ -1,21 +1,3 @@
-/**

Review Comment:
   Do we need this? I mean is it OK to add "The ASF licenses this file to you 
under the Apache License, Version 2.0 (the "License")" if the JS pair of this 
file this
   
   ```
   Copyright 2008-2021 SpryMedia Ltd.
   
This source file is free software, available under the following license:
  MIT license - http://datatables.net/license
   ```





> Upgrade DataTables to 1.11.5 to fix CVEs
> 
>
> Key: YARN-11356
> URL: https://issues.apache.org/jira/browse/YARN-11356
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 3.3.4
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Major
>  Labels: pull-request-available
>
> This ticket is intended to fix the following CVEs in the *DataTables.net* 
> lib, by upgrading the lib to 1.11.5 
> *CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
> vulnerable to Prototype Pollution due to an incomplete fix for 
> [https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].
> [https://nvd.nist.gov/vuln/detail/CVE-2020-28458]
> *CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
> before 1.11.3. If an array is passed to the HTML escape entities function it 
> would not have its contents escaped.
> [https://nvd.nist.gov/vuln/detail/CVE-2021-23445]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated YARN-11356:
--
Labels: pull-request-available  (was: )

> Upgrade DataTables to 1.11.5 to fix CVEs
> 
>
> Key: YARN-11356
> URL: https://issues.apache.org/jira/browse/YARN-11356
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 3.3.4
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Major
>  Labels: pull-request-available
>
> This ticket is intended to fix the following CVEs in the *DataTables.net* 
> lib, by upgrading the lib to 1.11.5 
> *CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
> vulnerable to Prototype Pollution due to an incomplete fix for 
> [https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].
> [https://nvd.nist.gov/vuln/detail/CVE-2020-28458]
> *CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
> before 1.11.3. If an array is passed to the HTML escape entities function it 
> would not have its contents escaped.
> [https://nvd.nist.gov/vuln/detail/CVE-2021-23445]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621047#comment-17621047
 ] 

ASF GitHub Bot commented on YARN-11356:
---

K0K0V0K opened a new pull request, #5052:
URL: https://github.com/apache/hadoop/pull/5052

   ### Description of PR
   
   This PR is intended to fix the following CVEs in the DataTables.net lib, by 
upgrading the lib to 1.11.5 
   
   CVE-2020-28458 (HIGH severity) - All versions of package datatables.net are 
vulnerable to Prototype Pollution due to an incomplete fix for 
https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806.
   
   https://nvd.nist.gov/vuln/detail/CVE-2020-28458
   
   CVE-2021-23445 (MEDIUM severity) - This affects the package datatables.net 
before 1.11.3. If an array is passed to the HTML escape entities function it 
would not have its contents escaped.
   
   https://nvd.nist.gov/vuln/detail/CVE-2021-23445
   
   ### How was this patch tested?
   
   On a local cluster I investigated the UI
   
   - macOS 12.2.1 (21D62)
   - Chrome Version 106.0.5249.119 (Official Build) (arm64)
   
   The new js code is loaded to the browser
   https://user-images.githubusercontent.com/109747532/196940864-5ccd0183-4e64-4654-9b50-20ba2ea1ac4d.png;>
   
   The sort feature is working:
   https://user-images.githubusercontent.com/109747532/196941043-70f359bc-ad11-41f3-9fa3-57d535ccd0b1.png;>
   
   The search feature is working:
   https://user-images.githubusercontent.com/109747532/196941171-032de115-563b-41de-9733-19d63f8e1d8c.png;>
   
   Empty tables are presented:
   https://user-images.githubusercontent.com/109747532/196941326-c007462b-e7ef-4823-a9d2-21d31924cead.png;>
   
   ### For code changes:
   
   - I had to update the bower.json and ember-cli build with the new versions
   - The UI is using a fix DataTable js code what is present in this repo so i 
updated that too.
   - I updated the Licence file also. 




> Upgrade DataTables to 1.11.5 to fix CVEs
> 
>
> Key: YARN-11356
> URL: https://issues.apache.org/jira/browse/YARN-11356
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 3.3.4
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Major
>
> This ticket is intended to fix the following CVEs in the *DataTables.net* 
> lib, by upgrading the lib to 1.11.5 
> *CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
> vulnerable to Prototype Pollution due to an incomplete fix for 
> [https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].
> [https://nvd.nist.gov/vuln/detail/CVE-2020-28458]
> *CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
> before 1.11.3. If an array is passed to the HTML escape entities function it 
> would not have its contents escaped.
> [https://nvd.nist.gov/vuln/detail/CVE-2021-23445]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Resolved] (YARN-11285) LocalizedResources are leaked and its LocalPath are not cleared

2022-10-20 Thread Prabhu Joseph (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph resolved YARN-11285.
--
Resolution: Duplicate

> LocalizedResources are leaked and its LocalPath are not cleared
> ---
>
> Key: YARN-11285
> URL: https://issues.apache.org/jira/browse/YARN-11285
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: nodemanager
>Affects Versions: 3.2.1
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> LocalizedResources are leaked and its LocalPath are not cleared from NM Local 
> Directories.  
> Each container has separate LocalizedResource object and separate local path 
> like below.
> {code}
>/mnt/yarn/usercache/hive/filecache/6/2552419:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug  7 10:24 
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552420:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug  7 10:24 
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552421:
>total 28456
>-r-x-- 1 yarn yarn 29135164 Aug  7 10:24 
> hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar
>/mnt/yarn/usercache/hive/filecache/6/2552422:
>total 28456
> {code}
> NM logs will be filled with below
> {code}
> 2022-08-07 09:00:00,275 INFO 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource
>  (IPC Server handler 4 on 8040): Resource 
> hdfs://hdfscluster/user/svc_di_data_eng/.hiveJars/hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar(->/mnt/yarn/usercache/data_eng_user/filecache/2498262/hive-exec-2.3.4.50-3fd48f33b0c0b82ab431013f0fe794dfe75c31a5027567e6865cccbb49de862b.jar)
>  transitioned from LOCALIZED to null
> 2022-08-07 09:00:00,340 INFO 
> org.apache.hadoop.yarn.util.ProcfsBasedProcessTree (Container Monitor): 
> SmapBasedCumulativeRssmem (bytes) : 0
> 2022-08-07 09:00:00,386 WARN 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource
>  (IPC Server handler 9 on 8040): Can't handle this event at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitionException: Invalid event: 
> LOCALIZED at LOCALIZED
> at 
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
> at 
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at 
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource.handle(LocalizedResource.java:198)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl.handle(LocalResourcesTrackerImpl.java:186)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl.handle(LocalResourcesTrackerImpl.java:58)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.processHeartbeat(ResourceLocalizationService.java:1048)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.processHeartbeat(ResourceLocalizationService.java:722)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.heartbeat(ResourceLocalizationService.java:356)
> at 
> org.apache.hadoop.yarn.server.nodemanager.api.impl.pb.service.LocalizationProtocolPBServiceImpl.heartbeat(LocalizationProtocolPBServiceImpl.java:48)
> at 
> org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2.callBlockingMethod(LocalizationProtocol.java:63)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:850)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:793)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2489)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To

[jira] [Updated] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread Bence Kosztolnik (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bence Kosztolnik updated YARN-11356:

Description: 
This ticket is intended to fix the following CVEs in the *DataTables.net* lib, 
by upgrading the lib to 1.11.5 

*CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
vulnerable to Prototype Pollution due to an incomplete fix for 
[https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].

[https://nvd.nist.gov/vuln/detail/CVE-2020-28458]

*CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
before 1.11.3. If an array is passed to the HTML escape entities function it 
would not have its contents escaped.

[https://nvd.nist.gov/vuln/detail/CVE-2021-23445]

  was:
This ticket is intended to fix the following CVEs in the *DataTables.net* lib.

*CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
vulnerable to Prototype Pollution due to an incomplete fix for 
[https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].

https://nvd.nist.gov/vuln/detail/CVE-2020-28458

*CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
before 1.11.3. If an array is passed to the HTML escape entities function it 
would not have its contents escaped.

https://nvd.nist.gov/vuln/detail/CVE-2021-23445


> Upgrade DataTables to 1.11.5 to fix CVEs
> 
>
> Key: YARN-11356
> URL: https://issues.apache.org/jira/browse/YARN-11356
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 3.3.4
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Major
>
> This ticket is intended to fix the following CVEs in the *DataTables.net* 
> lib, by upgrading the lib to 1.11.5 
> *CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
> vulnerable to Prototype Pollution due to an incomplete fix for 
> [https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].
> [https://nvd.nist.gov/vuln/detail/CVE-2020-28458]
> *CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
> before 1.11.3. If an array is passed to the HTML escape entities function it 
> would not have its contents escaped.
> [https://nvd.nist.gov/vuln/detail/CVE-2021-23445]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Created] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread Bence Kosztolnik (Jira)

Bence Kosztolnik created YARN-11356:
---

 Summary: Upgrade DataTables to 1.11.5 to fix CVEs
 Key: YARN-11356
 URL: https://issues.apache.org/jira/browse/YARN-11356
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: yarn
Affects Versions: 3.3.4
Reporter: Bence Kosztolnik


This ticket is intended to fix the following CVEs in the *DataTables.net* lib.

*CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
vulnerable to Prototype Pollution due to an incomplete fix for 
[https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].

https://nvd.nist.gov/vuln/detail/CVE-2020-28458

*CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
before 1.11.3. If an array is passed to the HTML escape entities function it 
would not have its contents escaped.

https://nvd.nist.gov/vuln/detail/CVE-2021-23445



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-11356) Upgrade DataTables to 1.11.5 to fix CVEs

2022-10-20 Thread Bence Kosztolnik (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bence Kosztolnik reassigned YARN-11356:
---

Assignee: Bence Kosztolnik

> Upgrade DataTables to 1.11.5 to fix CVEs
> 
>
> Key: YARN-11356
> URL: https://issues.apache.org/jira/browse/YARN-11356
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 3.3.4
>Reporter: Bence Kosztolnik
>Assignee: Bence Kosztolnik
>Priority: Major
>
> This ticket is intended to fix the following CVEs in the *DataTables.net* lib.
> *CVE-2020-28458 (HIGH severity)* - All versions of package datatables.net are 
> vulnerable to Prototype Pollution due to an incomplete fix for 
> [https://snyk.io/vuln/SNYK-JS-DATATABLESNET-598806].
> https://nvd.nist.gov/vuln/detail/CVE-2020-28458
> *CVE-2021-23445 (MEDIUM severity)* - This affects the package datatables.net 
> before 1.11.3. If an array is passed to the HTML escape entities function it 
> would not have its contents escaped.
> https://nvd.nist.gov/vuln/detail/CVE-2021-23445



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3

2022-10-20 Thread Prabhu Joseph (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph reassigned YARN-11355:


Assignee: Vineeth Naroju  (was: Prabhu Joseph)

> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3
> --
>
> Key: YARN-11355
> URL: https://issues.apache.org/jira/browse/YARN-11355
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: client
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Vineeth Naroju
>Priority: Major
>
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during 
> initial retry.
> *Repro:*
> {code:java}
> 1. YARN Cluster with three master nodes rm1,rm2 and rm3
> 2. rm3 is active
> 3. yarn node -list or any other yarn client calls takes more than 30 seconds.
>  {code}
> The initial failover to rm2 is immediate but then the failover to rm3 is 
> after ~3 ms. Current RetryPolicy does not honor the number of master 
> nodes. It has to perform atleast one immediate failover to every rm.
> {code:java}
> 2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider: 
> Failing over to rm2
> 2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler: 
> java.net.ConnectException: Call From local to remote:8032 failed on 
> connection exception: java.net.ConnectException: Connection refused; For more 
> details see:  http://wiki.apache.org/hadoop/ConnectionRefused, while invoking 
> ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1 
> failover attempts. Trying to failover after sleeping for 21139ms.
> {code}
>  
> *Workaround:*
> Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100. 
> This will do immediate failover to rm3 but there will be too many retries 
> when there is no active resourcemanager.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3

2022-10-20 Thread Prabhu Joseph (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620888#comment-17620888
 ] 

Prabhu Joseph commented on YARN-11355:
--

Yes Done.

> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3
> --
>
> Key: YARN-11355
> URL: https://issues.apache.org/jira/browse/YARN-11355
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: client
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Vineeth Naroju
>Priority: Major
>
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during 
> initial retry.
> *Repro:*
> {code:java}
> 1. YARN Cluster with three master nodes rm1,rm2 and rm3
> 2. rm3 is active
> 3. yarn node -list or any other yarn client calls takes more than 30 seconds.
>  {code}
> The initial failover to rm2 is immediate but then the failover to rm3 is 
> after ~3 ms. Current RetryPolicy does not honor the number of master 
> nodes. It has to perform atleast one immediate failover to every rm.
> {code:java}
> 2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider: 
> Failing over to rm2
> 2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler: 
> java.net.ConnectException: Call From local to remote:8032 failed on 
> connection exception: java.net.ConnectException: Connection refused; For more 
> details see:  http://wiki.apache.org/hadoop/ConnectionRefused, while invoking 
> ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1 
> failover attempts. Trying to failover after sleeping for 21139ms.
> {code}
>  
> *Workaround:*
> Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100. 
> This will do immediate failover to rm3 but there will be too many retries 
> when there is no active resourcemanager.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3

2022-10-20 Thread Vineeth Naroju (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620867#comment-17620867
 ] 

Vineeth Naroju commented on YARN-11355:
---

I would like to pick this up. [~prabhujoseph] could you please assign this to 
me ?

> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3
> --
>
> Key: YARN-11355
> URL: https://issues.apache.org/jira/browse/YARN-11355
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: client
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during 
> initial retry.
> *Repro:*
> {code:java}
> 1. YARN Cluster with three master nodes rm1,rm2 and rm3
> 2. rm3 is active
> 3. yarn node -list or any other yarn client calls takes more than 30 seconds.
>  {code}
> The initial failover to rm2 is immediate but then the failover to rm3 is 
> after ~3 ms. Current RetryPolicy does not honor the number of master 
> nodes. It has to perform atleast one immediate failover to every rm.
> {code:java}
> 2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider: 
> Failing over to rm2
> 2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler: 
> java.net.ConnectException: Call From local to remote:8032 failed on 
> connection exception: java.net.ConnectException: Connection refused; For more 
> details see:  http://wiki.apache.org/hadoop/ConnectionRefused, while invoking 
> ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1 
> failover attempts. Trying to failover after sleeping for 21139ms.
> {code}
>  
> *Workaround:*
> Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100. 
> This will do immediate failover to rm3 but there will be too many retries 
> when there is no active resourcemanager.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3

2022-10-20 Thread Prabhu Joseph (Jira)



 [ 
https://issues.apache.org/jira/browse/YARN-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prabhu Joseph updated YARN-11355:
-
Description: 
YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during 
initial retry.

*Repro:*
{code:java}
1. YARN Cluster with three master nodes rm1,rm2 and rm3
2. rm3 is active
3. yarn node -list or any other yarn client calls takes more than 30 seconds.
 {code}
The initial failover to rm2 is immediate but then the failover to rm3 is after 
~3 ms. Current RetryPolicy does not honor the number of master nodes. It 
has to perform atleast one immediate failover to every rm.
{code:java}
2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider: Failing 
over to rm2
2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler: 
java.net.ConnectException: Call From local to remote:8032 failed on connection 
exception: java.net.ConnectException: Connection refused; For more details see: 
 http://wiki.apache.org/hadoop/ConnectionRefused, while invoking 
ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1 failover 
attempts. Trying to failover after sleeping for 21139ms.
{code}
 

*Workaround:*

Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100. 
This will do immediate failover to rm3 but there will be too many retries when 
there is no active resourcemanager.
 

 

  was:
YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during 
initial retry.

*Repro:*
{code:java}
1. YARN Cluster with three master nodes rm1,rm2 and rm3
2. rm3 is active
3. yarn node -list or any other yarn client calls takes more than 30 seconds.
 {code}
The initial failover to rm2 is immediate but then the failover to rm3 is after 
~3 ms. Current RetryPolicy does not honor the number of master nodes. It 
has to perform atleast one immediate failover to every rm.
{code:java}
2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider: Failing 
over to rm2
2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler: 
java.net.ConnectException: Call From local to remote:8032 failed on connection 
exception: java.net.ConnectException: Connection refused; For more details see: 
 http://wiki.apache.org/hadoop/ConnectionRefused, while invoking 
ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1 failover 
attempts. Trying to failover after sleeping for 21139ms.
{code}
 

*{*}Workaround:{*}*

Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100. 
This will do immediate failover to rm3 but there will be too many retries when 
there is no active resourcemanager.
 

 


> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3
> --
>
> Key: YARN-11355
> URL: https://issues.apache.org/jira/browse/YARN-11355
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: client
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
>
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during 
> initial retry.
> *Repro:*
> {code:java}
> 1. YARN Cluster with three master nodes rm1,rm2 and rm3
> 2. rm3 is active
> 3. yarn node -list or any other yarn client calls takes more than 30 seconds.
>  {code}
> The initial failover to rm2 is immediate but then the failover to rm3 is 
> after ~3 ms. Current RetryPolicy does not honor the number of master 
> nodes. It has to perform atleast one immediate failover to every rm.
> {code:java}
> 2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider: 
> Failing over to rm2
> 2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler: 
> java.net.ConnectException: Call From local to remote:8032 failed on 
> connection exception: java.net.ConnectException: Connection refused; For more 
> details see:  http://wiki.apache.org/hadoop/ConnectionRefused, while invoking 
> ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1 
> failover attempts. Trying to failover after sleeping for 21139ms.
> {code}
>  
> *Workaround:*
> Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100. 
> This will do immediate failover to rm3 but there will be too many retries 
> when there is no active resourcemanager.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Created] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3

2022-10-20 Thread Prabhu Joseph (Jira)

Prabhu Joseph created YARN-11355:


 Summary: YARN Client Failovers immediately to rm2 but takes 
~3ms to rm3
 Key: YARN-11355
 URL: https://issues.apache.org/jira/browse/YARN-11355
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 3.4.0
Reporter: Prabhu Joseph
Assignee: Prabhu Joseph


YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during 
initial retry.

*Repro:*
{code:java}
1. YARN Cluster with three master nodes rm1,rm2 and rm3
2. rm3 is active
3. yarn node -list or any other yarn client calls takes more than 30 seconds.
 {code}
The initial failover to rm2 is immediate but then the failover to rm3 is after 
~3 ms. Current RetryPolicy does not honor the number of master nodes. It 
has to perform atleast one immediate failover to every rm.
{code:java}
2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider: Failing 
over to rm2
2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler: 
java.net.ConnectException: Call From local to remote:8032 failed on connection 
exception: java.net.ConnectException: Connection refused; For more details see: 
 http://wiki.apache.org/hadoop/ConnectionRefused, while invoking 
ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1 failover 
attempts. Trying to failover after sleeping for 21139ms.
{code}
 

*{*}Workaround:{*}*

Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100. 
This will do immediate failover to rm3 but there will be too many retries when 
there is no active resourcemanager.
 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11352) Support new API to get the total resource available in Yarn

2022-10-20 Thread Prabhu Joseph (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620841#comment-17620841
 ] 

Prabhu Joseph commented on YARN-11352:
--

Thanks [~SanjayKumarSahu] for reporting the issue. Currently Tez Splits is 
based on AMRMClient#getAvailableResources which is the HeadRoom based on How 
Much the Queue/Job User/Partition Limit is set. Changing Split calculation 
based on Total YARN Cluster Resource will lead to high Task Parallelism and Tez 
Job waiting for other queue/user/Partition resources which it won't get.

 
{code:java}
  /**
   * Get the currently available resources in the cluster.
   * A valid value is available after a call to allocate has been made
   * @return Currently available resources
   */
  public abstract Resource getAvailableResources();
{code}

> Support new API to get the total resource available in Yarn
> ---
>
> Key: YARN-11352
> URL: https://issues.apache.org/jira/browse/YARN-11352
> Project: Hadoop YARN
>  Issue Type: New Feature
>  Components: capacity scheduler, resourcemanager, yarn
>Affects Versions: 3.4.0
>Reporter: Sanjay Kumar Sahu
>Priority: Major
>
> Hive needs total resource available in yarn by AMRMClient interface. This 
> help hive to decide the split count (Fix the split calculation logic for Hive 
> on Tez/LLAP in  clusters).
>  
> The improvement is identified as a problem in split calculation.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11226) [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620836#comment-17620836
 ] 

ASF GitHub Bot commented on YARN-11226:
---

hadoop-yetus commented on PR #4892:
URL: https://github.com/apache/hadoop/pull/4892#issuecomment-1285058455

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 44s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 5 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  39m  7s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 47s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 38s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   0m 44s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 45s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 47s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   1m 11s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 35s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  20m 59s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 28s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 30s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 30s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 31s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   0m 31s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 19s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 32s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 25s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 23s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   0m 57s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m 15s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |   5m 36s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 47s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   |  98m 30s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/26/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/4892 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux df43a5ecf168 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 43bb877602faf41a95589c602dafa3b03802055f |
   | Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4892/26/testReport/ |
   | Max. process+thread count | 840 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
   | Console output |

[jira] [Commented] (YARN-11229) [Federation] Add checkUserAccessToQueue REST APIs for Router

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620820#comment-17620820
 ] 

ASF GitHub Bot commented on YARN-11229:
---

hadoop-yetus commented on PR #4929:
URL: https://github.com/apache/hadoop/pull/4929#issuecomment-1285046365

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 48s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 4 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  18m 51s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  26m 47s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   4m  6s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   3m 31s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | -0 :warning: |  checkstyle  |   1m 34s | 
[/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/10/artifact/out/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
 |  The patch fails to run checkstyle in hadoop-yarn-server  |
   | +1 :green_heart: |  mvnsite  |   1m 57s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 52s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 28s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 22s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 44s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  21m 10s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 26s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 47s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | -1 :x: |  javac  |   3m 47s | 
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/10/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt)
 |  
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
 with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 1 new + 439 
unchanged - 1 fixed = 440 total (was 440)  |
   | +1 :green_heart: |  compile  |   3m 14s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   3m 14s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   1m  7s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/10/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
 |  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 5 
new + 0 unchanged - 0 fixed = 5 total (was 0)  |
   | +1 :green_heart: |  mvnsite  |   1m 40s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 24s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 10s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  2s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m 45s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  99m  8s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |   5m  6s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 43s |  |  The patch does not

[jira] [Commented] (YARN-11342) [Federation] Refactor FederationClientInterceptor#submitApplication Use FederationActionRetry

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620795#comment-17620795
 ] 

ASF GitHub Bot commented on YARN-11342:
---

hadoop-yetus commented on PR #5005:
URL: https://github.com/apache/hadoop/pull/5005#issuecomment-1285039698

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 38s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +0 :ok: |  xmllint  |   0m  0s |  |  xmllint was not available.  |
   | +0 :ok: |  markdownlint  |   0m  0s |  |  markdownlint was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  17m  9s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  25m 54s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   9m 49s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   8m 44s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   2m  6s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   7m 16s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   6m 47s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   6m 26s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +0 :ok: |  spotbugs  |   0m 50s |  |  
branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no spotbugs output file 
(spotbugsXml.xml)  |
   | +1 :green_heart: |  shadedclient  |  20m 58s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 25s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   4m  5s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   9m 11s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |   9m 11s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   8m 38s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   8m 38s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  1s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m 47s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   6m 41s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   5m 41s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   5m 37s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +0 :ok: |  spotbugs  |   0m 50s |  |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from spotbugs  |
   | +1 :green_heart: |  shadedclient  |  20m 55s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 26s |  |  hadoop-yarn-api in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   5m 14s |  |  hadoop-yarn-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   3m 30s |  |  hadoop-yarn-server-common in 
the patch passed.  |
   | +1 :green_heart: |  unit  |  99m 34s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |   5m 29s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  unit  |   0m 57s |  |  hadoop-yarn-site in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   1m 12s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 313m 30s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5005/9/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/5005 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint 
markdownlint |
   | uname | Linux 4f1300d673dd 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   |

[jira] [Commented] (YARN-11229) [Federation] Add checkUserAccessToQueue REST APIs for Router

2022-10-20 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17620770#comment-17620770
 ] 

ASF GitHub Bot commented on YARN-11229:
---

hadoop-yetus commented on PR #4929:
URL: https://github.com/apache/hadoop/pull/4929#issuecomment-1285001579

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   1m  7s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  2s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  2s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 4 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  16m 37s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  29m 20s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   4m 23s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   3m 30s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  checkstyle  |   1m 26s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 50s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 36s |  |  trunk passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m 21s |  |  trunk passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m 13s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  23m 24s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  23m 45s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 24s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 23s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   4m  2s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | -1 :x: |  javac  |   4m  2s | 
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/9/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt)
 |  
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
 with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 1 new + 439 
unchanged - 1 fixed = 440 total (was 440)  |
   | +1 :green_heart: |  compile  |   3m 21s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  javac  |   3m 21s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   1m 10s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/9/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
 |  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 
new + 4 unchanged - 3 fixed = 5 total (was 7)  |
   | +1 :green_heart: |  mvnsite  |   1m 28s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 11s |  |  the patch passed with JDK 
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   1m  5s |  |  the patch passed with JDK 
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07  |
   | +1 :green_heart: |  spotbugs  |   3m  7s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 30s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  | 103m  8s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  unit  |   5m  3s |  |  hadoop-yarn-server-router in 
the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 40s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 239m 41s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4929/9/artifact/out/Dockerfile
 |
   | GITHUB

54 matches

Mail list logo