[jira] [Updated] (YARN-11538) CS UI: queue filter do not work as expected when submitting apps with leaf queue‘s name
[
https://issues.apache.org/jira/browse/YARN-11538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiandan Yang updated YARN-11538:
-
Attachment: (was: YARN-11538.patch)
> CS UI: queue filter do not work as expected when submitting apps with leaf
> queue‘s name
>
>
> Key: YARN-11538
> URL: https://issues.apache.org/jira/browse/YARN-11538
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Jiandan Yang
>Assignee: Jiandan Yang
>Priority: Major
> Attachments: YARN-11538.patch
>
>
> I have noticed that the application can not be filtered correctly in the
> CapacityScheduer page when I submit apps with leaf queue‘s name.
> *Steps to Reproduce:*
> 1. create a mr app with leaf queue's name without *root.*, for example:
> {code:java}
> hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
> -Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
> {code}
> 2. open the schedule web page of CapacityScheduler
> 3. click the queue of the app submited in step 1
> 4. check the apps showed in web page, and can not find the app submited in
> step 1
> *Problem Analysis:*
> when clicking a queue in scheduler page, it filters by full path of Queue in
> web page. Beause the queues of apps returned by *getApplications* is short
> path, the result of filter is empty
> *Solution:*
> getApplications returns full queue path of apps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11538) CS UI: queue filter do not work as expected when submitting apps with leaf queue‘s name
[
https://issues.apache.org/jira/browse/YARN-11538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiandan Yang updated YARN-11538:
-
Attachment: YARN-11538.patch
> CS UI: queue filter do not work as expected when submitting apps with leaf
> queue‘s name
>
>
> Key: YARN-11538
> URL: https://issues.apache.org/jira/browse/YARN-11538
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Jiandan Yang
>Assignee: Jiandan Yang
>Priority: Major
> Attachments: YARN-11538.patch
>
>
> I have noticed that the application can not be filtered correctly in the
> CapacityScheduer page when I submit apps with leaf queue‘s name.
> *Steps to Reproduce:*
> 1. create a mr app with leaf queue's name without *root.*, for example:
> {code:java}
> hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
> -Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
> {code}
> 2. open the schedule web page of CapacityScheduler
> 3. click the queue of the app submited in step 1
> 4. check the apps showed in web page, and can not find the app submited in
> step 1
> *Problem Analysis:*
> when clicking a queue in scheduler page, it filters by full path of Queue in
> web page. Beause the queues of apps returned by *getApplications* is short
> path, the result of filter is empty
> *Solution:*
> getApplications returns full queue path of apps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11538) CS UI: queue filter do not work as expected when submitting apps with leaf queue‘s name
[
https://issues.apache.org/jira/browse/YARN-11538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiandan Yang updated YARN-11538:
-
Description:
I have noticed that the application can not be filtered correctly in the
CapacityScheduer page when I submit apps with leaf queue‘s name.
*Steps to Reproduce:*
1. create a mr app with leaf queue's name without *root.*, for example:
{code:java}
hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
-Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
{code}
2. open the schedule web page of CapacityScheduler
3. click the queue of the app submited in step 1
4. check the apps showed in web page, and can not find the app submited in step
1
*Problem Analysis:*
when clicking a queue in scheduler page, it filters by full path of Queue in
web page. Beause the queues of apps returned by *getApplications* is short
path, the result of filter is empty
*Solution:*
getApplications returns full queue path of apps.
was:
I have noticed that the application can not be filtered correctly in the
CapacityScheduer page when I submit apps with leaf queue‘s name.
*Steps to Reproduce:*
1. create a mr app with leaf queue's name without *root.*, for example:
{code:java}
hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
-Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
{code}
2. open the schedule web page of CapacityScheduler
3. click the queue of the app submited in step 1
4. check the apps showed in web page, and can not find the app submited in step
1
*Problem Analysis:*
when clicking a queue in scheduler page, it filters by full path of Queue in
web page. Beause the queues of apps returned by *getApplications* is short
path, the result of filter is empty
*Solution:*
*getApplications* returns full queue path of apps.
> CS UI: queue filter do not work as expected when submitting apps with leaf
> queue‘s name
>
>
> Key: YARN-11538
> URL: https://issues.apache.org/jira/browse/YARN-11538
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Jiandan Yang
>Assignee: Jiandan Yang
>Priority: Major
>
> I have noticed that the application can not be filtered correctly in the
> CapacityScheduer page when I submit apps with leaf queue‘s name.
> *Steps to Reproduce:*
> 1. create a mr app with leaf queue's name without *root.*, for example:
> {code:java}
> hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
> -Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
> {code}
> 2. open the schedule web page of CapacityScheduler
> 3. click the queue of the app submited in step 1
> 4. check the apps showed in web page, and can not find the app submited in
> step 1
> *Problem Analysis:*
> when clicking a queue in scheduler page, it filters by full path of Queue in
> web page. Beause the queues of apps returned by *getApplications* is short
> path, the result of filter is empty
> *Solution:*
> getApplications returns full queue path of apps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11538) CS UI: queue filter do not work as expected when submitting apps with leaf queue‘s name
[
https://issues.apache.org/jira/browse/YARN-11538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiandan Yang updated YARN-11538:
-
Description:
I have noticed that the application can not be filtered correctly in the
CapacityScheduer page when I submit apps with leaf queue‘s name.
*Steps to Reproduce:*
1. create a mr app with leaf queue's name without *root.*, for example:
{code:java}
hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
-Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
{code}
2. open the schedule web page of CapacityScheduler
3. click the queue of the app submited in step 1
4. check the apps showed in web page, and can not find the app submited in step
1
*Problem Analysis:*
when clicking a queue in scheduler page, it filters by full path of Queue in
web page. Beause the queues of apps returned by *getApplications* is short
path, the result of filter is empty
*Solution:*
*getApplications* returns full queue path of apps.
was:
I have noticed that the application can not be filtered correctly in the
CapacityScheduer page when I submit apps with leaf queue‘s name.
*Steps to Reproduce:*
1. create a mr app with leaf queue's name without *root.*, for example:
{code:java}
hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
-Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
{code}
2. open the schedule web page of CapacityScheduler
3. click the queue of the app submited in step 1
4. check the apps showed in web page, and can not find the app submited in step
1
*Problem Analysis:*
1.
> CS UI: queue filter do not work as expected when submitting apps with leaf
> queue‘s name
>
>
> Key: YARN-11538
> URL: https://issues.apache.org/jira/browse/YARN-11538
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Jiandan Yang
>Assignee: Jiandan Yang
>Priority: Major
>
> I have noticed that the application can not be filtered correctly in the
> CapacityScheduer page when I submit apps with leaf queue‘s name.
> *Steps to Reproduce:*
> 1. create a mr app with leaf queue's name without *root.*, for example:
> {code:java}
> hadoop jar hadoop-mapreduce-examples-3.3.4.jar teragen
> -Dmapreduce.job.queuename=ia_ana 100 /user/yjd/teragen
> {code}
> 2. open the schedule web page of CapacityScheduler
> 3. click the queue of the app submited in step 1
> 4. check the apps showed in web page, and can not find the app submited in
> step 1
> *Problem Analysis:*
> when clicking a queue in scheduler page, it filters by full path of Queue in
> web page. Beause the queues of apps returned by *getApplications* is short
> path, the result of filter is empty
> *Solution:*
> *getApplications* returns full queue path of apps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11535) Snakeyaml should be excluded from jackson-dataformat-yaml-2.12.7 as it may cause transitive dependency issue.
[
https://issues.apache.org/jira/browse/YARN-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746644#comment-17746644
]
ASF GitHub Bot commented on YARN-11535:
---
hadoop-yetus commented on PR #5884:
URL: https://github.com/apache/hadoop/pull/5884#issuecomment-1648589852
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 20s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | shadedclient | 58m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | shadedclient | 22m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 85m 28s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 172m 44s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5884/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5884 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 8dc9fb39b73a 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / de77b035c8fc232696d07dba82df9819e6696803 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5884/1/testReport/ |
| Max. process+thread count | 963 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5884/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (YARN-11520) Support capacity vector for AQCv2 dynamic templates
[
https://issues.apache.org/jira/browse/YARN-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746600#comment-17746600
]
ASF GitHub Bot commented on YARN-11520:
---
hadoop-yetus commented on PR #5871:
URL: https://github.com/apache/hadoop/pull/5871#issuecomment-1648437872
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 9s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | jsonlint | 0m 1s | | jsonlint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 17s | | trunk passed |
| +1 :green_heart: | compile | 1m 12s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 14s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 35m 36s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 54s | | the patch passed |
| +1 :green_heart: | compile | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 0m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 46s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5871/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 29 new + 38 unchanged - 0 fixed = 67 total (was 38) |
| +1 :green_heart: | mvnsite | 0m 54s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 44s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 37s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 106m 33s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 242m 34s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5871/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5871 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jsonlint |
| uname | Linux 2504f49c90a3 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 70afb1f757d60d6bda2806565764d7a9ee966dcf |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK
[jira] [Commented] (YARN-11520) Support capacity vector for AQCv2 dynamic templates
[
https://issues.apache.org/jira/browse/YARN-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746601#comment-17746601
]
ASF GitHub Bot commented on YARN-11520:
---
hadoop-yetus commented on PR #5871:
URL: https://github.com/apache/hadoop/pull/5871#issuecomment-1648439396
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | jsonlint | 0m 0s | | jsonlint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 4s | | trunk passed |
| +1 :green_heart: | compile | 1m 10s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 58s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 0m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 30s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 35m 54s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 58s | | the patch passed |
| +1 :green_heart: | compile | 0m 48s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 0m 48s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 47s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5871/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 29 new + 38 unchanged - 0 fixed = 67 total (was 38) |
| +1 :green_heart: | mvnsite | 0m 53s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 43s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 106m 21s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 245m 14s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5871/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5871 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jsonlint |
| uname | Linux ba4d01778689 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 70afb1f757d60d6bda2806565764d7a9ee966dcf |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK
[jira] [Updated] (YARN-11535) Snakeyaml should be excluded from jackson-dataformat-yaml-2.12.7 as it may cause transitive dependency issue.
[ https://issues.apache.org/jira/browse/YARN-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11535: -- Labels: pull-request-available (was: ) > Snakeyaml should be excluded from jackson-dataformat-yaml-2.12.7 as it may > cause transitive dependency issue. > - > > Key: YARN-11535 > URL: https://issues.apache.org/jira/browse/YARN-11535 > Project: Hadoop YARN > Issue Type: Task > Components: build, yarn >Affects Versions: 3.4.0 >Reporter: Susheel Gupta >Assignee: Susheel Gupta >Priority: Major > Labels: pull-request-available > > Hadoop-project uses > [snakeyaml.version-2.0|https://github.com/apache/hadoop/blame/trunk/hadoop-project/pom.xml#L198] > and > [jackson-dataformat-yaml-2.12.7|https://github.com/apache/hadoop/blob/trunk/hadoop-project/pom.xml#L72]. > But internally jackson-dataformat-yaml-2.12.7 uses compile dependency > [snakeyaml.version-1.27|https://mvnrepository.com/artifact/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.12.7] > .This may cause a transitive dependency issue in other services using hadoop > jar having jackson-dataformat-yaml-2.12.7 as jackson-dataformat-yaml-2.12.7 > will use nearest dependency available of snakeyaml i.e 1.27 and ignore the > version of snakeyaml-2.0 from hadoop-project. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11535) Snakeyaml should be excluded from jackson-dataformat-yaml-2.12.7 as it may cause transitive dependency issue.
[ https://issues.apache.org/jira/browse/YARN-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746567#comment-17746567 ] ASF GitHub Bot commented on YARN-11535: --- susheelgupta7 opened a new pull request, #5884: URL: https://github.com/apache/hadoop/pull/5884 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Snakeyaml should be excluded from jackson-dataformat-yaml-2.12.7 as it may > cause transitive dependency issue. > - > > Key: YARN-11535 > URL: https://issues.apache.org/jira/browse/YARN-11535 > Project: Hadoop YARN > Issue Type: Task > Components: build, yarn >Affects Versions: 3.4.0 >Reporter: Susheel Gupta >Assignee: Susheel Gupta >Priority: Major > > Hadoop-project uses > [snakeyaml.version-2.0|https://github.com/apache/hadoop/blame/trunk/hadoop-project/pom.xml#L198] > and > [jackson-dataformat-yaml-2.12.7|https://github.com/apache/hadoop/blob/trunk/hadoop-project/pom.xml#L72]. > But internally jackson-dataformat-yaml-2.12.7 uses compile dependency > [snakeyaml.version-1.27|https://mvnrepository.com/artifact/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml/2.12.7] > .This may cause a transitive dependency issue in other services using hadoop > jar having jackson-dataformat-yaml-2.12.7 as jackson-dataformat-yaml-2.12.7 > will use nearest dependency available of snakeyaml i.e 1.27 and ignore the > version of snakeyaml-2.0 from hadoop-project. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11541) AsyncDispatcher causes ArithmeticException due to improper detailsInterval value checking
ConfX created YARN-11541:
Summary: AsyncDispatcher causes ArithmeticException due to
improper detailsInterval value checking
Key: YARN-11541
URL: https://issues.apache.org/jira/browse/YARN-11541
Project: Hadoop YARN
Issue Type: Bug
Reporter: ConfX
Attachments: reproduce.sh
h2. What happened
There is no value checking for parameter
{{{}yarn.dispatcher.print-events-info.threshold{}}}. This may cause improper
calculations and crashes the system like division by 0.
h2. Buggy code
In {{{}AsyncDispatcher.java{}}}, there is no value checking for
{{detailsInterval}} and this variable is directly used in the event handler.
When {{detailsInterval}} is mistakenly set to 0, the code would cause division
by 0 and throw ArithmeticException to crash the system.
```
public void handle(Event event) {
. . .
if (qSize != 0 && qSize % detailsInterval == 0 //
detailsInterval gets its value from the aforementioned config param
&& lastEventDetailsQueueSizeLogged != qSize) {
. . .
```
h2. How to reproduce
(1) set yarn.dispatcher.print-events-info.threshold=0
(2) run
org.apache.hadoop.yarn.event.TestAsyncDispatcher#testDispatcherMetricsHistogram
you should observe the following failure:
```
java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:348)
at
org.apache.hadoop.yarn.event.TestAsyncDispatcher.testDispatcherMetricsHistogram(TestAsyncDispatcher.java:370)
...
```
For an easy reproduction, run the reproduce.sh in the attachment.
We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11541) AsyncDispatcher causes ArithmeticException due to improper detailsInterval value checking
[
https://issues.apache.org/jira/browse/YARN-11541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ConfX updated YARN-11541:
-
Description:
h2. What happened
There is no value checking for parameter
{{{}yarn.dispatcher.print-events-info.threshold{}}}. This may cause improper
calculations and crashes the system like division by 0.
h2. Buggy code
In {{{}AsyncDispatcher.java{}}}, there is no value checking for
{{detailsInterval}} and this variable is directly used in the event handler.
When {{detailsInterval}} is mistakenly set to 0, the code would cause division
by 0 and throw ArithmeticException to crash the system.
{noformat}
public void handle(Event event) {
. . .
if (qSize != 0 && qSize % detailsInterval == 0 //
detailsInterval gets its value from the aforementioned config param
&& lastEventDetailsQueueSizeLogged != qSize) {
. . .{noformat}
h2. How to reproduce
(1) set yarn.dispatcher.print-events-info.threshold=0
(2) run
org.apache.hadoop.yarn.event.TestAsyncDispatcher#testDispatcherMetricsHistogram
you should observe the following failure:
{noformat}
java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:348)
at
org.apache.hadoop.yarn.event.TestAsyncDispatcher.testDispatcherMetricsHistogram(TestAsyncDispatcher.java:370)
...{noformat}
For an easy reproduction, run the reproduce.sh in the attachment.
We are happy to provide a patch if this issue is confirmed.
was:
h2. What happened
There is no value checking for parameter
{{{}yarn.dispatcher.print-events-info.threshold{}}}. This may cause improper
calculations and crashes the system like division by 0.
h2. Buggy code
In {{{}AsyncDispatcher.java{}}}, there is no value checking for
{{detailsInterval}} and this variable is directly used in the event handler.
When {{detailsInterval}} is mistakenly set to 0, the code would cause division
by 0 and throw ArithmeticException to crash the system.
```
public void handle(Event event) {
. . .
if (qSize != 0 && qSize % detailsInterval == 0 //
detailsInterval gets its value from the aforementioned config param
&& lastEventDetailsQueueSizeLogged != qSize) {
. . .
```
h2. How to reproduce
(1) set yarn.dispatcher.print-events-info.threshold=0
(2) run
org.apache.hadoop.yarn.event.TestAsyncDispatcher#testDispatcherMetricsHistogram
you should observe the following failure:
```
java.lang.ArithmeticException: / by zero
at
org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:348)
at
org.apache.hadoop.yarn.event.TestAsyncDispatcher.testDispatcherMetricsHistogram(TestAsyncDispatcher.java:370)
...
```
For an easy reproduction, run the reproduce.sh in the attachment.
We are happy to provide a patch if this issue is confirmed.
> AsyncDispatcher causes ArithmeticException due to improper detailsInterval
> value checking
> -
>
> Key: YARN-11541
> URL: https://issues.apache.org/jira/browse/YARN-11541
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: ConfX
>Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened
> There is no value checking for parameter
> {{{}yarn.dispatcher.print-events-info.threshold{}}}. This may cause improper
> calculations and crashes the system like division by 0.
> h2. Buggy code
> In {{{}AsyncDispatcher.java{}}}, there is no value checking for
> {{detailsInterval}} and this variable is directly used in the event handler.
> When {{detailsInterval}} is mistakenly set to 0, the code would cause
> division by 0 and throw ArithmeticException to crash the system.
> {noformat}
> public void handle(Event event) {
> . . .
> if (qSize != 0 && qSize % detailsInterval == 0 //
> detailsInterval gets its value from the aforementioned config param
> && lastEventDetailsQueueSizeLogged != qSize) {
> . . .{noformat}
> h2. How to reproduce
> (1) set yarn.dispatcher.print-events-info.threshold=0
> (2) run
> org.apache.hadoop.yarn.event.TestAsyncDispatcher#testDispatcherMetricsHistogram
> you should observe the following failure:
> {noformat}
> java.lang.ArithmeticException: / by zero
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:348)
> at
> org.apache.hadoop.yarn.event.TestAsyncDispatcher.testDispatcherMetricsHistogram(TestAsyncDispatcher.java:370)
> ...{noformat}
> For an easy reproduction, run the reproduce.sh in the attachment.
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (YARN-11522) Update documentation for the New capacity modes in CS
[ https://issues.apache.org/jira/browse/YARN-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746495#comment-17746495 ] ASF GitHub Bot commented on YARN-11522: --- brumi1024 commented on code in PR #5870: URL: https://github.com/apache/hadoop/pull/5870#discussion_r1272376852 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md: ## @@ -64,8 +64,14 @@ The `CapacityScheduler` supports the following features: * **Priority Scheduling** - This feature allows applications to be submitted and scheduled with different priorities. Higher integer value indicates higher priority for an application. Currently Application priority is supported only for FIFO ordering policy. +* **Percentage Resource Configuration** - Administrators could specify percentages of resources to a queue. + * **Absolute Resource Configuration** - Administrators could specify absolute resources to a queue instead of providing percentage based values. This provides better control for admins to configure required amount of resources for a given queue. +* **Weight Resource Configuration** - Administrators could specify weights to a queue instead of providing percentage based values. This provides better control for admins to configure resources for the queue in a dynamically changing queue hierarchy. + +* **Universal Capacity Vector Resource Configuration** - Administrators could specify resources in a mixed manner - combining absolute, weight or percentage - to a queue instead of providing percentage based values. This provides the most flexible way of control for admins to configure required amount of resources for a given queue. Review Comment: Nit: "combining absolute, weight or percentage" I would add values to the end, and drop the rest. For me in this form it sounds like this is an alternative to the percentage description, but it's a combination of all three before it. This provides the most flexible way **of control for admins** to configure required amount of resources for a given queue. -> I don't think "of control for admins" is needed here, without it sound cleaner. ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md: ## @@ -123,33 +129,77 @@ Configuration | Property | Description | |:-- > Update documentation for the New capacity modes in CS > - > > Key: YARN-11522 > URL: https://issues.apache.org/jira/browse/YARN-11522 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 3.4.0 >Reporter: Tamas Domok >Assignee: Tamas Domok >Priority: Major > Labels: pull-request-available > > Follow up ticket for YARN-11000, doc the changes mentioned in > (https://github.com/apache/hadoop/pull/5783). > Candidates: > - CapacityScheduler.md > - ResourceManagerRest.md > Also document this: > https://github.com/apache/hadoop/pull/5852#pullrequestreview-1537039656 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11521) Create a test set that runs with both Legacy/Uniform queue calculation
[
https://issues.apache.org/jira/browse/YARN-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746487#comment-17746487
]
ASF GitHub Bot commented on YARN-11521:
---
hadoop-yetus commented on PR #5883:
URL: https://github.com/apache/hadoop/pull/5883#issuecomment-1648055527
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | jsonlint | 0m 1s | | jsonlint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 20 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 50m 13s | | trunk passed |
| +1 :green_heart: | compile | 1m 6s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 54s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 0m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 47s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 41m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 56s | | the patch passed |
| +1 :green_heart: | compile | 0m 46s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 0m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 45s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5883/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 10 new + 29 unchanged - 0 fixed = 39 total (was 29) |
| +1 :green_heart: | mvnsite | 0m 49s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 44s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 40s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 119m 26s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 269m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5883/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5883 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets jsonlint |
| uname | Linux 33c3847cacc5 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4bce89632f317d4de4d99b47de95024651aee44e |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
[jira] [Commented] (YARN-11521) Create a test set that runs with both Legacy/Uniform queue calculation
[ https://issues.apache.org/jira/browse/YARN-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746326#comment-17746326 ] ASF GitHub Bot commented on YARN-11521: --- tomicooler opened a new pull request, #5883: URL: https://github.com/apache/hadoop/pull/5883 ### Description of PR NOTE: I cherry-picked YARN-11539, will rebase later when that gets merged to trunk. ### How was this patch tested? Tested locally, Yetus will verify it. ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'YARN-11521. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Create a test set that runs with both Legacy/Uniform queue calculation > -- > > Key: YARN-11521 > URL: https://issues.apache.org/jira/browse/YARN-11521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 3.4.0 >Reporter: Tamas Domok >Assignee: Tamas Domok >Priority: Major > > Follow-up ticket for YARN-11000. > The JSON assert tests in TestRMWebServicesCapacitySchedDynamicConfig are a > good candidate for this. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11521) Create a test set that runs with both Legacy/Uniform queue calculation
[ https://issues.apache.org/jira/browse/YARN-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YARN-11521: -- Labels: pull-request-available (was: ) > Create a test set that runs with both Legacy/Uniform queue calculation > -- > > Key: YARN-11521 > URL: https://issues.apache.org/jira/browse/YARN-11521 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacityscheduler >Affects Versions: 3.4.0 >Reporter: Tamas Domok >Assignee: Tamas Domok >Priority: Major > Labels: pull-request-available > > Follow-up ticket for YARN-11000. > The JSON assert tests in TestRMWebServicesCapacitySchedDynamicConfig are a > good candidate for this. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11536) [Federation] Router CLI Supports Batch Save the SubClusterPolicyConfiguration Of Queues.
[
https://issues.apache.org/jira/browse/YARN-11536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746314#comment-17746314
]
ASF GitHub Bot commented on YARN-11536:
---
hadoop-yetus commented on PR #5862:
URL: https://github.com/apache/hadoop/pull/5862#issuecomment-1647573283
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 36m 11s | | trunk passed |
| +1 :green_heart: | compile | 8m 5s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 7m 18s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 34s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 5m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 41s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 38s | | the patch passed |
| +1 :green_heart: | compile | 7m 16s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | cc | 7m 16s | | the patch passed |
| +1 :green_heart: | javac | 7m 16s | | the patch passed |
| +1 :green_heart: | compile | 7m 16s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | cc | 7m 16s | | the patch passed |
| +1 :green_heart: | javac | 7m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 51s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5862/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 3 new + 0 unchanged -
0 fixed = 3 total (was 0) |
| +1 :green_heart: | mvnsite | 2m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 9s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 5m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 56s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 8s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 31s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 28m 4s | | hadoop-yarn-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 234m 2s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5862/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5862 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint
bufcompat xmllint |
| uname | Linux 7ed40087f4fb 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64
[jira] [Commented] (YARN-11520) Support capacity vector for AQCv2 dynamic templates
[
https://issues.apache.org/jira/browse/YARN-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746272#comment-17746272
]
ASF GitHub Bot commented on YARN-11520:
---
tomicooler commented on code in PR #5871:
URL: https://github.com/apache/hadoop/pull/5871#discussion_r1271264069
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java:
##
@@ -463,12 +463,14 @@ protected void setDynamicQueueProperties() {
queueContext.getQueueManager().getConfiguredNodeLabelsForAllQueues()
.setLabelsByQueue(getQueuePath(), new
HashSet<>(parentNodeLabels));
}
+Map templateConfiguredResourceVector =
queueContext.getConfiguration()
+.parseConfiguredResourceVector(queuePath.getFullPath(),
parentNodeLabels);
+
Review Comment:
I'm missing the part where the configuredCapacityVector (and
configuredMaxCapacityVector) is updated with this information. Are those
already have the correctly parsed capacity vectors at the setupQueueConfigs
phase, if so there is no need for parsing them again.
Anyway the field member configurecCapacityVector should have the correct
value set somewhere. Some e2e test with JSON asserts should verify it later.
Maybe this new UT could also assert it.
> Support capacity vector for AQCv2 dynamic templates
> ---
>
> Key: YARN-11520
> URL: https://issues.apache.org/jira/browse/YARN-11520
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
>Affects Versions: 3.4.0
>Reporter: Tamas Domok
>Assignee: Benjamin Teke
>Priority: Major
> Labels: pull-request-available
>
> AQCv2 dynamic queue templates should support the new capacity modes.
> e.g.:
> {code}
> auto-queue-creation-v2.parent-template.capacity = [memory=12288, vcores=86%]
> auto-queue-creation-v2.leaf-template.capacity = [memory=1w, vcores=1]
> auto-queue-creation-v2.template.capacity = [memory=20%, vcores=50%]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10975) EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
[
https://issues.apache.org/jira/browse/YARN-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746217#comment-17746217
]
Rakesh Radhakrishnan commented on YARN-10975:
-
[~prabhujoseph] I have updated the fix version to 3.4.0 as this is merged only
to the trunk.
> EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
> --
>
> Key: YARN-10975
> URL: https://issues.apache.org/jira/browse/YARN-10975
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Ravuri Sushma sree
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: YARN-10975.001.patch, YARN-10975.002.patch,
> YARN-10975.003.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
> again and again even though there is no change in the file. This leads to
> unnecessary load on DFS where summary files resides and Timeline Store where
> timeline entities present.
> {code}
> 2021-10-10 19:20:43,940 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 275 msec
> 2021-10-10 19:21:44,079 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 341 msec
> 2021-10-10 19:22:44,065 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 335 msec
> 2021-10-10 19:23:44,038 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 370 msec
> 2021-10-10 19:24:44,087 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 317 msec
> 2021-10-10 19:25:44,092 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 336 msec
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10975) EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
[
https://issues.apache.org/jira/browse/YARN-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rakesh Radhakrishnan updated YARN-10975:
Fix Version/s: 3.4.0
(was: 3.3.2)
> EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
> --
>
> Key: YARN-10975
> URL: https://issues.apache.org/jira/browse/YARN-10975
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Ravuri Sushma sree
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: YARN-10975.001.patch, YARN-10975.002.patch,
> YARN-10975.003.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
> again and again even though there is no change in the file. This leads to
> unnecessary load on DFS where summary files resides and Timeline Store where
> timeline entities present.
> {code}
> 2021-10-10 19:20:43,940 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 275 msec
> 2021-10-10 19:21:44,079 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 341 msec
> 2021-10-10 19:22:44,065 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 335 msec
> 2021-10-10 19:23:44,038 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 370 msec
> 2021-10-10 19:24:44,087 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 317 msec
> 2021-10-10 19:25:44,092 INFO timeline.LogInfo - Parsed 6 entities from
> hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_01/summarylog-appattempt_1631559260564_0009_01_2331123893
> in 336 msec
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
