[jira] [Commented] (YARN-6412) aux-services classpath not documented
[
https://issues.apache.org/jira/browse/YARN-6412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655654#comment-17655654
]
ASF GitHub Bot commented on YARN-6412:
--
hadoop-yetus commented on PR #5242:
URL: https://github.com/apache/hadoop/pull/5242#issuecomment-1374388947
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 5s | | trunk passed |
| +1 :green_heart: | compile | 0m 49s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 62m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 40s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 21m 49s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 30s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 95m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5242/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5242 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux c2925d05135d 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 3fb11400f645eae5224772a27f64c5a649381249 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5242/2/testReport/ |
| Max. process+thread count | 673 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5242/2/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> aux-services classpath not documented
> -
>
> Key: YARN-6412
> URL: https://iss
[jira] [Updated] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3
[
https://issues.apache.org/jira/browse/YARN-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineeth Naroju updated YARN-11355:
--
Attachment: (was: YARN-11355-proxyCount.diff)
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3
> --
>
> Key: YARN-11355
> URL: https://issues.apache.org/jira/browse/YARN-11355
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Vineeth Naroju
>Priority: Major
> Attachments: YARN-11355.diff
>
>
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during
> initial retry.
> *Repro:*
> {code:java}
> 1. YARN Cluster with three master nodes rm1,rm2 and rm3
> 2. rm3 is active
> 3. yarn node -list or any other yarn client calls takes more than 30 seconds.
> {code}
> The initial failover to rm2 is immediate but then the failover to rm3 is
> after ~3 ms. Current RetryPolicy does not honor the number of master
> nodes. It has to perform atleast one immediate failover to every rm.
> {code:java}
> 2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider:
> Failing over to rm2
> 2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler:
> java.net.ConnectException: Call From local to remote:8032 failed on
> connection exception: java.net.ConnectException: Connection refused; For more
> details see: http://wiki.apache.org/hadoop/ConnectionRefused, while invoking
> ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1
> failover attempts. Trying to failover after sleeping for 21139ms.
> {code}
>
> *Workaround:*
> Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100.
> This will do immediate failover to rm3 but there will be too many retries
> when there is no active resourcemanager.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3
[
https://issues.apache.org/jira/browse/YARN-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineeth Naroju updated YARN-11355:
--
Attachment: YARN-11355.diff
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3
> --
>
> Key: YARN-11355
> URL: https://issues.apache.org/jira/browse/YARN-11355
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Vineeth Naroju
>Priority: Major
> Attachments: YARN-11355.diff
>
>
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during
> initial retry.
> *Repro:*
> {code:java}
> 1. YARN Cluster with three master nodes rm1,rm2 and rm3
> 2. rm3 is active
> 3. yarn node -list or any other yarn client calls takes more than 30 seconds.
> {code}
> The initial failover to rm2 is immediate but then the failover to rm3 is
> after ~3 ms. Current RetryPolicy does not honor the number of master
> nodes. It has to perform atleast one immediate failover to every rm.
> {code:java}
> 2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider:
> Failing over to rm2
> 2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler:
> java.net.ConnectException: Call From local to remote:8032 failed on
> connection exception: java.net.ConnectException: Connection refused; For more
> details see: http://wiki.apache.org/hadoop/ConnectionRefused, while invoking
> ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1
> failover attempts. Trying to failover after sleeping for 21139ms.
> {code}
>
> *Workaround:*
> Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100.
> This will do immediate failover to rm3 but there will be too many retries
> when there is no active resourcemanager.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11217) [Federation] Add dumpSchedulerLogs REST APIs for Router
[
https://issues.apache.org/jira/browse/YARN-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655510#comment-17655510
]
ASF GitHub Bot commented on YARN-11217:
---
hadoop-yetus commented on PR #5272:
URL: https://github.com/apache/hadoop/pull/5272#issuecomment-1373879656
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 48s | | trunk passed |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 24s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 24s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | compile | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 54s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 32s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 92m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5272/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5272 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 2be83470ece9 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24
18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0f2471f44101c351fd9d67a3a036729c1ef18601 |
| Default Java | Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5272/3/testReport/ |
| Max. process+thread count | 739 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5272/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatica
[jira] [Updated] (YARN-11395) Resource Manager UI, cluster/appattempt/*, can not present FINAL_SAVING state
[
https://issues.apache.org/jira/browse/YARN-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-11395:
--
Description:
If an attempt is in *FINAL_SAVING* state, the
*RMAppAttemptBlock#createAttemptHeadRoomTable* method fails with a convert
error, what will results a
{code:java}
RFC6265 Cookie values may not contain character: [ ]{code}
error in the UI an in the logs as well.
RM log:
{code:java}
...
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.FINAL_SAVING
at java.lang.Enum.valueOf(Enum.java:238)
at
org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.valueOf(YarnApplicationAttemptState.java:27)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppAttemptBlock.createAttemptHeadRoomTable(RMAppAttemptBlock.java:424)
at
org.apache.hadoop.yarn.server.webapp.AppAttemptBlock.render(AppAttemptBlock.java:151)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.appattempt(RmController.java:62)
... 63 more
2022-12-05 04:15:33,029 WARN org.eclipse.jetty.server.HttpChannel:
/cluster/appattempt/appattempt_1667297151262_0247_01
java.lang.IllegalArgumentException: RFC6265 Cookie values may not contain
character: [ ]
at
org.eclipse.jetty.http.Syntax.requireValidRFC6265CookieValue(Syntax.java:136)
...{code}
This bug was introduced with the YARN-1345 ticket what also caused a similar
error called YARN-4411. In case of the YARN-4411 the enum mapping logic from
RMAppAttemptStates to YarnApplicationAttemptState was modified like this:
- if the state is FINAL_SAVING we should represent the previous state
This error can also be occur in case of ALLOCATED_SAVING,
LAUNCHED_UNMANAGED_SAVING states as well.
So we should modify the *createAttemptHeadRoomTable* method to be able to
handle the previously mentioned 3 states just like in case of YARN-4411
was:
If an attempt is in *FINAL_SAVING* state, the
*RMAppAttemptBlock#createAttemptHeadRoomTable* method fails with a convert
error, what will results a
{code:java}
RFC6265 Cookie values may not contain character: [ ]{code}
error in the UI an in the logs as well.
RM log:
{code:java}
...
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.FINAL_SAVING
at java.lang.Enum.valueOf(Enum.java:238)
at
org.apache.hadoop.yarn.api.records.YarnApplicationAttemptState.valueOf(YarnApplicationAttemptState.java:27)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppAttemptBlock.createAttemptHeadRoomTable(RMAppAttemptBlock.java:424)
at
org.apache.hadoop.yarn.server.webapp.AppAttemptBlock.render(AppAttemptBlock.java:151)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
at
org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.appattempt(RmController.java:62)
... 63 more
2022-12-05 04:15:33,029 WARN org.eclipse.jetty.server.HttpChannel:
/cluster/appattempt/appattempt_1667297151262_0247_01
java.lang.IllegalArgumentException: RFC6265 Cookie values may not contain
character: [ ]
at
org.eclipse.jetty.http.Syntax.requireValidRFC6265CookieValue(Syntax.java:136)
...{code}
This bug was introduced with the YARN-1345 ticket what also caused a similar
error called YARN-4411. In case of the YARN-4411 the enum mapping logic from
RMAppAttemptStates to YarnApplicationAttemptStat
[jira] [Commented] (YARN-6971) Clean up different ways to create resources
[
https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655491#comment-17655491
]
ASF GitHub Bot commented on YARN-6971:
--
szilard-nemeth commented on PR #5113:
URL: https://github.com/apache/hadoop/pull/5113#issuecomment-1373819355
Hi @riyakhdl ,
Can you please check the checkstyle and unit test failures?
> Clean up different ways to create resources
> ---
>
> Key: YARN-6971
> URL: https://issues.apache.org/jira/browse/YARN-6971
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager, scheduler
>Reporter: Yufei Gu
>Assignee: Riya Khandelwal
>Priority: Minor
> Labels: newbie, pull-request-available
>
> There are several ways to create a {{resource}} object, e.g.,
> BuilderUtils.newResource() and Resources.createResource(). These methods not
> only cause confusing but also performance issues, for example
> BuilderUtils.newResource() is significant slow than
> Resources.createResource().
> We could merge them some how, and replace most BuilderUtils.newResource()
> with Resources.createResource().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11408) Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method
[
https://issues.apache.org/jira/browse/YARN-11408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655484#comment-17655484
]
ASF GitHub Bot commented on YARN-11408:
---
brumi1024 commented on code in PR #5278:
URL: https://github.com/apache/hadoop/pull/5278#discussion_r1063534266
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSQueueConverter.java:
##
@@ -309,6 +311,11 @@ private void checkMaxChildCapacitySetting(FSQueue queue) {
}
}
+ private boolean checkAutoQueueCreationV2Disabled(String queueName) {
+return !Objects.equals(capacitySchedulerConfig.get(
+PREFIX + queueName + DOT + AUTO_QUEUE_CREATION_V2_ENABLED), "true");
Review Comment:
This won't return true because Object.equals only returns true if both of
the compared objects refer to the same object, String.equals compares the
values themselves. But CapacitySchedulerConfiguration already has a helper
method which parses the value to a boolean, so I suggest using that. See
isAutoQueueCreationV2Enabled.
Also if percentage mode is used when converting it'll have the legacy Auto
Queue Creation property, which too has a helper method, called
isAutoCreateChildQueueEnabled. Both of these should be checked.
> Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor
> method
>
>
> Key: YARN-11408
> URL: https://issues.apache.org/jira/browse/YARN-11408
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> It is required to add user-limit-factor to -1 only for those queues which are
> leafqueue and auto-queue-creation is disabled.
> Follow-up of YARN-11393
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11217) [Federation] Add dumpSchedulerLogs REST APIs for Router
[ https://issues.apache.org/jira/browse/YARN-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655473#comment-17655473 ] ASF GitHub Bot commented on YARN-11217: --- slfan1989 commented on code in PR #5272: URL: https://github.com/apache/hadoop/pull/5272#discussion_r1063514791 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestableFederationInterceptorREST.java: ## @@ -30,13 +30,17 @@ import org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler; import org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler; import org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration; +import org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import javax.servlet.http.HttpServletResponse; Review Comment: @pjfanning Thanks for your suggestion, I will modify the code. > [Federation] Add dumpSchedulerLogs REST APIs for Router > --- > > Key: YARN-11217 > URL: https://issues.apache.org/jira/browse/YARN-11217 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.4.0, 3.3.4 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11355) YARN Client Failovers immediately to rm2 but takes ~30000ms to rm3
[
https://issues.apache.org/jira/browse/YARN-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineeth Naroju updated YARN-11355:
--
Attachment: YARN-11355-proxyCount.diff
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3
> --
>
> Key: YARN-11355
> URL: https://issues.apache.org/jira/browse/YARN-11355
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Vineeth Naroju
>Priority: Major
> Attachments: YARN-11355-proxyCount.diff
>
>
> YARN Client Failovers immediately to rm2 but takes ~3ms to rm3 during
> initial retry.
> *Repro:*
> {code:java}
> 1. YARN Cluster with three master nodes rm1,rm2 and rm3
> 2. rm3 is active
> 3. yarn node -list or any other yarn client calls takes more than 30 seconds.
> {code}
> The initial failover to rm2 is immediate but then the failover to rm3 is
> after ~3 ms. Current RetryPolicy does not honor the number of master
> nodes. It has to perform atleast one immediate failover to every rm.
> {code:java}
> 2022-10-20 06:37:44,123 INFO client.ConfiguredRMFailoverProxyProvider:
> Failing over to rm2
> 2022-10-20 06:37:44,129 INFO retry.RetryInvocationHandler:
> java.net.ConnectException: Call From local to remote:8032 failed on
> connection exception: java.net.ConnectException: Connection refused; For more
> details see: http://wiki.apache.org/hadoop/ConnectionRefused, while invoking
> ApplicationClientProtocolPBClientImpl.getClusterNodes over rm2 after 1
> failover attempts. Trying to failover after sleeping for 21139ms.
> {code}
>
> *Workaround:*
> Reduce yarn.resourcemanager.connect.retry-interval.ms from 3 to like 100.
> This will do immediate failover to rm3 but there will be too many retries
> when there is no active resourcemanager.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11178) Avoid CPU busy idling and resource wasting in DelegationTokenRenewerPoolTracker thread
[
https://issues.apache.org/jira/browse/YARN-11178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655368#comment-17655368
]
ASF GitHub Bot commented on YARN-11178:
---
hadoop-yetus commented on PR #4435:
URL: https://github.com/apache/hadoop/pull/4435#issuecomment-1373425771
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 8s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 8s | | trunk passed |
| +1 :green_heart: | compile | 10m 5s | | trunk passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | compile | 9m 16s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 13s | | trunk passed |
| -1 :x: | javadoc | 1m 1s |
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4435/4/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in trunk failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 2m 35s | | trunk passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 6m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 19s | | the patch passed |
| +1 :green_heart: | compile | 9m 39s | | the patch passed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 |
| +1 :green_heart: | javac | 9m 39s | | the patch passed |
| +1 :green_heart: | compile | 8m 35s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 8m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 35s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 56s | | the patch passed |
| -1 :x: | javadoc | 0m 56s |
[/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4435/4/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt)
| hadoop-yarn-server-resourcemanager in the patch failed with JDK
Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04. |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK
Private Build-1.8.0_352-8u352-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 6m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 12s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 43s | | hadoop-yarn-common in the patch
passed. |
| -1 :x: | unit | 102m 26s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4435/4/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicens
