[
https://issues.apache.org/jira/browse/YARN-11801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17948597#comment-17948597
]
ASF GitHub Bot commented on YARN-11801:
---------------------------------------
hadoop-yetus commented on PR #7657:
URL: https://github.com/apache/hadoop/pull/7657#issuecomment-2843361067
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 17m 30s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
|||| _ branch-3.4.0 Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 30s | | branch-3.4.0 passed |
| +1 :green_heart: | compile | 1m 3s | | branch-3.4.0 passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | compile | 0m 51s | | branch-3.4.0 passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | checkstyle | 0m 58s | | branch-3.4.0 passed |
| +1 :green_heart: | mvnsite | 0m 56s | | branch-3.4.0 passed |
| +1 :green_heart: | javadoc | 0m 57s | | branch-3.4.0 passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 46s | | branch-3.4.0 passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 53s | | branch-3.4.0 passed |
| +1 :green_heart: | shadedclient | 37m 59s | | branch has no errors
when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 45s | | the patch passed |
| +1 :green_heart: | compile | 0m 51s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javac | 0m 51s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7657/2/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 45s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7657/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 2 new + 403 unchanged - 0 fixed = 405 total (was 403) |
| +1 :green_heart: | mvnsite | 0m 47s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| +1 :green_heart: | spotbugs | 1m 55s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 23s | | patch has no errors
when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 117m 28s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 273m 55s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.49 ServerAPI=1.49 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7657/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/7657 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3c47fbc65719 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10
21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.4.0 / 0d6ff0409c8fb2fb6eee797d96a5bcce3a82a5e3 |
| Default Java | Private Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_442-8u442-b06~us1-0ubuntu1~20.04-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7657/2/testReport/ |
| Max. process+thread count | 885 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7657/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> NPE in FifoCandidatesSelector.selectCandidates when preempting resources for
> an auto-created queue without child queues
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-11801
> URL: https://issues.apache.org/jira/browse/YARN-11801
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Affects Versions: 3.4.0, 3.5.0
> Reporter: Susheel Gupta
> Assignee: Susheel Gupta
> Priority: Major
> Labels: pull-request-available
>
> When enabling the ProportionalCapacityPreemptionPolicy in the YARN
> SchedulingMonitor, I encountered a NullPointerException in
> {{{}FifoCandidatesSelector.selectCandidates{}}}. This happens when an
> auto-created queue exists but does not have any child queues.
> A childless ParentQueue will throw a NPE in
> FifoCandidatesSelector#selectCandidates:
> {code:java}
> LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
> RMNodeLabelsManager.NO_LABEL).leafQueue;{code}
> NullPointer stack trace:
> {code:java}
> 2025-03-24 08:36:12,593 ERROR monitor.SchedulingMonitor: Exception raised
> while executing preemption checker, skip this run..., exception=
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.FifoCandidatesSelector.selectCandidates(FifoCandidatesSelector.java:104)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.containerBasedPreemptOrKill(ProportionalCapacityPreemptionPolicy.java:515)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.editSchedule(ProportionalCapacityPreemptionPolicy.java:344)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.invokePolicy(SchedulingMonitor.java:100)
> at
> org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor$PolicyInvoker.run(SchedulingMonitor.java:112)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
> Capacity-scheduler used:
> {noformat}
> <configuration>
> <property>
> <name>yarn.scheduler.capacity.mapping-rule-json</name>
> <value/>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.client.capacity</name>
> <value>50</value>
> </property>
> <property>
>
> <name>yarn.scheduler.capacity.root.client.leaf-queue-template.capacity</name>
> <value>0</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.client.maximum-capacity</name>
> <value>100</value>
> </property>
> <property>
>
> <name>yarn.scheduler.capacity.root.client.auto-create-child-queue.enabled</name>
> <value>true</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.capacity</name>
> <value>50</value>
> </property>
> <property>
>
> <name>yarn.scheduler.capacity.root.client.leaf-queue-template.maximum-capacity</name>
> <value>100</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.queues</name>
> <value>default,client</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.capacity</name>
> <value>100</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
> <value>100</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.schedule-asynchronously.enable</name>
> <value>true</value>
> </property>
> <property>
> <name>yarn.webservice.mutation-api.version</name>
> <value>1742806178771</value>
> </property>
> <property>
>
> <name>yarn.scheduler.capacity.root.default.maximum-am-resource-percent</name>
> <value>0.2</value>
> </property>
> <property>
> <name>yarn.scheduler.capacity.mapping-rule-format</name>
> <value>json</value>
> </property>
> </configuration>{noformat}
> Add this in yarn-site.xml:
>
> {code:java}
> <property>
> <name>yarn.resourcemanager.scheduler.monitor.enable</name>
> <value>true</value>
> </property>
> <property>
> <name>yarn.resourcemanager.scheduler.monitor.policies</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy,org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.AutoCreatedQueueDeletionPolicy,org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.QueueManagementDynamicEditPolicy</value>
> </property>{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
