subject:"\[jira\] \[Commented\] \(YARN\-11582\) Improve WebUI diagnosticMessage to show AM Container resource request size"

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-31 Thread wangzhihui (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832700#comment-17832700
 ] 

wangzhihui commented on YARN-11582:
---

hi, [~hexiaoqiao] .   We have submitted a simple optimization for 
ReosurceManager.  Please help review and merge it.

> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, 
> image-2024-03-28-22-11-37-903.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-28 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831854#comment-17831854
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hiwangzhihui commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1542224470


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,79 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue a1 for label X can only support 2Gb AM resource.
+ * Queue c1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify at least one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+ TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue a1
+// as 0.2 (Label X) and for Queue c1 as 0.2 (Empty Label).
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config.
+MockRM rm = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm.getRMContext().setNodeLabelManager(mgr);
+rm.start();
+rm.registerNode("h1:1234", 10 * GB); // label = x
+rm.registerNode("h2:1234", 10 * GB); // label = y
+rm.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue a1 for label X
+MockRMAppSubmissionData data1 =
+ MockRMAppSubmissionData.Builder.createWithMemory(GB, rm)
+ .withAppName("app")
+ .withUser("user")
+ .withAcls(null)
+ .withQueue("a1")
+ .withAmLabel("x")
+ .build();
+RMApp app1 = MockRMAppSubmitter.submit(rm, data1);
+
+// Submit app2 with 1Gb AM resource to Queue a1 for label X
+MockRMAppSubmissionData data2 =
+ MockRMAppSubmissionData.Builder.createWithMemory(GB, rm)
+ .withAppName("app")
+ .withUser("user")
+ .withAcls(null)
+ .withQueue("a1")
+ .withAmLabel("x")
+ .build();
+RMApp app2 = MockRMAppSubmitter.submit(rm, data2);
+
+CapacityScheduler cs = (CapacityScheduler) rm.getResourceScheduler();
+LeafQueue leafQueue = (LeafQueue) cs.getQueue("a1");
+Assert.assertNotNull(leafQueue);
+
+// Only one AM will be activated here, and second AM will be still pending.
+Assert.assertEquals(2, leafQueue.getNumActiveApplications());
+String activatedDiagnostics="AM Resource Request = ";
+Assert.assertTrue("still doesn't show AMResource When Activated", 
app1.getDiagnostics()
+ .toString().contains(activatedDiagnostics));

Review Comment:
   Add a test checks for Am resource prompts would be better





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, 
> image-2024-03-28-22-11-37-903.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-28 Thread wangzhihui (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831827#comment-17831827
 ] 

wangzhihui commented on YARN-11582:
---

hi, [~slfan1989] This  [PR|https://github.com/apache/hadoop/pull/6139] has 
added valid Test content and passed the latest Jenkins check; please help merge 
it. Thanks!
 

> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, 
> image-2024-03-28-22-11-37-903.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-28 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831676#comment-17831676
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-2024708052

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 21s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  32m 24s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 36s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  compile  |   0m 32s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  checkstyle  |   0m 33s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 37s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m 11s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 19s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 26s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 26s |  |  the patch passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javac  |   0m 26s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 24s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  javac  |   0m 24s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 22s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 25s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 22s |  |  the patch passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 24s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m  6s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m  3s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  89m 20s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 24s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 172m 59s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.45 ServerAPI=1.45 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/13/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 64f3cc57e612 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / dca8ab0eade23a70756077c5e60ce865237cf340 |
   | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/13/testReport/ |
   | Max. process+thread count | 948 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/13/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-27 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831627#comment-17831627
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-2024431543

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 20s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  32m 40s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 35s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  compile  |   0m 31s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  checkstyle  |   0m 33s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 35s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m 12s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 21s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 27s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 28s |  |  the patch passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javac  |   0m 28s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 25s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  javac  |   0m 25s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   0m 22s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/12/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 2 new + 29 unchanged - 0 fixed = 31 total (was 29)  |
   | +1 :green_heart: |  mvnsite  |   0m 27s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 25s |  |  the patch passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 26s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m  6s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m  9s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  89m  7s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/12/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 25s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 173m 25s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | 
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy |
   |   | 
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition
 |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.45 ServerAPI=1.45 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/12/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 19ef12641e21 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-27 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831488#comment-17831488
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-2023438781

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 22s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  36m 16s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 36s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  compile  |   0m 33s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  checkstyle  |   0m 37s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 40s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 33s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m 22s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  24m 32s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 30s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 34s |  |  the patch passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javac  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 33s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  javac  |   0m 33s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 28s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 29s |  |  the patch passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 23s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m 17s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  22m 23s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  89m 20s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 25s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 185m 14s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.45 ServerAPI=1.45 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/11/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux d25913424258 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / d0015be4cf3cbd73f8c7a85ba67442a2b36e8bbc |
   | Default Java | Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/11/testReport/ |
   | Max. process+thread count | 951 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/11/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-27 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831387#comment-17831387
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-2022950080

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 22s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  37m  4s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 37s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  compile  |   0m 34s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  checkstyle  |   0m 35s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 41s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 35s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 29s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m  9s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m 46s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 27s |  |  the patch passed  |
   | -1 :x: |  compile  |   0m  6s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/10/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.  |
   | -1 :x: |  javac  |   0m  6s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/10/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.  |
   | +1 :green_heart: |  compile  |   0m 25s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  javac  |   0m 25s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 26s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 28s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 29s |  |  the patch passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 27s |  |  the patch passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m 15s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  25m 16s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  88m 53s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | -1 :x: |  asflicense  |   0m 26s | 
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/10/artifact/out/results-asflicense.txt)
 |  The patch generated 1 ASF License warnings.  |
   |  |   | 188m 48s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.45 ServerAPI=1.45 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/10/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux a1e82d469702 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 
15:25:40 UTC

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-27 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831269#comment-17831269
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-2022373151

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 23s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  38m 23s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 32s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  compile  |   0m 30s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  checkstyle  |   0m 32s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 32s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 35s |  |  trunk passed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1  |
   | +1 :green_heart: |  javadoc  |   0m 28s |  |  trunk passed with JDK 
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06  |
   | +1 :green_heart: |  spotbugs  |   1m  9s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m  1s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | -1 :x: |  mvninstall  |   0m 25s | 
[/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/9/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed.  |
   | -1 :x: |  compile  |   0m 29s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/9/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.  |
   | -1 :x: |  javac  |   0m 29s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/9/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK 
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1.  |
   | -1 :x: |  compile  |   0m 26s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/9/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK Private 
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.  |
   | -1 :x: |  javac  |   0m 26s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/9/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK Private 
Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06.  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 23s |  |  the patch passed  |
   | -1 :x: |  mvnsite  |   0m 25s |

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2024-03-27 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831205#comment-17831205
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-2022168620

   @hiwangzhihui Hi~, please help me review it.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-11-04 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782937#comment-17782937
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1793583818

   @xiaojunxiang2023 Can we trigger jenkins compilation again?




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-21 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778200#comment-17778200
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1773897417

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 37s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  46m  5s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  0s |  |  trunk passed with JDK 
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 57s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 54s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 58s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 56s |  |  trunk passed with JDK 
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 51s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   2m  1s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  34m 25s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 47s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 53s |  |  the patch passed with JDK 
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 53s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 48s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 48s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 44s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  |  the patch passed with JDK 
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 40s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 53s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  34m 19s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  99m 38s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 35s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 233m 26s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6139/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 6742394b118d 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / d61d4ce4fb6e0465cea2697844e435fa9ec59154 |
   | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6139/1/testReport/
 |
   | Max. process+thread count | 982 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch-windows-10/job/PR-6139/1/console
 |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-06 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772635#comment-17772635
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1751040090

   > @xiaojunxiang2023 We need to fix the checkstyle issue.
   
   Hi，The latest PR has completely passed the checkstyle. Please review again 
if you are free.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-06 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772633#comment-17772633
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1751029016

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 29s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  39m  5s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 47s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 40s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 42s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 44s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 33s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  28m 27s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 38s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 38s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 33s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 33s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 30s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 36s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 33s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 31s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 20s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  26m 21s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  86m 44s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 30s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 194m 48s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/8/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux cb88f513534a 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / d61d4ce4fb6e0465cea2697844e435fa9ec59154 |
   | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/8/testReport/ |
   | Max. process+thread count | 963 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/8/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-06 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772454#comment-17772454
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1750072658

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 31s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | -1 :x: |  mvninstall  |   0m 24s | 
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/branch-mvninstall-root.txt)
 |  root in trunk failed.  |
   | -1 :x: |  compile  |   0m 24s | 
[/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.  |
   | -1 :x: |  compile  |   0m 24s | 
[/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05.  |
   | -0 :warning: |  checkstyle  |   0m 22s | 
[/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  The patch fails to run checkstyle in hadoop-yarn-server-resourcemanager  |
   | -1 :x: |  mvnsite  |   0m 23s | 
[/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed.  |
   | -1 :x: |  javadoc  |   0m 24s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.  |
   | -1 :x: |  javadoc  |   0m 23s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05.  |
   | -1 :x: |  spotbugs  |   0m 23s | 
[/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/7/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed.  |
   | +1 :green_heart: |  shadedclient  |   2m 44s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | -1 :x: |  mvninstall  |   0m 28s |

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-05 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772432#comment-17772432
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1749963051

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 31s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | -1 :x: |  mvninstall  |   0m 24s | 
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/branch-mvninstall-root.txt)
 |  root in trunk failed.  |
   | -1 :x: |  compile  |   3m 52s | 
[/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.  |
   | -1 :x: |  compile  |   0m 24s | 
[/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05.  |
   | -0 :warning: |  checkstyle  |   0m 22s | 
[/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  The patch fails to run checkstyle in hadoop-yarn-server-resourcemanager  |
   | -1 :x: |  mvnsite  |   0m 23s | 
[/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed.  |
   | -1 :x: |  javadoc  |   0m 24s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04.  |
   | -1 :x: |  javadoc  |   0m 24s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05.  |
   | -1 :x: |  spotbugs  |   0m 24s | 
[/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/6/artifact/out/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed.  |
   | +1 :green_heart: |  shadedclient  |   2m 45s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | -1 :x: |  mvninstall  |   0m 23s |

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-05 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772426#comment-17772426
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1749950026

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 27s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  33m  9s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 39s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 37s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 42s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 20s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  22m 49s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 32s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 35s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 35s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 30s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 30s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 27s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 32s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 29s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m  9s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  23m 49s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  88m 16s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/5/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +0 :ok: |  asflicense  |   0m 28s |  |  ASF License check generated no 
output?  |
   |  |   | 181m  9s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | 
hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler |
   |   | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/5/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 7612417c69e8 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 8c24ef4479e8513a957d54ea5b619a1115cd51d9 |
   | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/5/testReport/ |
   | Max.

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-05 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772409#comment-17772409
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1749848048

   > > @xiaojunxiang2023 We need to fix the checkstyle issue.
   > 
   > I have fixed the issue of indenting 5 characters during line breaks. Is 
there any other style issue?
   
   According to the hint, the only warning should be that the variable name is 
not proper. I just fixed the variable name and updated the PR, and am waiting 
for recompilation.
   
   
![image](https://github.com/apache/hadoop/assets/65019264/92d76b16-c6e0-4af3-8718-122b6b5b0d91)
   
![image](https://github.com/apache/hadoop/assets/65019264/faf18ceb-fd51-44c9-b271-18a7a8140354)
   
   




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-05 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772407#comment-17772407
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1749829744

   > @xiaojunxiang2023 We need to fix the checkstyle issue.
   
   I have fixed the issue of indenting 5 characters during line breaks. Is 
there any other style issue?




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-05 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772406#comment-17772406
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1749826031

   @xiaojunxiang2023 We need to fix the checkstyle issue.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-04 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772047#comment-17772047
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1747927283

   @slfan1989, Hi Sir，could you re-review my pr in your free time?  if you 
approve it, please update the state, thanks.
   
![image](https://github.com/apache/hadoop/assets/65019264/c9beca7b-1ffd-45f1-9ec2-6c5e07dea2bb)
   




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771619#comment-17771619
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1745560860

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 27s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  33m  5s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 43s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 40s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 36s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 42s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 45s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 20s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 46s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 33s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 34s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 32s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 32s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   0m 27s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 2 new + 40 unchanged - 0 fixed = 42 total (was 40)  |
   | +1 :green_heart: |  mvnsite  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 14s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m 32s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  85m 56s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 29s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 175m 38s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/4/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 7ef41ef0eeda 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / d5f7e4f1487c80c8018cb442a86118d5d281306c |
   | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/4/testReport/ |
   | Max. process+thread count | 962 (vs. ulimit

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771541#comment-17771541
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1344375458


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm1.getRMContext().setNodeLabelManager(mgr);
+rm1.start();
+MockNM nm1 = rm1.registerNode("h1:1234", 10 * GB); // label = x
+rm1.registerNode("h2:1234", 10 * GB); // label = y
+MockNM nm3 = rm1.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data5 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)
+.withAppName("app")
+.withUser("user")
+.withAcls(null)
+.withQueue("a1")
+.withAmLabel("x")
+.build();
+RMApp app1 = MockRMAppSubmitter.submit(rm1, data5);
+
+// Submit app2 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data4 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)

Review Comment:
   Ok, I understand what you mean. I have modified the PR. 
   Please review it after compiling later. It takes about four hours





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771418#comment-17771418
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1744733787

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 27s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 2 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  32m 25s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 42s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 40s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 36s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 42s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 19s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 40s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 37s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 37s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 32s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 32s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   0m 27s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 8 new + 40 unchanged - 0 fixed = 48 total (was 40)  |
   | +1 :green_heart: |  mvnsite  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 33s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 30s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 15s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m 19s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  85m 54s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 28s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 174m 22s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 31df8d5c66fd 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 0eb16828b94d9adb6d4bec270238d037becf54ab |
   | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/3/testReport/ |
   | Max. process+thread count | 965 (vs. ulimit

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771368#comment-17771368
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1744416145

   @xiaojunxiang2023 Thanks for the contribution! The code looks fine overall, 
with some minor modifications in the comments.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771367#comment-17771367
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343670977


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm1.getRMContext().setNodeLabelManager(mgr);
+rm1.start();
+MockNM nm1 = rm1.registerNode("h1:1234", 10 * GB); // label = x
+rm1.registerNode("h2:1234", 10 * GB); // label = y
+MockNM nm3 = rm1.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data5 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)
+.withAppName("app")
+.withUser("user")
+.withAcls(null)
+.withQueue("a1")
+.withAmLabel("x")
+.build();
+RMApp app1 = MockRMAppSubmitter.submit(rm1, data5);
+
+// Submit app2 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data4 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)

Review Comment:
   In order to satisfy checkstyle, the indentation of the next line of code 
should generally be maintained at 5 characters.
   
   ```
   MockRMAppSubmissionData data4 =
   MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)
   ```





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771365#comment-17771365
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343666141


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm1.getRMContext().setNodeLabelManager(mgr);
+rm1.start();
+MockNM nm1 = rm1.registerNode("h1:1234", 10 * GB); // label = x
+rm1.registerNode("h2:1234", 10 * GB); // label = y
+MockNM nm3 = rm1.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data5 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)
+.withAppName("app")
+.withUser("user")
+.withAcls(null)
+.withQueue("a1")
+.withAmLabel("x")
+.build();
+RMApp app1 = MockRMAppSubmitter.submit(rm1, data5);
+
+// Submit app2 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data4 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)

Review Comment:
   I think there are some variables with bad names, I will change them later
   





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771362#comment-17771362
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343657080


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm1.getRMContext().setNodeLabelManager(mgr);
+rm1.start();
+MockNM nm1 = rm1.registerNode("h1:1234", 10 * GB); // label = x
+rm1.registerNode("h2:1234", 10 * GB); // label = y
+MockNM nm3 = rm1.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data5 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)
+.withAppName("app")
+.withUser("user")
+.withAcls(null)
+.withQueue("a1")
+.withAmLabel("x")
+.build();
+RMApp app1 = MockRMAppSubmitter.submit(rm1, data5);
+
+// Submit app2 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data4 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)

Review Comment:
   "5 chars， indentation"  what mean, I don't understand this.  Please be more 
specific, thanks.



##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771363#comment-17771363
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343657080


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm1.getRMContext().setNodeLabelManager(mgr);
+rm1.start();
+MockNM nm1 = rm1.registerNode("h1:1234", 10 * GB); // label = x
+rm1.registerNode("h2:1234", 10 * GB); // label = y
+MockNM nm3 = rm1.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data5 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)
+.withAppName("app")
+.withUser("user")
+.withAcls(null)
+.withQueue("a1")
+.withAmLabel("x")
+.build();
+RMApp app1 = MockRMAppSubmitter.submit(rm1, data5);
+
+// Submit app2 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data4 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)

Review Comment:
   "5 chars， indentation"  what means, I don't understand this.  Please be more 
specific, thanks.



##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771356#comment-17771356
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343607360


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm1.getRMContext().setNodeLabelManager(mgr);
+rm1.start();
+MockNM nm1 = rm1.registerNode("h1:1234", 10 * GB); // label = x
+rm1.registerNode("h2:1234", 10 * GB); // label = y
+MockNM nm3 = rm1.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data5 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)
+.withAppName("app")
+.withUser("user")
+.withAcls(null)
+.withQueue("a1")
+.withAmLabel("x")
+.build();
+RMApp app1 = MockRMAppSubmitter.submit(rm1, data5);
+
+// Submit app2 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data4 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)

Review Comment:
   indentation.





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771355#comment-17771355
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343606908


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##
@@ -1027,4 +1027,83 @@ public void testAMLimitByAllResources() throws Exception 
{
 rm.close();
 
   }
+
+  @Test(timeout = 12)
+  public void testDiagnosticWhenAMActivated() throws Exception {
+/*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue A1 for label X can only support 2Gb AM resource.
+ * Queue C1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify atleast one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+simpleNodeLabelMappingToManager();
+CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+TestUtils.getConfigurationWithQueueLabels(conf);
+
+// After getting queue conf, configure AM resource percent for Queue A1
+// as 0.2 (Label X) and for Queue C1 as 0.2 (Empty Label)
+final String A1 = CapacitySchedulerConfiguration.ROOT + ".a" + ".a1";
+final String C1 = CapacitySchedulerConfiguration.ROOT + ".c" + ".c1";
+config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+// Now inject node label manager with this updated config
+MockRM rm1 = new MockRM(config) {
+  @Override
+  public RMNodeLabelsManager createNodeLabelManager() {
+return mgr;
+  }
+};
+
+rm1.getRMContext().setNodeLabelManager(mgr);
+rm1.start();
+MockNM nm1 = rm1.registerNode("h1:1234", 10 * GB); // label = x
+rm1.registerNode("h2:1234", 10 * GB); // label = y
+MockNM nm3 = rm1.registerNode("h3:1234", 10 * GB); // label = 
+
+// Submit app1 with 1Gb AM resource to Queue A1 for label X
+MockRMAppSubmissionData data5 =
+MockRMAppSubmissionData.Builder.createWithMemory(GB, rm1)

Review Comment:
   5 chars





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-03 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771348#comment-17771348
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1744289927

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 28s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 2 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  33m 18s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 39s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 37s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 42s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 22s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 27s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 33s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 37s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 37s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 33s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 33s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   0m 26s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 8 new + 40 unchanged - 0 fixed = 48 total (was 40)  |
   | +1 :green_heart: |  mvnsite  |   0m 33s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 33s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 29s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 15s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m 29s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  85m 57s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 28s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 175m 18s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 3fafba7a4f52 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 0b78bfd5c130df194d8b590a6bb8f8609b9a0c22 |
   | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/2/testReport/ |
   | Max. process+thread count | 922 (vs. ulimit

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771305#comment-17771305
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343290820


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/TestFicaSchedulerAPP.java:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertThat;
+import static org.hamcrest.CoreMatchers.containsString;
+
+public class TestFicaSchedulerAPP {
+
+  @Test
+  public void testGetActivedAppDiagnosticMessage() throws 
IllegalAccessException, InstantiationException {

Review Comment:
   I took a closer look at this unit test, which uses simple string 
concatenation for testing. I think this is not sufficient. Can we rewrite a 
more meaningful unit test?





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771306#comment-17771306
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1743994028

   Indeed, I will learn how to write test cases with context first, and I will 
catch up later




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771304#comment-17771304
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343286751


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/TestFicaSchedulerAPP.java:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertThat;

Review Comment:
   We recommend using `org.assertj.core.api.Assertions.assertThat` instead of 
`org.junit.Assert.assertThats.assertThat`.





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771303#comment-17771303
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343290820


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/TestFicaSchedulerAPP.java:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertThat;
+import static org.hamcrest.CoreMatchers.containsString;
+
+public class TestFicaSchedulerAPP {
+
+  @Test
+  public void testGetActivedAppDiagnosticMessage() throws 
IllegalAccessException, InstantiationException {

Review Comment:
   I took a closer look at this unit test, which uses simple string 
concatenation for testing. I think this is not sufficient. Can we rewrite a 
more meaningful unit test?





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771302#comment-17771302
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343287506


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/TestFicaSchedulerAPP.java:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertThat;
+import static org.hamcrest.CoreMatchers.containsString;
+
+public class TestFicaSchedulerAPP {
+
+  @Test
+  public void testGetActivedAppDiagnosticMessage() throws 
IllegalAccessException, InstantiationException {
+  StringBuilder diagnosticMessage = new StringBuilder(
+  "Application is Activated, waiting for resources to be assigned 
for AM");
+  getActivedAppDiagnosticMessage(diagnosticMessage);
+  assertThat("AM Resource Request information was not successfully 
displayed.",
+  diagnosticMessage.toString(), containsString("AM Resource 
Request ="));
+  }
+
+  // copy from FiCaSchedulerApp#getActivedAppDiagnosticMessage
+  protected void getActivedAppDiagnosticMessage(
+  StringBuilder diagnosticMessage) {
+  diagnosticMessage.append(" Details : AM Partition = ")
+  .append(" ; ")

Review Comment:
   5chars.





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771301#comment-17771301
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343287151


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/TestFicaSchedulerAPP.java:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertThat;
+import static org.hamcrest.CoreMatchers.containsString;
+
+public class TestFicaSchedulerAPP {
+
+  @Test
+  public void testGetActivedAppDiagnosticMessage() throws 
IllegalAccessException, InstantiationException {
+  StringBuilder diagnosticMessage = new StringBuilder(
+  "Application is Activated, waiting for resources to be assigned 
for AM");

Review Comment:
   indentation 5chars.





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771300#comment-17771300
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1343286751


##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/TestFicaSchedulerAPP.java:
##
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertThat;

Review Comment:
   We recommend using `org.assertj.core.api.Assertions.assertThat` instead of 
`org.junit.Assert.assertThats.assertThat`.





> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771135#comment-17771135
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1743058539

   Good idea,  tomorrow I will try to see if the fair scheduler has this 
problem.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17771131#comment-17771131
 ] 

ASF GitHub Bot commented on YARN-11582:
---

slfan1989 commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1743052847

   @xiaojunxiang2023 Can we check if fairscheduler has this issue? We need to 
fix checkstyle.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770987#comment-17770987
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6137:
URL: https://github.com/apache/hadoop/pull/6137#issuecomment-1742483471

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 40s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  0s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  45m 58s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  7s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 59s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   1m  0s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m  5s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m  1s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 51s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   2m  9s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  36m 52s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 51s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 57s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 57s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 48s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 48s |  |  the patch passed  |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/2/artifact/out/blanks-eol.txt)
 |  The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | -0 :warning: |  checkstyle  |   0m 43s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/2/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 6 new + 5 unchanged - 0 fixed = 11 total (was 5)  |
   | +1 :green_heart: |  mvnsite  |   0m 53s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 47s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 43s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   2m  7s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  34m 55s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  | 101m  2s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/2/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 40s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 238m 52s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | 
hadoop.yarn.server.resourcemanager.scheduler.common.fica.TestFicaSchedulerAPP |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6137 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux f3bae2fd167c 4.15.0-213-generic #224-Ubuntu

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770984#comment-17770984
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 commented on PR #6137:
URL: https://github.com/apache/hadoop/pull/6137#issuecomment-1742471639

   cancel it，to PR. 6139




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-02 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770982#comment-17770982
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#issuecomment-1742467901

   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 28s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  32m 40s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 39s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 36s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 42s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 36s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 19s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 24s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 34s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 35s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 35s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 33s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 33s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   0m 26s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 6 new + 5 unchanged - 0 fixed = 11 total (was 5)  |
   | +1 :green_heart: |  mvnsite  |   0m 32s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 32s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 29s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 16s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  21m 17s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  85m 42s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 29s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 174m 20s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6139 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 180ea3a03372 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 947216c6c8393996ccaf9aadc30c48765985587e |
   | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6139/1/testReport/ |
   | Max. process+thread count | 950 (vs. ulimit of

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770956#comment-17770956
 ] 

ASF GitHub Bot commented on YARN-11582:
---

hadoop-yetus commented on PR #6137:
URL: https://github.com/apache/hadoop/pull/6137#issuecomment-1742390045

   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 38s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +0 :ok: |  detsecrets  |   0m  1s |  |  detect-secrets was not available.  
|
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  45m 41s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  4s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  compile  |   0m 59s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  checkstyle  |   0m 59s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m  4s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m  3s |  |  trunk passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 51s |  |  trunk passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   2m  1s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  35m  4s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 54s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javac  |   0m 54s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 50s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  javac  |   0m 50s |  |  the patch passed  |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/1/artifact/out/blanks-eol.txt)
 |  The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | -0 :warning: |  checkstyle  |   0m 44s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 8 new + 5 unchanged - 0 fixed = 13 total (was 5)  |
   | +1 :green_heart: |  mvnsite  |   0m 52s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 43s |  |  the patch passed with JDK 
Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04  |
   | +1 :green_heart: |  javadoc  |   0m 43s |  |  the patch passed with JDK 
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05  |
   | +1 :green_heart: |  spotbugs  |   1m 54s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  34m 34s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  | 100m 16s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 38s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 235m  7s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | 
hadoop.yarn.server.resourcemanager.scheduler.common.fica.TestFicaSchedulerAPP |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.43 ServerAPI=1.43 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6137/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/6137 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
   | uname | Linux 8b699b0020ec 4.15.0-213-generic #224-Ubuntu

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770955#comment-17770955
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 opened a new pull request, #6139:
URL: https://github.com/apache/hadoop/pull/6139

   jira：https://issues.apache.org/jira/browse/YARN-11582
   
   When Yarn resources are insufficient, the newly submitted job AM may be in 
the state of "Application is Activated, waiting for resources to be assigned 
for AM". This is obviously because Yarn doesn't have enough resources to 
allocate another AM Container, so we want to know how large the AM Container is 
currently allocated. Unfortunately, the current diagnosticMessage on the Web 
page does not show this data. Therefore, it is necessary to add the resource 
size of the AM Container in the diagnosticMessage, which will be very useful 
for us to troubleshoise the production faults on line.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770952#comment-17770952
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 closed pull request #6137: YARN-11582. Improve WebUI 
diagnosticMessage to show AM Container resource request size.
URL: https://github.com/apache/hadoop/pull/6137




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-11582) Improve WebUI diagnosticMessage to show AM Container resource request size

2023-10-01 Thread ASF GitHub Bot (Jira)



[ 
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770943#comment-17770943
 ] 

ASF GitHub Bot commented on YARN-11582:
---

xiaojunxiang2023 opened a new pull request, #6137:
URL: https://github.com/apache/hadoop/pull/6137

   jira：https://issues.apache.org/jira/browse/YARN-11582
   
   When Yarn resources are insufficient, the newly submitted job AM may be in 
the state of "Application is Activated, waiting for resources to be assigned 
for AM". This is obviously because Yarn doesn't have enough resources to 
allocate another AM Container, so we want to know how large the AM Container is 
currently allocated. Unfortunately, the current diagnosticMessage on the Web 
page does not show this data. Therefore, it is necessary to add the resource 
size of the AM Container in the diagnosticMessage, which will be very useful 
for us to troubleshoise the production faults on line.




> Improve WebUI diagnosticMessage to show AM Container resource request size
> --
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: applications, resourcemanager
>Affects Versions: 3.3.4
>Reporter: xiaojunxiang
>Priority: Major
> Attachments: image-2023-10-02-00-05-34-337.png
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in 
> the state of "Application is Activated, waiting for resources to be assigned 
> for AM". This is obviously because Yarn doesn't have enough resources to 
> allocate another AM Container, so we want to know how large the AM Container 
> is currently allocated. Unfortunately, the current diagnosticMessage on the 
> Web page does not show this data. Therefore, it is necessary to add the 
> resource size of the AM Container in the diagnosticMessage, which will be 
> very useful for us to troubleshoise the production faults on line.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

47 matches

Mail list logo