[
https://issues.apache.org/jira/browse/YARN-11582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17831854#comment-17831854
]
ASF GitHub Bot commented on YARN-11582:
---------------------------------------
hiwangzhihui commented on code in PR #6139:
URL: https://github.com/apache/hadoop/pull/6139#discussion_r1542224470
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimitsByPartition.java:
##########
@@ -1027,4 +1027,79 @@ public void testAMLimitByAllResources() throws Exception
{
rm.close();
}
+
+ @Test(timeout = 120000)
+ public void testDiagnosticWhenAMActivated() throws Exception {
+ /*
+ * Test Case:
+ * Verify AM resource limit per partition level and per queue level. So
+ * we use 2 queues to verify this case.
+ * Queue a1 supports labels (x,y). Configure am-resource-limit as 0.2 (x)
+ * Queue c1 supports default label. Configure am-resource-limit as 0.2
+ *
+ * Queue a1 for label X can only support 2Gb AM resource.
+ * Queue c1 (empty label) can support 2Gb AM resource.
+ *
+ * Verify at least one AM is launched, and AM resources should not go more
+ * than 2GB in each queue.
+ */
+
+ simpleNodeLabelMappingToManager();
+ CapacitySchedulerConfiguration config = (CapacitySchedulerConfiguration)
+ TestUtils.getConfigurationWithQueueLabels(conf);
+
+ // After getting queue conf, configure AM resource percent for Queue a1
+ // as 0.2 (Label X) and for Queue c1 as 0.2 (Empty Label).
+ config.setMaximumAMResourcePercentPerPartition(A1, "x", 0.2f);
+ config.setMaximumApplicationMasterResourcePerQueuePercent(C1, 0.2f);
+
+ // Now inject node label manager with this updated config.
+ MockRM rm = new MockRM(config) {
+ @Override
+ public RMNodeLabelsManager createNodeLabelManager() {
+ return mgr;
+ }
+ };
+
+ rm.getRMContext().setNodeLabelManager(mgr);
+ rm.start();
+ rm.registerNode("h1:1234", 10 * GB); // label = x
+ rm.registerNode("h2:1234", 10 * GB); // label = y
+ rm.registerNode("h3:1234", 10 * GB); // label = <empty>
+
+ // Submit app1 with 1Gb AM resource to Queue a1 for label X
+ MockRMAppSubmissionData data1 =
+ MockRMAppSubmissionData.Builder.createWithMemory(GB, rm)
+ .withAppName("app")
+ .withUser("user")
+ .withAcls(null)
+ .withQueue("a1")
+ .withAmLabel("x")
+ .build();
+ RMApp app1 = MockRMAppSubmitter.submit(rm, data1);
+
+ // Submit app2 with 1Gb AM resource to Queue a1 for label X
+ MockRMAppSubmissionData data2 =
+ MockRMAppSubmissionData.Builder.createWithMemory(GB, rm)
+ .withAppName("app")
+ .withUser("user")
+ .withAcls(null)
+ .withQueue("a1")
+ .withAmLabel("x")
+ .build();
+ RMApp app2 = MockRMAppSubmitter.submit(rm, data2);
+
+ CapacityScheduler cs = (CapacityScheduler) rm.getResourceScheduler();
+ LeafQueue leafQueue = (LeafQueue) cs.getQueue("a1");
+ Assert.assertNotNull(leafQueue);
+
+ // Only one AM will be activated here, and second AM will be still pending.
+ Assert.assertEquals(2, leafQueue.getNumActiveApplications());
+ String activatedDiagnostics="AM Resource Request = ";
+ Assert.assertTrue("still doesn't show AMResource When Activated",
app1.getDiagnostics()
+ .toString().contains(activatedDiagnostics));
Review Comment:
Add a test checks for Am resource prompts would be better
> Improve WebUI diagnosticMessage to show AM Container resource request size
> --------------------------------------------------------------------------
>
> Key: YARN-11582
> URL: https://issues.apache.org/jira/browse/YARN-11582
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: applications, resourcemanager
> Affects Versions: 3.3.4
> Reporter: xiaojunxiang
> Priority: Minor
> Labels: pull-request-available
> Attachments: image-2023-10-02-00-05-34-337.png,
> image-2024-03-28-22-11-37-903.png, success_ShowAMInfo.jpg
>
>
> When Yarn resources are insufficient, the newly submitted job AM may be in
> the state of "Application is Activated, waiting for resources to be assigned
> for AM". This is obviously because Yarn doesn't have enough resources to
> allocate another AM Container, so we want to know how large the AM Container
> is currently allocated. Unfortunately, the current diagnosticMessage on the
> Web page does not show this data. Therefore, it is necessary to add the
> resource size of the AM Container in the diagnosticMessage, which will be
> very useful for us to troubleshoise the production faults on line.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
