[ https://issues.apache.org/jira/browse/YARN-8842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639171#comment-16639171 ]
Szilard Nemeth commented on YARN-8842: -------------------------------------- Hi [~wilfreds]! Thanks for the comments. 1. Good idea to decide on the flag. However, it's not immediately straightforward for me whether we only need to treat memory / cores as default resources. See: https://github.com/apache/hadoop/blob/12a095a496dd59066d73a7a6c24129b5b6a9d650/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceInformation.java#L56 , here we also have GPU / FPGA in the mandatory resources list. How would you exactly check in the constructor whether we have custom resources configured? I can think of the following: As we receive the configuration object in the constructor, we could check for how many keys are in there for {{YarnConfiguration.RESOURCE_TYPES}}. If it's 2 (4 in case we treat GPU / FPGA as standard resources), then we are not creating the {{QueueMetricsForCustomResources}}, otherwise we create it. Can you think of any better way to do this? 2. It's a good catch, thanks! 3. Good point, fixed. 4. Refactored {{QueueMetricsTestcase}} in order to handle more than 2 queues, so that I can not only specify root and leaf queues, but multiple queues in a hierarchy. > Update QueueMetrics with custom resource values > ------------------------------------------------ > > Key: YARN-8842 > URL: https://issues.apache.org/jira/browse/YARN-8842 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Szilard Nemeth > Assignee: Szilard Nemeth > Priority: Major > Attachments: YARN-8842.001.patch, YARN-8842.002.patch > > > This is the 2nd dependent jira of YARN-8059. > As updating the metrics is an independent step from handling preemption, this > jira only deals with the queue metrics update of custom resources. > The following metrics should be updated: > * allocated resources > * available resources > * pending resources > * reserved resources > * aggregate seconds preempted -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org