[
https://issues.apache.org/jira/browse/YARN-8842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639171#comment-16639171
]
Szilard Nemeth commented on YARN-8842:
--------------------------------------
Hi [~wilfreds]!
Thanks for the comments.
1. Good idea to decide on the flag. However, it's not immediately
straightforward for me whether we only need to treat memory / cores as default
resources. See:
https://github.com/apache/hadoop/blob/12a095a496dd59066d73a7a6c24129b5b6a9d650/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceInformation.java#L56
, here we also have GPU / FPGA in the mandatory resources list.
How would you exactly check in the constructor whether we have custom resources
configured?
I can think of the following: As we receive the configuration object in the
constructor, we could check for how many keys are in there for
{{YarnConfiguration.RESOURCE_TYPES}}. If it's 2 (4 in case we treat GPU / FPGA
as standard resources), then we are not creating the
{{QueueMetricsForCustomResources}}, otherwise we create it.
Can you think of any better way to do this?
2. It's a good catch, thanks!
3. Good point, fixed.
4. Refactored {{QueueMetricsTestcase}} in order to handle more than 2 queues,
so that I can not only specify root and leaf queues, but multiple queues in a
hierarchy.
> Update QueueMetrics with custom resource values
> ------------------------------------------------
>
> Key: YARN-8842
> URL: https://issues.apache.org/jira/browse/YARN-8842
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Szilard Nemeth
> Assignee: Szilard Nemeth
> Priority: Major
> Attachments: YARN-8842.001.patch, YARN-8842.002.patch
>
>
> This is the 2nd dependent jira of YARN-8059.
> As updating the metrics is an independent step from handling preemption, this
> jira only deals with the queue metrics update of custom resources.
> The following metrics should be updated:
> * allocated resources
> * available resources
> * pending resources
> * reserved resources
> * aggregate seconds preempted
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
