[
https://issues.apache.org/jira/browse/YARN-9200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755286#comment-16755286
]
Aihua Xu commented on YARN-9200:
--------------------------------
[~leftnoteasy], [~rohithsharma] and [~cheersyang] I'm trying to add the support
to have separate percentage value for each resource (i.e., have an array
instead of float value for each capacity). Seems there are much more change
than I originally thought, especially the changes in QueueCapacities and the
related ones. Before I move forward with such massive changes, I attached the
draft and want to check with you folks if there is a better way.
What I have done and I'm planning to do is: CapacitySchedulerConfiguration
supports both "45" or "memory=80,vCores=20" and internally keeps an array - one
value per resource; in QueueCapacities to map each label to an array of
Capacities.
You feedback is appreciated.
> Enable resource configuration of queue capacity for different resources
> independently
> -------------------------------------------------------------------------------------
>
> Key: YARN-9200
> URL: https://issues.apache.org/jira/browse/YARN-9200
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
> Affects Versions: 3.1.0
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
> Attachments: YARN-9200.draft
>
>
> In capacity scheduler, currently two resource allocations are supported. 1.
> percentage allocation for child queues - the child queue gets a defined
> percentage of the resources for all the resource types; 2. absolute values
> (YARN-5881) - each resource is configured an absolute values.
> Right now we can't mix these case together and it would also very confusing
> to mix them in one cluster. The second case actually is more targeting toward
> cloud env.
> In a non-cloud env, the ability to configure each resource independently is
> also useful, but percentage is preferable over absolute value. One thought
> here is to add the percentage configuration for each resource type on the
> queue. That would allow us to configure memory bounded queues, or CPU bounded
> queues. We can also keep backward compatible: each resource type just gets
> the same percentage if no percentage is configured for individual resource
> type.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
