[
https://issues.apache.org/jira/browse/YARN-9936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated YARN-9936:
---------------------------------
Description:
Currently, the Capacity Scheduler queue configuration supports two ways to set
queue capacity.
* In percentage of all available resources as a float ( eg. 25.0 ) means 25%
of the resources of its parent queue for all resource types equally (eg. 25% of
all memory, 25% of all CPU cores, and 25% of all available GPU in the cluster)
The percentages of all queues has to add up to 100%.
* In an absolute amount of resources ( e.g. memory=4GB,vcores=20,yarn.io/gpu=4
). The amount of all resources in the queues has to be less than or equal to
all resources in the cluster.{color:#de350b}Actually, the above is not
supported, we only support memory and vcores now in absolute mode, we should
extend in {color} YARN-10503.
Apart from these two already existing ways, there is a demand to add capacity
percentage of each available resource type separately. (eg.
{{memory=20%,vcores=40%,yarn.io/gpu=100%}}).
At the same time, a similar concept should be included with queues
maximum-capacity as well.
was:
Currently, the Capacity Scheduler queue configuration supports two ways to set
queue capacity.
* In percentage of all available resources as a float ( eg. 25.0 ) means 25%
of the resources of its parent queue for all resource types equally (eg. 25% of
all memory, 25% of all CPU cores, and 25% of all available GPU in the cluster)
The percentages of all queues has to add up to 100%.
* In an absolute amount of resources ( e.g. memory=4GB,vcores=20,yarn.io/gpu=4
). The amount of all resources in the queues has to be less than or equal to
all resources in the cluster.{color:#de350b}Actually, the above is not
supported, we only support memory and vcores now in absolute mode, we should
extend in {color}YARN-10503.
Apart from these two already existing ways, there is a demand to add capacity
percentage of each available resource type separately. (eg.
{{memory=20%,vcores=40%,yarn.io/gpu=100%}}).
At the same time, a similar concept should be included with queues
maximum-capacity as well.
> Support vector of capacity percentages in Capacity Scheduler configuration
> --------------------------------------------------------------------------
>
> Key: YARN-9936
> URL: https://issues.apache.org/jira/browse/YARN-9936
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Zoltan Siegl
> Assignee: Andras Gyori
> Priority: Major
> Attachments: Capacity Scheduler support of “vector of resources
> percentage”.pdf
>
>
> Currently, the Capacity Scheduler queue configuration supports two ways to
> set queue capacity.
> * In percentage of all available resources as a float ( eg. 25.0 ) means 25%
> of the resources of its parent queue for all resource types equally (eg. 25%
> of all memory, 25% of all CPU cores, and 25% of all available GPU in the
> cluster) The percentages of all queues has to add up to 100%.
> * In an absolute amount of resources ( e.g.
> memory=4GB,vcores=20,yarn.io/gpu=4 ). The amount of all resources in the
> queues has to be less than or equal to all resources in the
> cluster.{color:#de350b}Actually, the above is not supported, we only support
> memory and vcores now in absolute mode, we should extend in {color}
> YARN-10503.
> Apart from these two already existing ways, there is a demand to add capacity
> percentage of each available resource type separately. (eg.
> {{memory=20%,vcores=40%,yarn.io/gpu=100%}}).
> At the same time, a similar concept should be included with queues
> maximum-capacity as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
