[
https://issues.apache.org/jira/browse/YARN-9623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tao Yang updated YARN-9623:
---------------------------
Description:
Currently we can use configuration entry
"yarn.resourcemanager.activities-manager.app-activities.max-queue-length" to
control max queue length of app activities, but in some scenarios , this
configuration may need to be updated in a growing cluster. Moreover, it's
better for users to ignore that conf therefor it should be auto adjusted
internally.
There are some differences among different scheduling modes:
* multi-node placement disabled
** Heartbeat driven scheduling: max queue length of app activities should not
less than the number of nodes, considering nodes can not be always in order, we
should make some room for misorder, for example, we can guarantee that max
queue length should not be less than 1.2 * numNodes
** Async scheduling: every async scheduling thread goes through all nodes in
order, in this mode, we should guarantee that max queue length should be
numThreads * numNodes.
* multi-node placement enabled: activities on all nodes can be involved in a
single app allocation, therefor there's no need to adjust for this mode.
To sum up, we can adjust the max queue length of app activities like this:
{code}
int configuredMaxQueueLength;
int maxQueueLength;
serviceInit(){
...
configuredMaxQueueLength = ...; //read configured max queue length
maxQueueLength = configuredMaxQueueLength; //take configured value as default
}
CleanupThread#run(){
...
if (multiNodeDisabled) {
if (asyncSchedulingEnabled) {
maxQueueLength = max(configuredMaxQueueLength, numSchedulingThreads *
numNodes);
} else {
maxQueueLength = max(configuredMaxQueueLength, 1.2 * numNodes);
}
} else if (maxQueueLength != configuredMaxQueueLength) {
maxQueueLength = configuredMaxQueueLength;
}
}
{code}
was:
Currently we can use configuration entry
"yarn.resourcemanager.activities-manager.app-activities.max-queue-length" to
control max queue length of app activities, but in some scenarios , this
configuration may need to be updated in a growing cluster. Moreover, it's
better for users to ignore that conf therefor it should be auto adjusted
internally.
There are some differences among different scheduling modes:
* multi-node placement disabled
** Heartbeat driven scheduling: max queue length of app activities should not
less than the number of nodes, considering nodes can not be always in order, we
should make some room for misorder, for example, we can guarantee that max
queue length should not be less than 1.2 * numNodes
** Async scheduling: every async scheduling thread goes through all nodes in
order, in this mode, we should guarantee that max queue length should be
numThreads * numNodes.
* multi-node placement enabled: activities on all nodes can be involved in a
single app allocation, therefor there's no need to adjust for this mode.
To sum up, we can adjust the max queue length of app activities like this:
{code}
int configuredMaxQueueLength;
int maxQueueLength;
serviceInit(){
...
configuredMaxQueueLength = ...; //read configured max queue length
maxQueueLength = configuredMaxQueueLength; //take configured value as default
}
CleanupThread#run(){
...
if (multiNodeDisabled) {
if (asyncSchedulingEnabled) {
maxQueueLength = max(configuredMaxQueueLength, numSchedulingThreads *
numNodes);
} else {
maxQueueLength = max(configuredMaxQueueLength, 1.2 * numNodes);
}
}
}
{code}
> Auto adjust max queue length of app activities to make sure activities on all
> nodes can be covered
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-9623
> URL: https://issues.apache.org/jira/browse/YARN-9623
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Tao Yang
> Assignee: Tao Yang
> Priority: Major
>
> Currently we can use configuration entry
> "yarn.resourcemanager.activities-manager.app-activities.max-queue-length" to
> control max queue length of app activities, but in some scenarios , this
> configuration may need to be updated in a growing cluster. Moreover, it's
> better for users to ignore that conf therefor it should be auto adjusted
> internally.
> There are some differences among different scheduling modes:
> * multi-node placement disabled
> ** Heartbeat driven scheduling: max queue length of app activities should
> not less than the number of nodes, considering nodes can not be always in
> order, we should make some room for misorder, for example, we can guarantee
> that max queue length should not be less than 1.2 * numNodes
> ** Async scheduling: every async scheduling thread goes through all nodes in
> order, in this mode, we should guarantee that max queue length should be
> numThreads * numNodes.
> * multi-node placement enabled: activities on all nodes can be involved in a
> single app allocation, therefor there's no need to adjust for this mode.
> To sum up, we can adjust the max queue length of app activities like this:
> {code}
> int configuredMaxQueueLength;
> int maxQueueLength;
> serviceInit(){
> ...
> configuredMaxQueueLength = ...; //read configured max queue length
> maxQueueLength = configuredMaxQueueLength; //take configured value as
> default
> }
> CleanupThread#run(){
> ...
> if (multiNodeDisabled) {
> if (asyncSchedulingEnabled) {
> maxQueueLength = max(configuredMaxQueueLength, numSchedulingThreads *
> numNodes);
> } else {
> maxQueueLength = max(configuredMaxQueueLength, 1.2 * numNodes);
> }
> } else if (maxQueueLength != configuredMaxQueueLength) {
> maxQueueLength = configuredMaxQueueLength;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
