[
https://issues.apache.org/jira/browse/YARN-9671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated YARN-9671:
--------------------------------------
Description:
When a cluster is very busy, scheduling opportunities are few and far between.
Scheduling opportunities are how an application knows when to give up looking
for decent locality.
It doesn't make sense to work hard waiting for locality when the odds of it
coming are very small and it may actually take a very long time to actually
give up.
This causes the priority of queues to be violated which is the last thing we
want to do when the cluster is full.
Add a mode to disable skipping locality when cluster is busy.
was:
When a cluster is very busy, scheduling opportunities are few and far between.
Scheduling opportunities are how an application knows when to give up looking
for decent locality.
It doesn't make sense to work hard waiting for locality when the odds of it
coming are very small and it may actually take a very long time to actually
give up.
This causes the priority of queues to be violated which is the last thing we
want to do when the cluster is full.
* Add metrics for queue priority inversions.
* Add mode to disable skipping locality when cluster is busy.
> Improve Locality Scheduling when cluster is busy
> ------------------------------------------------
>
> Key: YARN-9671
> URL: https://issues.apache.org/jira/browse/YARN-9671
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Muhammad Samir Khan
> Assignee: Muhammad Samir Khan
> Priority: Major
>
> When a cluster is very busy, scheduling opportunities are few and far
> between. Scheduling opportunities are how an application knows when to give
> up looking for decent locality.
> It doesn't make sense to work hard waiting for locality when the odds of it
> coming are very small and it may actually take a very long time to actually
> give up.
> This causes the priority of queues to be violated which is the last thing we
> want to do when the cluster is full.
> Add a mode to disable skipping locality when cluster is busy.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
