[
https://issues.apache.org/jira/browse/YUNIKORN-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qi Zhu updated YUNIKORN-2156:
-----------------------------
Description:
Try to improve the case, Spark Dynamic Allocation when we enable gang
scheduling:
If we can add some improvements to this case? Such as:
*For preemption case:*
lower priority for anything more than the gang and thus get them killed earlier
than when they do not belong to a gang
*For quota hit:*
For normal gang scheduling, we will reject jobs with total gang size > quota,
but for dynamic allocations, if we can add monitor logic to monitor the
exceeded resource allocations for those jobs?
* We may extend the quota some value when it happens.
was:
Try to improve the case, Spark Dynamic Allocation when we enable gang
scheduling:
If we can add some improvements to this case? Such as: * For preemption case:
#
** lower priority for anything more than the gang and thus get them killed
earlier than when they do not belong to a gang
* For quota hit:
#
** For normal gang scheduling, we will reject jobs with total gang size > quota
** But for dynamic allocations, if we can add monitor logic to monitor the
exceeded resource allocations for those jobs?
#
##
*** We may extend the quota some value when it happens.
> Improvement for gang scheduling when spark dynamic allocation enabled
> ---------------------------------------------------------------------
>
> Key: YUNIKORN-2156
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2156
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - scheduler
> Reporter: Qi Zhu
> Priority: Major
>
> Try to improve the case, Spark Dynamic Allocation when we enable gang
> scheduling:
> If we can add some improvements to this case? Such as:
> *For preemption case:*
> lower priority for anything more than the gang and thus get them killed
> earlier than when they do not belong to a gang
> *For quota hit:*
> For normal gang scheduling, we will reject jobs with total gang size > quota,
> but for dynamic allocations, if we can add monitor logic to monitor the
> exceeded resource allocations for those jobs?
> * We may extend the quota some value when it happens.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
