[
https://issues.apache.org/jira/browse/SPARK-55555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-55555:
----------------------------------
Description:
Historically, the Apache Spark manages executors in some smart ways via (1)
Dynamic Allocation, (2) Resource profiles, (3) Executor Rolling.
On top of that, this issue aims to cover more advanced resource managements by
taking advantage of new K8s features. For example,
- We can resize CPU and Memory requests and limits of executors via In-Place
Pod Update.
- We can scale-up the PVC volumes via `AllowVolumeExpansion`.
was:
Historically, the Apache Spark manages executors in a smart way via Dynamic
Allocation or Resource profiles.
On top of that, this issue aims to cover more advanced resource managements by
taking advantage of new K8s features. For example,
- We can resize CPU and Memory requests and limits of executors via In-Place
Pod Update.
- We can scale-up the PVC volumes via `AllowVolumeExpansion`.
> Support heterogeneous K8s executors
> -----------------------------------
>
> Key: SPARK-55555
> URL: https://issues.apache.org/jira/browse/SPARK-55555
> Project: Spark
> Issue Type: Umbrella
> Components: Kubernetes
> Affects Versions: 4.2.0
> Reporter: Dongjoon Hyun
> Priority: Critical
>
> Historically, the Apache Spark manages executors in some smart ways via (1)
> Dynamic Allocation, (2) Resource profiles, (3) Executor Rolling.
> On top of that, this issue aims to cover more advanced resource managements
> by taking advantage of new K8s features. For example,
> - We can resize CPU and Memory requests and limits of executors via In-Place
> Pod Update.
> - We can scale-up the PVC volumes via `AllowVolumeExpansion`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
