Enrico Minack created SPARK-52504:
-------------------------------------
Summary: Spark on Kubernetes with storage decommissioning
Key: SPARK-52504
URL: https://issues.apache.org/jira/browse/SPARK-52504
Project: Spark
Issue Type: Umbrella
Components: k8s, Kubernetes
Affects Versions: 4.1.0
Reporter: Enrico Minack
Spark on Kubernetes provides the storage decommissioning feature where shuffle
data stored on an executor are migrated to other executors and optionally the
fallback storage when an executor gets decommissioned.
Using this feature allows for dynamic allocation on Kubernetes. However, using
this in anger has showed a number of issues that render this setup not fit for
production use.
This issue collects those issues to provide an overview.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
