[
https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17224905#comment-17224905
]
Dongjoon Hyun edited comment on SPARK-24432 at 11/2/20, 7:17 PM:
-----------------------------------------------------------------
BTW, FYI, for K8s environment, the followings are the current status.
- The initial K8s dynamic allocation is already shipped at Apache Spark 3.0.0
with shuffle tracking.
- The K8s dynamic allocation with storage migration between executors is
already in `master` branch for Apache Spark 3.1.0.
was (Author: dongjoon):
BTW, FYI, for K8s environment, the followings are the current status.
- The initial K8s dynamic allocation is already shipped at Apache Spark 3.0.0
with shuffle tracking.
- The K8s dynamic allocation with storage migration is already in `master`
branch for Apache Spark 3.1.0.
> Add support for dynamic resource allocation
> -------------------------------------------
>
> Key: SPARK-24432
> URL: https://issues.apache.org/jira/browse/SPARK-24432
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes, Spark Core
> Affects Versions: 3.1.0
> Reporter: Yinan Li
> Priority: Major
>
> This is an umbrella ticket for work on adding support for dynamic resource
> allocation into the Kubernetes mode. This requires a Kubernetes-specific
> external shuffle service. The feature is available in our fork at
> github.com/apache-spark-on-k8s/spark.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
