Marcelo Vanzin created SPARK-27963: -------------------------------------- Summary: Allow dynamic allocation without an external shuffle service Key: SPARK-27963 URL: https://issues.apache.org/jira/browse/SPARK-27963 Project: Spark Issue Type: New Feature Components: Spark Core Affects Versions: 3.0.0 Reporter: Marcelo Vanzin
It would be useful for users to be able to enable dynamic allocation without the need to provision an external shuffle service. One immediate use case is the ability to use dynamic allocation on Kubernetes, which doesn't yet have that service. This has been suggested before (e.g. https://github.com/apache/spark/pull/24083, which was attached to the k8s-specific SPARK-24432), and can actually be done without affecting the internals of the Spark scheduler (aside from the dynamic allocation code). -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org