Vincent Janelle created SPARK-55120:
---------------------------------------
Summary: Embedding spark-submit
Key: SPARK-55120
URL: https://issues.apache.org/jira/browse/SPARK-55120
Project: Spark
Issue Type: Improvement
Components: Kubernetes, Spark Submit
Affects Versions: 3.1.0
Reporter: Vincent Janelle
When combined with the kubeflow spark operator, the spark-submit process tends
to consume a lot of resources from either concurrent or multiple JVM
executions. The process could be simplified to a single process (design
pending), however the spark-submit process statically assigns two UUID values
for the driver and configmap names.
To reduce the total implementation (ie to track the upstream spark), it would
be advantageous to create these UUID values dynamically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
