Henry Yu created SPARK-27696:
--------------------------------
Summary: kubernetes driver pod not deleted after finish.
Key: SPARK-27696
URL: https://issues.apache.org/jira/browse/SPARK-27696
Project: Spark
Issue Type: Bug
Components: Kubernetes
Affects Versions: 2.4.0
Reporter: Henry Yu
when submit to k8s, driver pod not deleted after job completion.
While k8s checks driver pod name not existing, It is especially painful when we
use workflow tool to resubmit the failed spark job. (by the way, client always
exist with 0 is another painful issue)
I have fix this with a new config
spark.kubernetes.submission.deleteCompletedPod=true in our home maintained
spark version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
