GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20722
[SPARK-23571][K8S] Delete auxiliary Kubernetes resources upon application
completion
## What changes were proposed in this pull request?
Spark on Kubernetes creates some auxiliary Kubernetes resources such as the
headless driver service used by the executors to connect to the driver and the
ConfigMap that carries configuration for the init-container. Such resources are
no longer needed once an application completes, but they are still persisted by
the API server and there should be deleted upon completion. This PR handles the
cases when the submission waits for the application to complete, which is the
default behavior as `spark.kubernetes.submission.waitAppCompletion` defaults to
`true`.
## How was this patch tested?
Unit tested.
@felixcheung @foxish
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liyinan926/spark-k8s SPARK-23571
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20722.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20722
----
commit 9890ebc543eb226f395391fb14b877d4b1bf0560
Author: Yinan Li <ynli@...>
Date: 2018-03-02T19:57:35Z
[SPARK-23571][K8S] Delete auxiliary Kubernetes resources upon application
completion
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
