vanzin commented on a change in pull request #23599: [SPARK-24793][K8s] Enhance spark-submit for app management URL: https://github.com/apache/spark/pull/23599#discussion_r264904918
########## File path: docs/running-on-kubernetes.md ########## @@ -403,6 +403,36 @@ RBAC authorization and how to configure Kubernetes service accounts for pods, pl [Using RBAC Authorization](https://kubernetes.io/docs/admin/authorization/rbac/) and [Configure Service Accounts for Pods](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/). +## Spark Application Management + +Like Mesos and standalone, Kubernetes provides simple application management via the spark-submit CLI tool in cluster mode. +Users can kill a job by providing the submission ID that is printed when submitting their job. +The submission ID follows the format ``namespace:driver-pod-name`` and is similar to the driverId found in standalone mode. +If user omits the namespace then the namespace set in current k8s context is used. +For example if user has set a specific namespace as follows `kubectl config set-context minikube --namespace=spark` +then the `spark` namespace will be used by default. On the other hand, if there is no namespace added to the specific context +then all namespaces will be considered by default. That means operations will affect all spark applications. Review comment: > That means operations will affect all spark applications. Not sure I follow? Don't you mean all namespaces? (Which is what the previous sentence says.) If you want to clarify the previous sentence, you have to add stuff here (e.g. "will affect all Spark applications matching the given submission ID regardless of namespace"). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
