kezhenxu94 opened a new issue, #13511: URL: https://github.com/apache/dolphinscheduler/issues/13511
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement. ### Description In [the Spark doc](https://spark.apache.org/docs/latest/running-on-kubernetes.html), now we can directly submit jobs to Kubernetes and schedule the jobs in Pods, so users can directly use the Kubernetes and don't need to deploy yarn anymore. However DolphinScheduler currently hardcoded the master to yarn (`--master yarn`) https://github.com/apache/dolphinscheduler/blob/0ee82c5492167f01e4ee5f1d55f2f248181c65fe/dolphinscheduler-task-plugin/dolphinscheduler-task-spark/src/main/java/org/apache/dolphinscheduler/plugin/task/spark/SparkTask.java#L131-L139 In order to submit jobs to Kubernetes, the `--master` option must be configurable so users can specify it as the Kubernetes API server address. ### Use case There are at least 2 use cases: - DolphinScheduler is not deployed inside Kubernetes, but users want to submit jobs to Kubernetes cluster. - DolphinScheduler is also deployed inside Kubernetes, and users also want to submit jobs to Kubernetes cluster. ### Related issues https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue+is%3Aopen+spark+kubernetes ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
