yangwwei opened a new pull request #35663: URL: https://github.com/apache/spark/pull/35663
### What changes were proposed in this pull request? Add yunikorn feature step in order to support yunikorn scheduler. [Apache yunikorn](https://yunikorn.apache.org/) is an open-source batch scheduler for K8s, designed to solve the problems of running batch workloads, e.g Spark, on K8s. This PR will add a build profile yunikorn, the yunikorn feature step, UT, and integration tests. ### Why are the changes needed? This is a part of the effort [SPARK-36057](https://issues.apache.org/jira/browse/SPARK-36057), in order to make Spark natively support customized K8s schedulers. This PR particularly is similar to https://github.com/apache/spark/pull/35422. ### Does this PR introduce _any_ user-facing change? Once this is done, user can easily submit their jobs to yunikorn scheduler with the following configs: ``` --conf spark.kubernete.driver.scheduler.name=yunikorn \ --conf spark.kubernetes.driver.pod.featureSteps=org.apache.spark.deploy.k8s.features.scheduler.YuniKornFeatureStep ``` on a yunikorn enabled cluster, the Spark job will be scheduled by the yunikorn scheduler instead of the default scheduler. ### How was this patch tested? To be added. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
