Yikun commented on a change in pull request #35870: URL: https://github.com/apache/spark/pull/35870#discussion_r837102501
########## File path: docs/running-on-kubernetes.md ########## @@ -1732,6 +1732,95 @@ Spark allows users to specify a custom Kubernetes schedulers. - Create additional Kubernetes custom resources for driver/executor scheduling. - Set scheduler hints according to configuration or existing Pod info dynamically. +#### Using Volcano as Customized Scheduler for Spark on Kubernetes + +**This feature is currently experimental. In future versions, there may be behavioral changes around configuration, feature step improvement.** + +##### Prerequisites +* Spark on Kubernetes with Volcano as a custom scheduler is supported since Spark v3.3.0 and Volcano v1.5.1. +* See also [Volcano installation](https://volcano.sh/en/docs/installation). + +##### Build +To create a Spark distribution along with Volcano suppport like those distributed by the Spark [Downloads page](https://spark.apache.org/downloads.html): + +``` +./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes -Pvolcano Review comment: sure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
