Yikun opened a new pull request #35436: URL: https://github.com/apache/spark/pull/35436
### What changes were proposed in this pull request? This patch inroduces 5 scheduling related configurations: - `spark.kubernetes.job.queue`: The name of the queue to which the job is submitted. - `spark.kubernetes.job.minCPU`: The minimum CPU for running the job. - `spark.kubernetes.job.minMemory`: The minimum memory for running the job, in MiB unless otherwise specified. - `spark.kubernetes.job.minMember`: The minimum number of pods running in a job. - `spark.kubernetes.job.priorityClassName`: The priority of the running job. These info will be stored in configuration and passed to specified feature step, such as it will be used by Yunikorn/Volcano feature step. ### Why are the changes needed? This PR help user integrate Spark with Volcano Scheduler to enable minResource/queue/priority support. See also: [SPARK-36057](https://issues.apache.org/jira/browse/SPARK-36057) ### Does this PR introduce _any_ user-facing change? Yes, configuration added. ### How was this patch tested? CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
