Yikun commented on PR #37802: URL: https://github.com/apache/spark/pull/37802#issuecomment-1247701070
> Is there a way to specify a queue dynamically when using volcano to submit spark applicaiton? @zheniantoushipashi Currnently, as you know we can only specify volcano queue by [using PodGroupTemplate](https://github.com/apache/spark/blob/master/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/VolcanoTestsSuite.scala#L255-L260). @dongjoon-hyun meant Spark + Yunikorn can set queue by setting label but volcano couldn't. Actually, there are mainly two style to specify scheduler hint in common scheduler: - Using annotation/label - Using a separate CRD Because in K8S scheduling, the queue is only one of the capabilities, cooporate with other PodGroup information. If it is also placed in the label or annotation, will be very complicated, it is not the same way as [K8S annotation suggests](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#attaching-metadata-to-objects). The queue and other parts of the PodGroup are usually considered at the same time, so the queue is used as a field of the PodGroup in Volcano. Users can configure all batch scheduling through the pod group CRD. Therefore, you might want to `use PodGroupTemplate to cover the whole scene, and then increase flexibility through configuration items` in your own downstream version at current. Also cc @william-wang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
