mridulm commented on pull request #35683: URL: https://github.com/apache/spark/pull/35683#issuecomment-1083361565
To clarify, I can see a need for supporting node decomissioning in yarn - for both on-prem clusters and for cloud deployments - but the solution will need to take into account what the cluster manager supports. As I detailed above, yarn supports shuffle services and executors come and go on different nodes. Decomissioning will need to account for it, which is a different model compared to k8s support. @tgravescs might have insights into how Tez/MR handles decomissioning w.r.t shuffle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
