zuston commented on PR #35683: URL: https://github.com/apache/spark/pull/35683#issuecomment-1141631786
Thanks for @abhishekd0907 to submit this PR. I think this is a good improvement for the Spark jobs' stability, especially in the scenario where we implement elastic Yarn NMs on K8s. > we do not want to move shuffle blocks around if decomissioning is enabled when external shuffle service is in use. > @mridulm But in our internal cluster, we deployed the ess on Yarn nodemangers, and sometimes nm will be gracefully decommissioned, it means that the ess will be invalid on these nms. So why not allow trigger decommission event to executors when ess is in use? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
