abhishekd0907 commented on pull request #35683: URL: https://github.com/apache/spark/pull/35683#issuecomment-1081453295
> What happens to nodes which are getting decomissioned, which have shuffle data on them (but no active executors) ? @mridulm If decommissioning is enabled, shuffle blocks are already migrated to another node before the executor exits. However, this is best effort. So shuffle data might be left if shuffle blocks migration is left incomplete or the executor doesn't exit cleanly. Can you please clarify what are the expectations for shuffle data when a node is put in decommissioning state? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
