q2w commented on pull request #32677: URL: https://github.com/apache/spark/pull/32677#issuecomment-860602779
> This looks like a good direction to go. But given the decommission is only a best effort, I don't think it worth such complexity. > > Maybe, making the StorageLevel as DISK_ONLY (with a conf option) for the migration blocks should be enough? @Ngone51 Thanks for taking a look. I have opened a separate PR for making StorageLevel to DISK_ONLY with a conf. https://github.com/apache/spark/pull/32902. Please have a look. On the current PR, a part of the PR is just focusing on transferring a flag to peer block managers to distinguish request for genuine replication and block migration. Should we continue with this in part? This will enable us to put more checks on peer block managers to restrict block migration(like putting limit on disk used for migrating block etc.). Would this be useful? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
