attilapiros commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641294356
Hi @holdenk! I have opened a new PR for your branch: https://github.com/holdenk/spark/pull/6 I know this seams to be a lot of change but it is really just moving code out from the `BlockManager`. One advantage is testability: now the `BlockManagerDecommissionManager` can be tested quite easily without a `BlockManager`. The other is now the `BlockManagerDecommissionManager` has very clean interface for its dependencies: - `MigratableResolver` to handle Shuffle Blocks - `MigratableRDDResolver` to handle cached RDD - `BlockTransferService` for handling block uploads - `BlockManagerPeerProvider` to find the peers where to migrate those blocks I suggest to quickly check the commits and you will see it is not that risky and as refactors on production code is quite discouraged probably this is the best time to make these steps. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
