wankunde commented on PR #37922: URL: https://github.com/apache/spark/pull/37922#issuecomment-1250572130
Hi, @mridulm Thanks for your review. In my opinion, I do not think we should add `ShuffleMergeId` in RemoveShuffleMerge. The push-based shuffle service will only keep the latest shuffleMergeId data for each shuffle, and RemoveShuffleMerge will cleanup its data. Just like cleaning up normal shuffle data written by mapTasks, these methods only need shuffleId: `def removeShuffle(shuffleId: Int)` What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
