mridulm commented on PR #46934: URL: https://github.com/apache/spark/pull/46934#issuecomment-2175063071
@gaoyajun02 , trying to understand the scenario better here - did you observe disk issues which resulted in this inconsistency ? If yes, should this be checksum'ed - to ensure correctness. I would prefer that to adding an additional rpc calls to the driver ? Given this should be a rare enough scenario. Thoughts ? Also, +CC @zhouyejoe as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
