gauravkm commented on PR #3261: URL: https://github.com/apache/celeborn/pull/3261#issuecomment-2982430445
> What will happen if split: HARD_SPLIT happened? These checks work end to end. It doesn't matter which workers get written to. As long the reducers read the requisite number of bytes and crc32 matches, data will be accurate > What mechanism guarantees the integrity of data sent from the client to the worker? Celeborn already allows you to enable shuffle data compression that guarantees the integrity of every package exchanged between the client and worker -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
