cxzl25 commented on PR #39918: URL: https://github.com/apache/spark/pull/39918#issuecomment-1420242612
Thanks everyone for the quick review. I want to add a few more information checksums so that we can analyze the cause of corruption. Like this ```bash 23/02/07 13:56:13.833 main INFO ShuffleChecksumHelper: Shuffle corruption diagnosis took 0 ms, checksum file /private/var/folders/49/rnfrb9c53sg3z4f3fjyqfvz00000gp/T/spark-124b60fb-1de0-4893-8140-544029128392/shuffle_0_0_0.checksum.ADLER32, cause DISK_ISSUE, checksumByReader 196609, checksumByWriter 196608, checksumByReCalculation 196609 23/02/07 13:58:28.002 main INFO ShuffleChecksumHelper: Shuffle corruption diagnosis took 0 ms, checksum file /private/var/folders/49/rnfrb9c53sg3z4f3fjyqfvz00000gp/T/spark-5d2ab0e1-6f7f-437e-902c-0e6ac247a0ea/shuffle_0_0_0.checksum.ADLER32, cause NETWORK_ISSUE, checksumByReader 196608, checksumByWriter 196609, checksumByReCalculation 196609 23/02/07 13:58:59.072 main INFO ShuffleChecksumHelper: Shuffle corruption diagnosis took -1 ms, checksum file /private/var/folders/49/rnfrb9c53sg3z4f3fjyqfvz00000gp/T/spark-ceaeb1b3-efb2-4ef1-9f42-28387222d948/shuffle_0_0_0.checksum.ADLER32, cause UNKNOWN_ISSUE, checksumByReader 196609, checksumByWriter -1, checksumByReCalculation -1 23/02/07 13:58:59.072 main INFO ShuffleChecksumHelper: Shuffle corruption diagnosis took -1 ms, checksum file /private/var/folders/49/rnfrb9c53sg3z4f3fjyqfvz00000gp/T/spark-ceaeb1b3-efb2-4ef1-9f42-28387222d948/shuffle_0_0_0.checksum.ADLER32, cause UNKNOWN_ISSUE, checksumByReader 196609, checksumByWriter -1, checksumByReCalculation -1 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
