Dreamstick9 commented on code in PR #57748: URL: https://github.com/apache/spark/pull/57748#discussion_r3714795445
########## docs/configuration.md: ########## @@ -4124,6 +4124,22 @@ Push-based shuffle helps improve the reliability and performance of spark shuffl </td> <td>3.2.0</td> </tr> +<tr> + <td><code>spark.shuffle.push.server.mergedShuffleChecksum.enabled</code></td> + <td><code>true</code></td> + <td> + Whether the external shuffle service calculates the checksum of every chunk of a merged shuffle partition while it merges the pushed blocks. The checksums are stored next to the merged shuffle data and are only used to diagnose the cause of a corrupted shuffle chunk, in the same way <code>spark.shuffle.checksum.enabled</code> is used for the shuffle blocks that are not merged. Diagnosing a corrupted shuffle chunk additionally requires <code>spark.shuffle.checksum.enabled</code> to be set in the application reading the chunk. + </td> + <td>4.3.0</td> Review Comment: hmm nice catch.the version numbers in this PR were out of date. dev/next_version_candidates.py now shows 4.4.0 for `branch-4.x`, and `branch-4.3` is cut. I changed the two configuration rows and the `@since` tag to 4.4.0.This change is a new feature. Therefore, it goes to `master` and `branch-4.x` only. I do not backport it to `branch-4.3`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
