cloud-fan commented on code in PR #44709:
URL: https://github.com/apache/spark/pull/44709#discussion_r1450537255
##########
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java:
##########
@@ -150,11 +150,21 @@ public long[] getChecksums() {
* Sorts the in-memory records and writes the sorted records to an on-disk
file.
* This method does not free the sort data structures.
*
- * @param isLastFile if true, this indicates that we're writing the final
output file and that the
- * bytes written should be counted towards shuffle spill
metrics rather than
- * shuffle write metrics.
+ * @param isFinalFile if true, this indicates that we're writing the final
output file and that
+ * the bytes written should be counted towards shuffle
write metrics rather
Review Comment:
the comment was wrong before. If this flag is true, we are writing the final
shuffle output file and will increase the shuffle write metrics rather than the
spill metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
