Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22754#discussion_r228601036
--- Diff:
core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java
---
@@ -62,6 +62,8 @@ public UnsafeSorterSpillWriter(
int fileBufferSize,
ShuffleWriteMetrics writeMetrics,
int numRecordsToWrite) throws IOException {
+ // Space used by prefix + len + recordLength is more than 4 + 8 bytes
+ assert (diskWriteBufferSize > 12);
--- End diff --
Where is the best place of this comment? I am neutral on this.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
