Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137981394
--- Diff:
core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java
---
@@ -72,12 +77,27 @@ public UnsafeSorterSpillReader(
bufferSizeBytes = DEFAULT_BUFFER_SIZE_BYTES;
}
+ try (InputStream bs = new NioBufferedFileInputStream(file, (int)
bufferSizeBytes);
--- End diff --
Please add a comment here to say we don't need to hold the file open until
we actually want to load the records.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
