Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19184#discussion_r137980253
--- Diff:
core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java
---
@@ -104,6 +124,10 @@ public void loadNext() throws IOException {
if (taskContext != null) {
taskContext.killTaskIfInterrupted();
}
+ if (this.din == null) {
+ // Good time to init (if all files are opened, we can get Too Many
files exception)
+ initStreams();
+ }
--- End diff --
I think you need to first describe more about how to fix this issue in the
description.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
