Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19862#discussion_r154567585
--- Diff:
sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java
---
@@ -182,18 +183,14 @@ public UnsafeRow next() {
}
};
} catch (IOException e) {
- cleanupResources();
throw e;
+ } finally {
+ // Since we won't ever call next() on an empty iterator, we need to
clean up resources
+ // here in order to prevent memory leaks.
+ cleanupResources();
--- End diff --
This makes the resource cleaned up when we return iterator too.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
