Github user sitalkedia commented on a diff in the pull request:
https://github.com/apache/spark/pull/20014#discussion_r157841590
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -341,7 +342,7 @@ private void growPointerArrayIfNecessary() throws
IOException {
// should have trigger spilling
if (!inMemSorter.hasSpaceForAnotherRecord()) {
logger.error("Unable to grow the pointer array");
- throw e;
+ throw new SparkOutOfMemoryError(e);
--- End diff --
Actually, it already throws `SparkOutOfMemoryError`, so we can just rethrow
here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
