Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20014#discussion_r157818026
--- Diff:
core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ---
@@ -341,7 +342,7 @@ private void growPointerArrayIfNecessary() throws
IOException {
// should have trigger spilling
if (!inMemSorter.hasSpaceForAnotherRecord()) {
logger.error("Unable to grow the pointer array");
- throw e;
+ throw new SparkOutOfMemoryError(e);
--- End diff --
Should `allocateArray` also throw `SparkOutOfMemoryError` then we can just
rethrow it here?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
