Github user juliuszsompolski commented on a diff in the pull request:
https://github.com/apache/spark/pull/19181#discussion_r141842918
--- Diff:
core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java
---
@@ -503,6 +511,39 @@ public void testGetIterator() throws Exception {
verifyIntIterator(sorter.getIterator(279), 279, 300);
}
+ @Test
+ public void testOOMDuringSpill() throws Exception {
+ final UnsafeExternalSorter sorter = newSorter();
+ for (int i = 0; sorter.hasSpaceForAnotherRecord(); ++i) {
+ insertNumber(sorter, i);
+ }
+ // todo: this might actually not be zero if pageSize is somehow
configured differently,
+ // so we actually have to compute the expected spill size according to
the configured page size
+ assertEquals(0, sorter.getSpillSize());
--- End diff --
If this might actually not be zero, maybe don't test this assertion?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
