Github user Ben-Zvi commented on a diff in the pull request:
https://github.com/apache/drill/pull/932#discussion_r137963963
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/managed/SortMemoryManager.java
---
@@ -680,9 +680,11 @@ public MergeTask consolidateBatches(long allocMemory,
int inMemCount, int spille
// spilled run.
// Maximum spill batches that fit into available memory.
+ // Use the maximum buffer size since spill batches seem to
+ // be read with almost 50% internal fragmentation.
int memMergeLimit = (int) ((mergeMemoryLimit - allocMemory) /
- spillBatchSize.expectedBufferSize);
+ spillBatchSize.maxBufferSize);
memMergeLimit = Math.max(0, memMergeLimit);
--- End diff --
This line of code is not needed ( Math.max(....) ), as the code above
(around line 675) ensures that mergeMemoryLimit >= allocMemory .
---
