Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1075#discussion_r157830252
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/managed/SortConfig.java
---
@@ -84,7 +85,7 @@ public SortConfig(DrillConfig config) {
if (limit > 0) {
mergeLimit = Math.max(limit, MIN_MERGE_LIMIT);
} else {
- mergeLimit = Integer.MAX_VALUE;
+ mergeLimit = DEFAULT_MERGE_LIMIT;
--- End diff --
The merge limit is already a config option. (I'd forgotten about that.) The
comment on the config option says "Limit on the number of spilled batches that
can be merged in a single pass." So, let's just set that default (in
`drill-override-conf`) to your new value of 128 and leave the code here
unchanged.
---
