Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1075#discussion_r158150320
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/managed/SortConfig.java
---
@@ -84,7 +85,7 @@ public SortConfig(DrillConfig config) {
if (limit > 0) {
mergeLimit = Math.max(limit, MIN_MERGE_LIMIT);
} else {
- mergeLimit = Integer.MAX_VALUE;
+ mergeLimit = DEFAULT_MERGE_LIMIT;
--- End diff --
There may be a misunderstanding of how config options work. We define the
defaults in Drill's own source code: `drill-module.conf` in each module. (Here
it is in `java-exec`.)
To change the default option, we change the value in `drill-module.conf`.
In the highly unlikely case that a user has overridden this value in
`drill-override.conf`, their value will be used. But, the option is not
documented in `drill-override-example.conf` so it is very, very unlikely that
anyone created an override. (The property is meant to be internal, for use in
tests.)
So, rather than introducing yet another variable, we might as well use the
existing config property. This has the added advantage that, if experience
suggests that we need a smaller or larger limit for some scenarios, we can make
the adjustment in the field via the config system.
---
