[
https://issues.apache.org/jira/browse/DRILL-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850645#comment-15850645
]
ASF GitHub Bot commented on DRILL-5080:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/717#discussion_r99037413
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java ---
@@ -63,13 +63,29 @@
String SPOOLING_BUFFER_DELETE = "drill.exec.buffer.spooling.delete";
String SPOOLING_BUFFER_MEMORY = "drill.exec.buffer.spooling.size";
String BATCH_PURGE_THRESHOLD = "drill.exec.sort.purge.threshold";
+
+ // External Sort Boot configuration
+
String EXTERNAL_SORT_TARGET_BATCH_SIZE =
"drill.exec.sort.external.batch.size";
String EXTERNAL_SORT_TARGET_SPILL_BATCH_SIZE =
"drill.exec.sort.external.spill.batch.size";
String EXTERNAL_SORT_SPILL_GROUP_SIZE =
"drill.exec.sort.external.spill.group.size";
String EXTERNAL_SORT_SPILL_THRESHOLD =
"drill.exec.sort.external.spill.threshold";
String EXTERNAL_SORT_SPILL_DIRS =
"drill.exec.sort.external.spill.directories";
String EXTERNAL_SORT_SPILL_FILESYSTEM =
"drill.exec.sort.external.spill.fs";
+ String EXTERNAL_SORT_SPILL_FILE_SIZE =
"drill.exec.sort.external.spill.file_size";
String EXTERNAL_SORT_MSORT_MAX_BATCHSIZE =
"drill.exec.sort.external.msort.batch.maxsize";
+ String EXTERNAL_SORT_DISABLE_MANAGED =
"drill.exec.sort.external.disable_managed";
+ String EXTERNAL_SORT_MERGE_LIMIT =
"drill.exec.sort.external.merge_limit";
+ String EXTERNAL_SORT_MIN_SPILL =
"drill.exec.sort.external.spill.min_batches";
+ String EXTERNAL_SORT_MAX_SPILL =
"drill.exec.sort.external.spill.max_batches";
+ String EXTERNAL_SORT_MAX_MEMORY = "drill.exec.sort.external.mem_limit";
+ String EXTERNAL_SORT_BATCH_LIMIT =
"drill.exec.sort.external.batch_limit";
+
+ // External Sort Runtime options
+
+ BooleanValidator EXTERNAL_SORT_DISABLE_MANAGED_OPTION = new
BooleanValidator("exec.sort.disable_managed", false);
--- End diff --
True. In the 1.10 release, this will change to "true". But, to prevent
instability in this initial checkin, the old sort is left as the default. Of
course, this PR has taken a while, so the new PR to set this to "true" may come
only a few days after this PR is finally accepted...
> Create a memory-managed version of the External Sort operator
> -------------------------------------------------------------
>
> Key: DRILL-5080
> URL: https://issues.apache.org/jira/browse/DRILL-5080
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.8.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Fix For: 1.10.0
>
> Attachments: ManagedExternalSortDesign.pdf
>
>
> We propose to create a "managed" version of the external sort operator that
> works to a clearly-defined memory limit. Attached is a design specification
> for the work.
> The project will include fixing a number of bugs related to the external
> sort, include as sub-tasks of this umbrella task.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
