[
https://issues.apache.org/jira/browse/DRILL-5284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15883485#comment-15883485
]
ASF GitHub Bot commented on DRILL-5284:
---------------------------------------
GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/761
DRILL-5284: Roll-up of final fixes for managed sort
See subtasks for details.
* Provide detailed, accurate estimate of size consumed by a record batch
* Managed external sort spills too often with Parquet data
* Managed External Sort fails with OOM
* External sort refers to the deprecated HDFS fs.default.name param
* Config param drill.exec.sort.external.batch.size is not used
* NPE in managed external sort while spilling to disk
* External Sort BatchGroup leaks memory if an OOM occurs during read
* Ensure at least two batches are merged in low-memory conditions
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/paul-rogers/drill DRILL-5284
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/761.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #761
----
commit 5558e9439805d595cfc9625591a276385454625f
Author: Paul Rogers <[email protected]>
Date: 2017-02-24T18:31:25Z
DRILL-5284: Roll-up of final fixes for managed sort
See subtasks for details.
* Provide detailed, accurate estimate of size consumed by a record batch
* Managed external sort spills too often with Parquet data
* Managed External Sort fails with OOM
* External sort refers to the deprecated HDFS fs.default.name param
* Config param drill.exec.sort.external.batch.size is not used
* NPE in managed external sort while spilling to disk
* External Sort BatchGroup leaks memory if an OOM occurs during read
commit 0028f26fef5d9b462700a28b689d47241ee3a1ce
Author: Paul Rogers <[email protected]>
Date: 2017-02-24T20:45:18Z
Fix for DRILL-5294
Under certain low-memory conditions, need to force the sort to merge
two batches to make progress, even though this is a bit more than
comfortably fits into memory.
----
> Roll-up of final fixes for managed sort
> ---------------------------------------
>
> Key: DRILL-5284
> URL: https://issues.apache.org/jira/browse/DRILL-5284
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.10.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Fix For: 1.10.0
>
>
> The managed external sort was introduced in DRILL-5080. Since that time,
> extensive testing has identified a number of minor fixes and improvements.
> Given the long PR cycles, it is not practical to spend a week or two to do a
> PR for each fix individually. This ticket represents a roll-up of a
> combination of a number of fixes. Small fixes are listed here, larger items
> appear as sub-tasks.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
