[ https://issues.apache.org/jira/browse/DRILL-5284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15883485#comment-15883485 ]
ASF GitHub Bot commented on DRILL-5284: --------------------------------------- GitHub user paul-rogers opened a pull request: https://github.com/apache/drill/pull/761 DRILL-5284: Roll-up of final fixes for managed sort See subtasks for details. * Provide detailed, accurate estimate of size consumed by a record batch * Managed external sort spills too often with Parquet data * Managed External Sort fails with OOM * External sort refers to the deprecated HDFS fs.default.name param * Config param drill.exec.sort.external.batch.size is not used * NPE in managed external sort while spilling to disk * External Sort BatchGroup leaks memory if an OOM occurs during read * Ensure at least two batches are merged in low-memory conditions You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-rogers/drill DRILL-5284 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/761.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #761 ---- commit 5558e9439805d595cfc9625591a276385454625f Author: Paul Rogers <prog...@maprtech.com> Date: 2017-02-24T18:31:25Z DRILL-5284: Roll-up of final fixes for managed sort See subtasks for details. * Provide detailed, accurate estimate of size consumed by a record batch * Managed external sort spills too often with Parquet data * Managed External Sort fails with OOM * External sort refers to the deprecated HDFS fs.default.name param * Config param drill.exec.sort.external.batch.size is not used * NPE in managed external sort while spilling to disk * External Sort BatchGroup leaks memory if an OOM occurs during read commit 0028f26fef5d9b462700a28b689d47241ee3a1ce Author: Paul Rogers <prog...@maprtech.com> Date: 2017-02-24T20:45:18Z Fix for DRILL-5294 Under certain low-memory conditions, need to force the sort to merge two batches to make progress, even though this is a bit more than comfortably fits into memory. ---- > Roll-up of final fixes for managed sort > --------------------------------------- > > Key: DRILL-5284 > URL: https://issues.apache.org/jira/browse/DRILL-5284 > Project: Apache Drill > Issue Type: Bug > Affects Versions: 1.10.0 > Reporter: Paul Rogers > Assignee: Paul Rogers > Fix For: 1.10.0 > > > The managed external sort was introduced in DRILL-5080. Since that time, > extensive testing has identified a number of minor fixes and improvements. > Given the long PR cycles, it is not practical to spend a week or two to do a > PR for each fix individually. This ticket represents a roll-up of a > combination of a number of fixes. Small fixes are listed here, larger items > appear as sub-tasks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)