GitHub user paul-rogers opened a pull request: https://github.com/apache/drill/pull/761
DRILL-5284: Roll-up of final fixes for managed sort See subtasks for details. * Provide detailed, accurate estimate of size consumed by a record batch * Managed external sort spills too often with Parquet data * Managed External Sort fails with OOM * External sort refers to the deprecated HDFS fs.default.name param * Config param drill.exec.sort.external.batch.size is not used * NPE in managed external sort while spilling to disk * External Sort BatchGroup leaks memory if an OOM occurs during read * Ensure at least two batches are merged in low-memory conditions You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-rogers/drill DRILL-5284 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/761.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #761 ---- commit 5558e9439805d595cfc9625591a276385454625f Author: Paul Rogers <prog...@maprtech.com> Date: 2017-02-24T18:31:25Z DRILL-5284: Roll-up of final fixes for managed sort See subtasks for details. * Provide detailed, accurate estimate of size consumed by a record batch * Managed external sort spills too often with Parquet data * Managed External Sort fails with OOM * External sort refers to the deprecated HDFS fs.default.name param * Config param drill.exec.sort.external.batch.size is not used * NPE in managed external sort while spilling to disk * External Sort BatchGroup leaks memory if an OOM occurs during read commit 0028f26fef5d9b462700a28b689d47241ee3a1ce Author: Paul Rogers <prog...@maprtech.com> Date: 2017-02-24T20:45:18Z Fix for DRILL-5294 Under certain low-memory conditions, need to force the sort to merge two batches to make progress, even though this is a bit more than comfortably fits into memory. ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---