GitHub user paul-rogers opened a pull request:
https://github.com/apache/drill/pull/860
DRILL-5601: Rollup of external sort fixes an improvements
- DRILL-5513: Managed External Sort : OOM error during the merge phase
- DRILL-5519: Sort fails to spill and results in an OOM
- DRILL-5522: OOM during the merge and spill process of the managed
external sort
- DRILL-5594: Excessive buffer reallocations during merge phase of external
sort
- DRILL-5597: Incorrect "bits" vector allocation in nullable vectors
allocateNew()
- DRILL-5602: Repeated List Vector fails to initialize the offset vector
All of the bugs have to do with handling low-memory conditions, and with
correctly estimating the sizes of vectors, even when those vectors come
from the spill file or from an exchange. Hence, the changes for all of
the above issues are interrelated.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/paul-rogers/drill DRILL-5601
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/860.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #860
----
commit 63faa99ba0ef2e51bb18ac5a91cd95101a54ae88
Author: Paul Rogers <[email protected]>
Date: 2017-04-06T20:57:19Z
DRILL-5601: Rollup of external sort fixes an improvements
- DRILL-5513: Managed External Sort : OOM error during the merge phase
- DRILL-5519: Sort fails to spill and results in an OOM
- DRILL-5522: OOM during the merge and spill process of the managed
external sort
- DRILL-5594: Excessive buffer reallocations during merge phase of external
sort
- DRILL-5597: Incorrect "bits" vector allocation in nullable vectors
allocateNew()
- DRILL-5602: Repeated List Vector fails to initialize the offset vector
All of the bugs have to do with handling low-memory conditions, and with
correctly estimating the sizes of vectors, even when those vectors come
from the spill file or from an exchange. Hence, the changes for all of
the above issues are interrelated.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
