[
https://issues.apache.org/jira/browse/DRILL-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731643#comment-14731643
]
ASF GitHub Bot commented on DRILL-3732:
---------------------------------------
GitHub user adeneche opened a pull request:
https://github.com/apache/drill/pull/147
DRILL-3732: Drill leaks memory if external sort hits out of disk spac…
…e exception
- ExternalSort.mergeAndSpill() cleans all it's data in case an errors
occurs while it's spilling to disk
- made BatchGroup AutoCloseable so it can easily be closed with
AutoCloseables.close() if an error occurs
- added injection site while External sort is spilling to disk
- added unit test that forces a 2 batch query to spill to disk and injects
an exception while it does so
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/adeneche/incubator-drill DRILL-3732
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/147.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #147
----
commit a113d9a5e5d434715159c01e97ff7ffa4bad9b38
Author: adeneche <[email protected]>
Date: 2015-09-05T00:10:17Z
DRILL-3732: Drill leaks memory if external sort hits out of disk space
exception
- ExternalSort.mergeAndSpill() cleans all it's data in case an errors
occurs while it's spilling to disk
- made BatchGroup AutoCloseable so it can easily be closed with
AutoCloseables.close() if an error occurs
- added injection site while External sort is spilling to disk
- added unit test that forces a 2 batch query to spill to disk and injects
an exception while it does so
----
> Drill leaks memory if external sort hits out of disk space exception
> --------------------------------------------------------------------
>
> Key: DRILL-3732
> URL: https://issues.apache.org/jira/browse/DRILL-3732
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Affects Versions: 1.2.0
> Reporter: Victoria Markman
> Assignee: Deneche A. Hakim
> Priority: Critical
> Fix For: 1.2.0
>
> Attachments: drillbit.log
>
>
> Ran into it when running CTAS with partition by.
> Here is what reproduction looks like:
> {code}
> 0: jdbc:drill:schema=dfs> create table store_sales_4(ss_item_sk,
> ss_customer_sk, ss_cdemo_sk, ss_hdemo_sk, s_sold_date_sk, ss_promo_sk)
> partition by (ss_promo_sk) as
> . . . . . . . . . . . . > select
> . . . . . . . . . . . . > case when columns[2] = '' then cast(null as
> varchar(100)) else cast(columns[2] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[3] = '' then cast(null as
> varchar(100)) else cast(columns[3] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[4] = '' then cast(null as
> varchar(100)) else cast(columns[4] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[5] = '' then cast(null as
> varchar(100)) else cast(columns[5] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as
> varchar(100)) else cast(columns[0] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[8] = '' then cast(null as
> varchar(100)) else cast(columns[8] as varchar(100)) end
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > `store_sales.dat` ss
> . . . . . . . . . . . . > ;
> Error: SYSTEM ERROR: IllegalStateException: Failure while closing accountor.
> Expected private and shared pools to be set to initial values. However, one
> or more were not. Stats are
> zone init allocated delta
> private 10000000 9680512 319488
> shared 10000000 10000000 0.
> Fragment 1:21
> [Error Id: bd0d7d59-8693-476b-8671-70f0b2e7a176 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Setup:
> single node
> 8GB direct memory
> 4GB heap memory
> store_sales.dat is a file from TPCDS SF100
> drillbit.log attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
