[
https://issues.apache.org/jira/browse/DRILL-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790794#comment-14790794
]
ASF GitHub Bot commented on DRILL-3732:
---------------------------------------
Github user amansinha100 commented on a diff in the pull request:
https://github.com/apache/drill/pull/147#discussion_r39662237
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
---
@@ -514,13 +516,20 @@ public BatchGroup
mergeAndSpill(LinkedList<BatchGroup> batchGroups) throws Schem
outputContainer.setRecordCount(count);
newGroup.addBatch(outputContainer);
}
+ injector.injectChecked(context.getExecutionControls(),
INTERRUPTION_WHILE_SPILLING, IOException.class);
newGroup.closeOutputStream();
- for (BatchGroup group : batchGroupList) {
- group.cleanup();
- }
- hyperBatch.clear();
+ threw = false;
--- End diff --
I think there is an implicit assumption here that threw = false; statement
will be the last statement in the try {} block. Is that right ? If so, that
could be prone to some future problems if new code gets added here.
> Drill leaks memory if external sort hits out of disk space exception
> --------------------------------------------------------------------
>
> Key: DRILL-3732
> URL: https://issues.apache.org/jira/browse/DRILL-3732
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Affects Versions: 1.2.0
> Reporter: Victoria Markman
> Assignee: Aman Sinha
> Priority: Critical
> Fix For: 1.2.0
>
> Attachments: drillbit.log
>
>
> Ran into it when running CTAS with partition by.
> Here is what reproduction looks like:
> {code}
> 0: jdbc:drill:schema=dfs> create table store_sales_4(ss_item_sk,
> ss_customer_sk, ss_cdemo_sk, ss_hdemo_sk, s_sold_date_sk, ss_promo_sk)
> partition by (ss_promo_sk) as
> . . . . . . . . . . . . > select
> . . . . . . . . . . . . > case when columns[2] = '' then cast(null as
> varchar(100)) else cast(columns[2] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[3] = '' then cast(null as
> varchar(100)) else cast(columns[3] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[4] = '' then cast(null as
> varchar(100)) else cast(columns[4] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[5] = '' then cast(null as
> varchar(100)) else cast(columns[5] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as
> varchar(100)) else cast(columns[0] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[8] = '' then cast(null as
> varchar(100)) else cast(columns[8] as varchar(100)) end
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > `store_sales.dat` ss
> . . . . . . . . . . . . > ;
> Error: SYSTEM ERROR: IllegalStateException: Failure while closing accountor.
> Expected private and shared pools to be set to initial values. However, one
> or more were not. Stats are
> zone init allocated delta
> private 10000000 9680512 319488
> shared 10000000 10000000 0.
> Fragment 1:21
> [Error Id: bd0d7d59-8693-476b-8671-70f0b2e7a176 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Setup:
> single node
> 8GB direct memory
> 4GB heap memory
> store_sales.dat is a file from TPCDS SF100
> drillbit.log attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
