[
https://issues.apache.org/jira/browse/DRILL-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15102133#comment-15102133
]
Deneche A. Hakim commented on DRILL-4274:
-----------------------------------------
Another error we saw in the same "wide table" tests:
{noformat}
/root/drillAutomation/framework-master/framework/resources/Functional/data-shapes/wide-columns/5000/1000rows/parquet/q168_DRILL-2046.q
Query:
select ws1.* from widestrings ws1 INNER JOIN (select str_var_null_empty from
widestrings where str_var_null_empty is not null and length(str_var_null_empty)
<> 0 )ws2 on ws1.str_null=ws2.str_var_null_empty where ws1.str_null is not null
and length(ws1.str_null) <> 0
Failed with exception
java.sql.SQLException: SYSTEM ERROR: DrillRuntimeException: Failed to
pre-allocate memory for SV. Existing recordCount*4 = 0, incoming batch
recordCount*4 = 676
{noformat}
I debugged this one, and if I remember correctly, the sort exceeds it's limit
on the first batch, tries to spill but doesn't (because mergeAndSpill()
requires at least 2 batches to spill). Later, it finds out it didn't spill and
assumes it's because it can merge the batches in memory but fails with this
exception.
> ExternalSort doesn't always handle low memory condition well, failing
> execution instead of spilling in some cases
> -----------------------------------------------------------------------------------------------------------------
>
> Key: DRILL-4274
> URL: https://issues.apache.org/jira/browse/DRILL-4274
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Reporter: Jacques Nadeau
> Assignee: Jacques Nadeau
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
