[
https://issues.apache.org/jira/browse/IMPALA-12695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812986#comment-17812986
]
ASF subversion and git services commented on IMPALA-12695:
----------------------------------------------------------
Commit 825e059dc5c1bf325a61b588f1d9615d569ee04b in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=825e059dc ]
IMPALA-12695: Crash with UNION with complex types
If we unnest an array coming from a UNION ALL, we read invalid memory
and in ASAN builds we crash.
Example:
with v as (select arr1 from complextypes_arrays
union all select arr1 from complextypes_arrays)
select am.item from v, v.arr1 am;
The problem seems to be that in the item tuple of the collections, the
item slots are present twice. This is because both the inline view
analyzer and the main analyzer add slots with the same path to the
tuple. This is possible because
- the target tuple is determined based on the path via
Path.getRootDesc(), so it will be the same both in the inline view
and in the main scope
AND
- the inline view analyzer and the main one do not share
'slotPathMap_', so the analyzer cannot recognise that a slot for the
path has already been added.
This commit solves the problem by checking the target tuple whether a
slot with the same path already exists in it, and if it does, we reuse
that slot. Note, however, that when Analyzer.registerSlotRef() is called
with 'duplicateIfCollections=true', a separate slot is added for
collections which should not be reused. This commit adds a set,
'duplicateCollectionSlots', in Analyzer.GlobalState to keep track of
such collection slots, and these slots are never reused.
Note that there is another bug, IMPALA-12753, that a predicate on the
collection item in the above query is only enforced on the first child
of the union. Therefore this commit disallows placing a predicate on a
collection item when the unnested collection comes from a union.
Testing:
- added test queries in nested-array-in-select-list.test,
nested-map-in-select-list.test, zipping-unnest-in-from-clause.test
and zipping-unnest-in-select-list.test
Change-Id: I340adc50e6d7cda6f59dacd7a46b6adc31635d46
Reviewed-on: http://gerrit.cloudera.org:8080/20953
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Crash with UNION with complex types
> -----------------------------------
>
> Key: IMPALA-12695
> URL: https://issues.apache.org/jira/browse/IMPALA-12695
> Project: IMPALA
> Issue Type: Bug
> Reporter: Daniel Becker
> Assignee: Daniel Becker
> Priority: Major
>
> The following query crashes:
> {code:java}
> with tmp as (select all_mix from collection_struct_mix
> union all select all_mix from collection_struct_mix)
> select value from tmp, tmp.all_mix am;{code}
> We should find out the cause and fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
