[
https://issues.apache.org/jira/browse/IMPALA-15432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noémi Pap-Takács resolved IMPALA-15432.
---------------------------------------
Resolution: Fixed
> [DOCS] Fix and extend the EXPAND_COMPLEX_TYPES default-change documentation
> ---------------------------------------------------------------------------
>
> Key: IMPALA-15432
> URL: https://issues.apache.org/jira/browse/IMPALA-15432
> Project: IMPALA
> Issue Type: Documentation
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 5.0.0
>
>
> IMPALA-15054 changed the default of EXPAND_COMPLEX_TYPES from FALSE to TRUE
> and documented it. Three problems remain.
> (a) The incompatible-changes entry has a wrong example.
> impala_incompatible_changes.xml, concept incompatible_changes_500x, says the
> failure case is "a complex column whose type is not supported in the select
> list (for example a STRUCT nested inside a collection)". Mixed complex types
> in the select list have been supported since Impala
> 4.3 (IMPALA-9551), so that is not a failure case. The cases that really fail
> under the new default:
> - CREATE TABLE AS SELECT * and INSERT ... SELECT * from a table with complex
> columns: "... because the column '<c>' has a complex type '<t>' and Impala
> doesn't support inserting complex types" (InsertStmt)
> - SELECT DISTINCT * and GROUP BY over an expanded collection: "SELECT
> DISTINCT expression '<e>' with type '<t>' is not supported."
> (Type.throwIfNotComparable)
> - UNION, INTERSECT or EXCEPT without ALL: "UNION, EXCEPT and INTERSECT with
> type '<t>' is not supported." (SetOperationStmt)
> - SELECT * ... ORDER BY when a STRUCT contains a collection: "Sorting is not
> supported if the select list contains collection(s) nested in struct(s)."
> (QueryStmt.checkTypeValidityForSorting)
> - STRUCT columns in tables that are not Parquet or ORC, e.g. Paimon ROW
> columns or RCFile/SequenceFile partitions: "Querying STRUCT is only supported
> for ORC and Parquet file formats." (SlotRef)
> Also worth a sentence: unnest joins such as SELECT * FROM t, t.arr now
> return the parent's complex columns too, so clients that read columns by
> position are affected. Keep the SET EXPAND_COMPLEX_TYPES=false workaround and
> add "list the columns explicitly" as the alternative.
> (b) The option page doesn't record the version.
> impala_expand_complex_types.xml says only "Default: TRUE / Added in: Impala
> 4.2". Add "The default changed from FALSE to TRUE in Impala 5.0" (the pattern
> used by impala_default_file_format.xml) and xref the 5.0.x
> incompatible-changes entry.
> (c) Two stale spots in impala_complex_types.xml.
> - Around line 2109, in the view example: "SELECT * would bring back all the
> data, unlike SELECT * queries on the original table with complex type
> columns". SELECT * on the base table now returns the complex columns too.
> - The "not supported in the select list" list (STRUCT<a: ARRAY<INT>>,
> ARRAY<STRUCT<i: INT>>, MAP<INT, STRUCT<s: STRING>>) and the sentence above it
> about collections embedded in structs. Stale since 4.3, and now actively
> misleading, because SELECT * expands exactly these columns by default. The
> real remaining limits: no sorting of
> collections nested in structs, no DISTINCT or set operations over
> collections, and STRUCT only for Parquet/ORC.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
