[
https://issues.apache.org/jira/browse/IMPALA-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy resolved IMPALA-13364.
----------------------------------------
Fix Version/s: Impala 4.5.0
Resolution: Fixed
> Schema resolution doesn't work for migrated partitioned Iceberg tables that
> have complex types
> ----------------------------------------------------------------------------------------------
>
> Key: IMPALA-13364
> URL: https://issues.apache.org/jira/browse/IMPALA-13364
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 4.5.0
>
>
> Schema resolution doesn't work correctly for migrated partitioned Iceberg
> tables that have complex types.
> When we face a Parquet/ORC file in an Iceberg table that doesn't have field
> IDs in the file metadata, we assume that it is an old data file before
> migration, and the schema is the very first one, hence we can mimic Iceberg's
> field ID generation to assign field IDs to the file schema elements.
> This process didn't take the partition columns into account. This only
> matters when there are complex types in the table, as partition columns are
> always the last columns in legacy Hive tables, and field IDs are assigned via
> a "BFS-like" traversal. I.e. if there are only primitive types in the table
> we don't have any problems, but the children of complex types columns are
> assigned incorrectly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
