[
https://issues.apache.org/jira/browse/IMPALA-14290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18018124#comment-18018124
]
ASF subversion and git services commented on IMPALA-14290:
----------------------------------------------------------
Commit edd5ff6e2a43ec38115eb6e5125f996e63b3cfec in impala's branch
refs/heads/master from Daniel Vanko
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=edd5ff6e2 ]
IMPALA-14290: Make Iceberg partitioning column names case insensitive
When creating or altering partitions of Iceberg tables, Impala only
accepts column names if they are in lowercase and throws
ImpalaRuntimeException otherwise.
This patch allows the usage of other cases as well in PARTITION SPEC
clauses. IcebergPartitionField converts field names to lower case in its
constructor, similar to ColumnDef and PartitionKeyValue.
Testing:
* ran existing tests
* add new test with mixed letter case columns
Change-Id: I4080a6b7468fff940435e2e780322d4ba1f0de49
Reviewed-on: http://gerrit.cloudera.org:8080/23334
Reviewed-by: Daniel Becker <daniel.bec...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Iceberg partitioning should be case insensitive of column names
> ---------------------------------------------------------------
>
> Key: IMPALA-14290
> URL: https://issues.apache.org/jira/browse/IMPALA-14290
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Dániel Gábor Vankó
> Assignee: Dániel Gábor Vankó
> Priority: Major
> Labels: impala-iceberg
>
> When creating or altering partitions of Iceberg tables, Impala only accepts
> column names if they are in lower case.
> E.g. START_TIME will throw exception in the PARTITIONED BY SPEC clause. (Also
> with YEAR, MONTH, TRUNCATE and BUCKET transforms as well.)
> {noformat}
> [localhost:21050] default> CREATE TABLE tab ( START_TIME TIMESTAMP )
> PARTITIONED BY SPEC (DAY(START_TIME))
> STORED AS ICEBERG;
> Query: CREATE TABLE tab ( START_TIME TIMESTAMP )
> PARTITIONED BY SPEC (DAY(START_TIME))
> STORED AS ICEBERG
> 2025-08-06 12:50:04 [Exception] ERROR: Query
> b7450d257090b184:8cb6c76000000000 failed:
> ImpalaRuntimeException: Error making 'createTable' RPC to Hive Metastore:
> CAUSED BY: IllegalArgumentException: Cannot find source column: START_TIME
> {noformat}
> The table is successfully created if start_time is lowercase in the
> PARTITIONED BY SPEC, but still uppercase in the column definition list:
> {noformat}
> [localhost:21050] default> CREATE TABLE tab ( START_TIME TIMESTAMP )
> PARTITIONED BY SPEC (DAY(start_time))
> STORED AS ICEBERG;
> Query: CREATE TABLE tab ( START_TIME TIMESTAMP )
> PARTITIONED BY SPEC (DAY(start_time))
> STORED AS ICEBERG
> +-------------------------+
> | summary |
> +-------------------------+
> | Table has been created. |
> +-------------------------+
> Fetched 1 row(s) in 0.21s{noformat}
> Moreover, altering partition on an existing table also throws exception when
> column name is not lowercase:
> {noformat}
> [localhost:21050] default> ALTER TABLE tab SET PARTITION SPEC
> (MONTH(START_TIME));
> Query: ALTER TABLE tab SET PARTITION SPEC (MONTH(START_TIME))
> 2025-08-06 13:22:58 [Exception] ERROR: Query
> bd439760e55e7cd2:9211459000000000 failed:
> ImpalaRuntimeException: Failed to ALTER table 'tab': Cannot find field
> 'START_TIME' in struct: struct<1: start_time: optional timestamp>{noformat}
>
> During parsing, column name is set to lowercase:
> [https://github.com/apache/impala/blob/09a6f0e6cd912f573f0d8950abf40f498385c628/fe/src/main/java/org/apache/impala/analysis/ColumnDef.java#L109]
> For legacy tables, partitioning columns are either handled by ColumnDef (in
> CREATE TABLE staements) or by PartitionKeyValue (in INSERT statements):
> https://github.com/apache/impala/blob/09a6f0e6cd912f573f0d8950abf40f498385c628/fe/src/main/java/org/apache/impala/analysis/PartitionKeyValue.java#L41-L44
> In contrast, an Iceberg partition field's name is processed as-is:
> [https://github.com/apache/impala/blob/09a6f0e6cd912f573f0d8950abf40f498385c628/fe/src/main/java/org/apache/impala/analysis/IcebergPartitionField.java#L52-L61]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
