Zoltán Borók-Nagy created IMPALA-13463:
------------------------------------------
Summary: Impala should ignore case of Iceberg schema elements
Key: IMPALA-13463
URL: https://issues.apache.org/jira/browse/IMPALA-13463
Project: IMPALA
Issue Type: Bug
Reporter: Zoltán Borók-Nagy
Assignee: Zoltán Borók-Nagy
Schema is case insensitive in Impala.
Via Spark it's possible to create schema elements with upper/lower case letters
and store them in the metadata JSON files of Iceberg, e.g.:
{noformat}
"schemas" : [ {
"type" : "struct",
"schema-id" : 0,
"fields" : [ {
"id" : 1,
"name" : "ID",
"required" : false,
"type" : "string"
}, {
"id" : 2,
"name" : "OWNERID",
"required" : false,
"type" : "string"
} ]
} ],
{noformat}
This can cause problems in Impala during predicate pushdown, as we can get a
ValidationException from the Iceberg library (as Impala pushes down predicates
with lower case column names, while Iceberg sees upper case names).
We should invoke Scan.caseSensitive(boolean caseSensitive) on the TableScan
object to set case insensitivity.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
