[
https://issues.apache.org/jira/browse/IMPALA-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy updated IMPALA-10358:
---------------------------------------
Description:
The Iceberg format spec defines what types to use for different file formats,
e.g.:
[https://iceberg.apache.org/spec/#parquet]
Impala should follow the specification:
* Strings in Iceberg tables should be annotated with UTF8 (when file format is
Parquet)
* Revise fixed(L) <-> CHAR(L) mapping
** CHAR(L) in Impala is annotated with UTF8
** fixed(L) is for fixed-length binary data
** We should probably remove this mapping
UPDATE: A [recent pull request|https://github.com/apache/iceberg/pull/1612]
added a new Iceberg class,
[HiveSchemaConverter|https://github.com/apache/iceberg/blob/master/mr/src/main/java/org/apache/iceberg/mr/hive/HiveSchemaConverter.java].
We should use this class to convert between Iceberg and Hive schemas.
was:
The Iceberg format spec defines what types to use for different type formats,
e.g.:
[https://iceberg.apache.org/spec/#parquet]
Impala should follow the specification:
* Strings in Iceberg tables should be annotated with UTF8 (when file format is
Parquet)
* Revise fixed(L) <-> CHAR(L) mapping
** CHAR(L) in Impala is annotated with UTF8
** fixed(L) is for fixed-length binary data
** We should probably remove this mapping
UPDATE: A [recent pull request|https://github.com/apache/iceberg/pull/1612]
added a new Iceberg class,
[HiveSchemaConverter|https://github.com/apache/iceberg/blob/master/mr/src/main/java/org/apache/iceberg/mr/hive/HiveSchemaConverter.java].
We should use this class to convert between Iceberg and Hive schemas.
> Correct Iceberg type mappings
> -----------------------------
>
> Key: IMPALA-10358
> URL: https://issues.apache.org/jira/browse/IMPALA-10358
> Project: IMPALA
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> The Iceberg format spec defines what types to use for different file formats,
> e.g.:
> [https://iceberg.apache.org/spec/#parquet]
> Impala should follow the specification:
> * Strings in Iceberg tables should be annotated with UTF8 (when file format
> is Parquet)
> * Revise fixed(L) <-> CHAR(L) mapping
> ** CHAR(L) in Impala is annotated with UTF8
> ** fixed(L) is for fixed-length binary data
> ** We should probably remove this mapping
> UPDATE: A [recent pull request|https://github.com/apache/iceberg/pull/1612]
> added a new Iceberg class,
> [HiveSchemaConverter|https://github.com/apache/iceberg/blob/master/mr/src/main/java/org/apache/iceberg/mr/hive/HiveSchemaConverter.java].
> We should use this class to convert between Iceberg and Hive schemas.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
