[
https://issues.apache.org/jira/browse/HIVE-26658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622273#comment-17622273
]
Stamatis Zampetakis commented on HIVE-26658:
--------------------------------------------
Allowing mapping of INT64 timestamp types to other numerics might also be
beneficial for:
* performing up/down casting to other types with low performance overhead
* schema evolution to other types
Having said that, I don't really see an interesting real-life use-case where it
makes sense to migrate/cast a timestamp to another numeric (except BIGINT which
is already fixed) so I don't consider this a high-priority issue.
> INT64 Parquet timestamps cannot be mapped to most Hive numeric types
> --------------------------------------------------------------------
>
> Key: HIVE-26658
> URL: https://issues.apache.org/jira/browse/HIVE-26658
> Project: Hive
> Issue Type: Bug
> Components: Parquet, Serializers/Deserializers
> Affects Versions: 4.0.0-alpha-1
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Minor
> Labels: backwards-compatibility
>
> When attempting to read a Parquet file with column of primitive type INT64
> and logical type
> [TIMESTAMP|https://github.com/apache/parquet-format/blob/54e53e5d7794d383529dd30746378f19a12afd58/LogicalTypes.md?plain=1#L337]
> an error is raised when the Hive type is different from TIMESTAMP and BIGINT.
> Consider a Parquet file (e.g., ts_file.parquet) with the following schema:
> {code:json}
> {
> "name": "eventtime",
> "type": ["null", {
> "type": "long",
> "logicalType": "timestamp-millis"
> }],
> "default": null
> }
> {code}
>
> Mapping the column to a Hive numeric type among TINYINT, SMALLINT, INT,
> FLOAT, DOUBLE, DECIMAL, and trying to run a SELECT will give back an error.
> The following snippet can be used to reproduce the problem.
> {code:sql}
> CREATE TABLE ts_table (eventtime INT) STORED AS PARQUET;
> LOAD DATA LOCAL INPATH 'ts_file.parquet' into table ts_table;
> SELECT * FROM ts_table;
> {code}
> This is a regression caused by HIVE-21215. Although, HIVE-21215 allows to
> read INT64 types as Hive TIMESTAMP, which was not possible before, at the
> same time it broke the mapping to every other Hive numeric type. The problem
> was addressed selectively for BIGINT type very recently (HIVE-26612).
> The primary goal of this ticket is to restore backward compatibility since
> these use-cases were working before HIVE-21215.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
