[
https://issues.apache.org/jira/browse/IMPALA-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer updated IMPALA-7853:
------------------------------------
Description:
PARQUET-1387 added int64 timestamps with nanosecond precision.
As 64 bits are not enough to represent the whole 1400..9999 range of Impala
timestamps, this new new type works with a limited range:
1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
The benefit of the reduced range is that no validation is necessary during
scanning, as every possible 64 bit value represents a valid timestamp in
Impala. This may mean that this has the potential be the fastest way to store
timestamps in Impala + Parquet.
Another way NANO differs from MICRO and MILLI is that NANO can be only
described with new logical types in Parquet, it has no converted type
equivalent.
was:
PARQUET-1387 added int64 timestamps with nanosecond precision.
As 64 bits are not enough to represent the whole 1400..9999 range of Impala
timestamps, this new new type works with a limited range:
1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
The benefit of the reduced range is that no validation is necessary during
scanning, as every possible 64 bit value represents a valid timestamp in
Impala. This may mean that this has the potential be the fastest way to store
timestamps in Impala + Parquet.
Another way NANO differs from MICRO and MILLI is that NANO can be be only
described with new logical types in Parquet, it has no converted type
equivalent.
> Add support to read int64 NANO timestamps to the parquet scanner
> ----------------------------------------------------------------
>
> Key: IMPALA-7853
> URL: https://issues.apache.org/jira/browse/IMPALA-7853
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
>
> PARQUET-1387 added int64 timestamps with nanosecond precision.
> As 64 bits are not enough to represent the whole 1400..9999 range of Impala
> timestamps, this new new type works with a limited range:
> 1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
> The benefit of the reduced range is that no validation is necessary during
> scanning, as every possible 64 bit value represents a valid timestamp in
> Impala. This may mean that this has the potential be the fastest way to store
> timestamps in Impala + Parquet.
> Another way NANO differs from MICRO and MILLI is that NANO can be only
> described with new logical types in Parquet, it has no converted type
> equivalent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
