[
https://issues.apache.org/jira/browse/IMPALA-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer updated IMPALA-7853:
------------------------------------
Description:
PARQUET-1387 added int64 timestamps with nanosecond precision.
As 64 bits are not enough to represent the whole 1400..9999 range of Impala
timestamps, this new new type works with a limited range:
1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
The benefit of the reduced range is that no validation is necessary during
scanning, as every possible 64 bit value represents a valid timestamp in
Impala. This may mean that this has the potential be the fastest way to store
timestamps in Impala + Parquet.
Another way NANO differs from MICRO and MILLI is that NANO can be be only
described with new logical types in Parquet, it has no converted type
equivalent.
was:
PARQUET-1387 added int64 timestamps with nanosecond precision.
As 64 bits are not enough to represent the whole 1400..9999 range of Impala
timestamps, so this new new type works with a limited range:
1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
The benefit of the reduced range is that no validation is necessary during
scanning, as every possible 64 bit value represents a valid timestamp in
Impala. This may mean that this has the potential be the fastest way to store
timestamps.
Another way NANO differs from MICRO and MILLI is that NANO can be be only
described with new logical types in Parquet, it has no converted type
equivalent.
> Add support to read int64 NANO timestamps to the parquet scanner
> ----------------------------------------------------------------
>
> Key: IMPALA-7853
> URL: https://issues.apache.org/jira/browse/IMPALA-7853
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
>
> PARQUET-1387 added int64 timestamps with nanosecond precision.
> As 64 bits are not enough to represent the whole 1400..9999 range of Impala
> timestamps, this new new type works with a limited range:
> 1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
> The benefit of the reduced range is that no validation is necessary during
> scanning, as every possible 64 bit value represents a valid timestamp in
> Impala. This may mean that this has the potential be the fastest way to store
> timestamps in Impala + Parquet.
> Another way NANO differs from MICRO and MILLI is that NANO can be be only
> described with new logical types in Parquet, it has no converted type
> equivalent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
