Maxim Gekk created SPARK-29448:
----------------------------------
Summary: Support the `INTERVAL` type by Parquet datasource
Key: SPARK-29448
URL: https://issues.apache.org/jira/browse/SPARK-29448
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 2.4.4
Reporter: Maxim Gekk
Parquet format allows to store intervals as triple of (milliseconds, days,
months) see
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#interval .
The `INTERVAL` logical type is used for an interval of time. _It must annotate
a fixed_len_byte_array of length 12. This array stores three little-endian
unsigned integers that represent durations at different granularities of time.
The first stores a number in months, the second stores a number in days, and
the third stores a number in milliseconds. This representation is independent
of any particular timezone or date._
Need to support writing and reading values of Catalyst's CalendarIntervalType.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
