[
https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834570#comment-16834570
]
ASF subversion and git services commented on IMPALA-7368:
---------------------------------------------------------
Commit 2bce974990e19788ec359deec50f06d44ec92048 in impala's branch
refs/heads/master from Attila Jeges
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2bce974 ]
IMPALA-7370: DATE: Read/Write to parquet.
This change is a follow-up to IMPALA-7368 and adds support for DATE
type to the parquet scanner/writer. CREATE TABLE LIKE PARQUET
statements associated with data files that contain dates are also
supported.
Parquet uses DATE logical type for dates. DATE logical type annotates
an INT32 that stores the number of days from the Unix epoch, 1 January
1970.
This representation introduces a parquet interoperability issue
between Impala and older versions of Hive:
- Before version 3.1, Hive used Julian calendar to represent dates
up to 1582-10-05 and Gregorian calendar for dates starting with
1582-10-15. Dates between 1582-10-05 and 1582-10-15 were lost.
- Impala uses proleptic Gregorian calendar, extending the Gregorian
calendar backward to dates preceding its official introduction in
1582-10-15.
This means that pre-1582-10-15 dates written to a parquet table by
Hive will be read back incorrectly by Impala and vice versa.
Note that Hive 3.1 switched to proleptic Gregorian calendar too, so
for Hive 3.1+ this is no longer an issue.
Change-Id: I67da03754531660bc8de3b6935580d46deae1814
Reviewed-on: http://gerrit.cloudera.org:8080/13189
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add initial support for DATE type
> ---------------------------------
>
> Key: IMPALA-7368
> URL: https://issues.apache.org/jira/browse/IMPALA-7368
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Attila Jeges
> Assignee: Attila Jeges
> Priority: Major
> Fix For: Impala 3.3.0
>
>
> DATE values describe a particular year/month/day, in the form YYYY-MM-DD.
> For example, DATE '2013-01-01'. Date types do not have a time of day
> component. The range of values supported for the Date type is 0000-01-01 to
> 9999-12-31.
> The initial DATE type support should incluide the following changes:
> - new internal type
> - casting between DATE and other types
> - codegen infrastructure for expression evaluation
> - "IS [NOT] NULL" and "[NOT] IN" predicates
> - common comparison operators
> - BETWEEN operator
> - conditional functions
> - infrastructure changes for builtin scalar functions.
> - some built-in functions: aggregate functions, analytical functions, math
> functions.
> - support partitioning.
> - text support only.
> These items are tightly coupled and it makes sense to implement them in one
> change-set.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
