[
https://issues.apache.org/jira/browse/SPARK-26651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Gekk updated SPARK-26651:
-------------------------------
Description:
Spark 2.4 and previous versions use a hybrid calendar - Julian + Gregorian in
date/timestamp parsing, functions and expressions. The ticket aims to switch
Spark on Proleptic Gregorian calendar, and use java.time classes introduced in
Java 8 for timestamp/date manipulations. One of the purpose of switching on
Proleptic Gregorian calendar is to conform to SQL standard which supposes such
calendar.
Release notes:
Spark 3.0 has switched on Proleptic Gregorian calendar in parsing, formatting,
and converting dates and timestamps as well as in extracting sub-components
like years, days and etc. It uses Java 8 API classes from the java.time
packages that based on [ISO chronology
|https://docs.oracle.com/javase/8/docs/api/java/time/chrono/IsoChronology.html].
Previous versions of Spark performed those operations by using [the hybrid
calendar|https://docs.oracle.com/javase/7/docs/api/java/util/GregorianCalendar.html]
(Julian + Gregorian). The changes might impact on the results for dates and
timestamps before October 15, 1582 (Gregorian).
was:Spark 2.4 and previous versions use a hybrid calendar - Julian +
Gregorian in date/timestamp parsing, functions and expressions. The ticket aims
to switch Spark on Proleptic Gregorian calendar, and use java.time classes
introduced in Java 8 for timestamp/date manipulations. One of the purpose of
switching on Proleptic Gregorian calendar is to conform to SQL standard which
supposes such calendar.
> Use Proleptic Gregorian calendar
> --------------------------------
>
> Key: SPARK-26651
> URL: https://issues.apache.org/jira/browse/SPARK-26651
> Project: Spark
> Issue Type: Umbrella
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Major
>
> Spark 2.4 and previous versions use a hybrid calendar - Julian + Gregorian in
> date/timestamp parsing, functions and expressions. The ticket aims to switch
> Spark on Proleptic Gregorian calendar, and use java.time classes introduced
> in Java 8 for timestamp/date manipulations. One of the purpose of switching
> on Proleptic Gregorian calendar is to conform to SQL standard which supposes
> such calendar.
> Release notes:
> Spark 3.0 has switched on Proleptic Gregorian calendar in parsing,
> formatting, and converting dates and timestamps as well as in extracting
> sub-components like years, days and etc. It uses Java 8 API classes from the
> java.time packages that based on [ISO chronology
> |https://docs.oracle.com/javase/8/docs/api/java/time/chrono/IsoChronology.html].
> Previous versions of Spark performed those operations by using [the hybrid
> calendar|https://docs.oracle.com/javase/7/docs/api/java/util/GregorianCalendar.html]
> (Julian + Gregorian). The changes might impact on the results for dates and
> timestamps before October 15, 1582 (Gregorian).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
