MaxGekk opened a new pull request #23913: [SPARK-27008][SQL] Support java.time.LocalDate as an external type of DateType URL: https://github.com/apache/spark/pull/23913 ## What changes were proposed in this pull request? In the PR, I propose to add new Catalyst type converter for `DateType`. It should be able to convert `java.time.LocalDate` to/from `DateType`. Main motivations for the changes: - Smoothly support Java 8 time API - Avoid inconsistency of calendars used inside of Spark 3.0 (Proleptic Gregorian calendar) and `java.sql.Date` (hybrid calendar - Julian + Gregorian). - Make conversion independent from current system timezone. By default, Spark converts values of `DateType` to `java.sql.Date` instances but the SQL config `spark.sql.catalyst.dateType` can change the behavior. It accepts two values `Date` (default) and `LocalDate`. If the former one is set, Spark returns `java.time.LocalDate` instances for timestamp values. ## How was this patch tested? Added new testes to `CatalystTypeConvertersSuite` to check conversion of `DateType` to/from `java.time.LocalDate`, `JavaUDFSuite`/ `UDFSuite` to test usage of `LocalDate` type in Scala/Java UDFs.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
