Weicheng Wang created SPARK-39017: ------------------------------------- Summary: Change Java8 datetime support to configurable Key: SPARK-39017 URL: https://issues.apache.org/jira/browse/SPARK-39017 Project: Spark Issue Type: Brainstorming Components: SQL Affects Versions: 3.2.1 Reporter: Weicheng Wang
*Background:* In the Spark 3.1.0 there is an improvement introduced to enable Java 8 datetime API by default. It stops user set this configuration, *spark.sql.datetime.java8API.enabled,* in command line or configuration file when using Spark SQL shell or Spark Thrift Server. The only way to set it is in the SQL Session using SET command like: {code:java} spark-sql> SET spark.sql.datetime.java8API.enabled=false {code} There are a few issues related to this improvement * [https://github.com/apache/iceberg/issues/2530] * [https://github.com/delta-io/delta/issues/760] There is a solution in 3.2.0 for it by having *LocalDateConverter* uses *DateConverter* and in the *DateConverter* it handles both *LocalDate* and *Date* types. *Discussion:* I think we should give back the ability to user to set this option in configuration file and command line. Both of the improvement defeats the reason why we have a java8API option configurable. Please advise Thanks -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org