MaxGekk commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds URL: https://github.com/apache/spark/pull/26134#issuecomment-542793129 > @MaxGekk do you think Spark violates the SQL standard if ZoneId is used to represent the session local timezone? @cloud-fan I think so. If we accept the session time zone as zone offset (`TIMEZONE_HOUR:TIMEZONE_MINUTE`) only some problems will gone for sure. And this will speed up date-time operations significantly because conversion of `ZoneId` -> `ZoneOffset` is very time consuming. It has to look up at historical rules, for instance: http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/time/zone/ZoneRules.java#l633 > I see many datetime operations in DateTimeUtils are using ZoneId. ZoneId itself is not the problem because ZoneOffset is sub-type of ZoneId and Java 8 time functions can optimally handle ZoneId instances when they are ZoneOffset actually. Theoretically, we could require `spark.sql.session.timeZone` to be set as zone offset `TIMEZONE_HOUR:TIMEZONE_MINUTE` but this will break user apps highly likely.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
