cloud-fan commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds URL: https://github.com/apache/spark/pull/26134#issuecomment-542738622 I spend more time thinking about it. There are 3 kinds of timestamps in the SQL world: 1. TIMESTAMP WITHOUT TIMEZONE: It's timezone-agnostic, and is simply a combination of the datetime fields YEAR, MONTH, DAY, HOUR, MINUTE and SECOND. So adding one day is the same as adding 24 hours. 2. TIMESTAMP WITH TIMEZONE: It's similar to the above one but has a timezone associated with each value. Since SQL spec requires a timezone offset (`TIMEZONE_HOUR:TIMEZONE_MINUTE`), there is no ambiguity and adding 1 day is the same as adding 24 hours. 3. TIMESTAMP WITH LOCAL TIMEZONE (the Spark timestamp): It's similar to the first one but has a local timezone associated with all the values. Since SQL spec requires a timezone offset, the local timezone should also be a specific offset, and has no ambiguity as well. However, things are different in the Java world. The java time API use `ZoneId`, which can be a region name with undeterminate zone offset (see the [doc](https://docs.oracle.com/javase/8/docs/api/java/time/ZoneId.html)). @MaxGekk do you think Spark violates the SQL standard if `ZoneId` is used to represent the session local timezone? I see many datetime operations in `DateTimeUtils` are using `ZoneId`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
