MaxGekk commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-533073064 > Spark's interval can combine microseconds and months, whereas the java ones can't. @hvanhovell And do you know why? Read the comment of `java.time.Period`: _"...Durations and periods differ in their treatment of daylight savings time when added to `ZonedDateTime`... For example, consider adding a period of one day and a duration of one day to 18:00 on the evening before a daylight savings gap. The `Period` will add the conceptual day and result in a `ZonedDateTime` at 18:00 the following day. By contrast, the `Duration` will add exactly 24 hours, resulting in a `ZonedDateTime` at 19:00 the following day (assuming a one hour DST gap)."_ If you have both in one type like Spark has, the result of applying the interval depends on the order of applying its components. And the order is not defined for now. I even don't speak about various number of days in months, and microseconds in `CalendarInterval` can contain more micros than 31 days * MICROS_PER_DAY. Bringing such uncertainties to user apps only because `CalendarInterval` has been already exposed partially doesn't make sense for me.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
