MaxGekk commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package URL: https://github.com/apache/spark/pull/25022#issuecomment-533127763 > Seems java Period (defines years, months and days) doesn't match the SQL year-month interval either. I think it matches. Just provide zero days while converting to periods: `java.time.Period.of(years, months, 0)` > I don't quite understand why the SQL standard defines 2 interval types. It complicates the type system and seems to me the only benefit is to make interval comparable. To be comparable is one reason. Another one (related) is all fields of the `day-time interval` type have constant number of sub-units (1 day = 24 hours, 1 hour = 60 sec, and etc.). The same is for the `year-month interval` type. Total number of microseconds (nanoseconds) in a value of the `day-time interval` is the same, and does not depends on time zone, or number of days in the month of some timestamp. I would like to highlight that `day-time interval` has **unconstrained** number of days. So, you can represent any intervals by this type. I think the `year-month interval` type was added for convenience. If you have a local timestamp like `2019-09-19 10:00:00` and want to add 2 months to it, `year-month interval` would be easier to use but you can reach the same result with `day-time interval`. > But I don't quite understand why java Period defines both years and months. AFAIK 1 year is always 12 months. I guess they wanted to make it more flexible. The type reflects what users want to express like `6 years, -9 months and 12 days`. Just in case, `Period` supports negative years, months and days. > The proposal from me is: > update CalendarInterval to follow parquet/pgsql, and expose it. `CalendarInterval` has been already exposed partially. If we change this type, won't this break user apps?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
