[GitHub] [spark] MaxGekk commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` to org.apache.spark.sql.types package

[GitBox]

MaxGekk commented on issue #25022: [SPARK-24695][SQL] Move `CalendarInterval` 
to org.apache.spark.sql.types package
URL: https://github.com/apache/spark/pull/25022#issuecomment-533067577
 
 
   > do you have an estimation about how hard it is?
   
   @cloud-fan I think we could follow the SQL standard and define 2 separate 
types, maybe under the new flag `spark.sql.ansi.enabled` introduced by 
https://github.com/apache/spark/pull/25693 :
   - [SPARK-27793: Support SQL day-time INTERVAL 
type](https://issues.apache.org/jira/browse/SPARK-27793)
   - [SPARK-27791: Support SQL year-month INTERVAL 
type](https://issues.apache.org/jira/browse/SPARK-27791)
   
   For both types, need to support external types 
`java.time.Duration`/`java.time.Period` (3 man-days each, like it takes me for 
`TimeType` in https://github.com/apache/spark/pull/25678), and additionally 
need to support literals (can be taken from existing code for 
`CalendarInterval`) (1-2 days), UDF (1-2 days), and support ops defined by SQL 
standard (5-10 days):
   <img width="736" alt="Screen Shot 2019-09-19 at 15 13 26" 
src="https://user-images.githubusercontent.com/1580697/65235467-17ee2500-daf0-11e9-92e7-fa8abf4a7d09.png";>
   <img width="734" alt="Screen Shot 2019-09-19 at 15 13 34" 
src="https://user-images.githubusercontent.com/1580697/65235484-22102380-daf0-11e9-9e3c-d546232f0f54.png";>
   plus need to support new types in existing expressions (> 10 days).
   I could create sub-tasks for that 
https://issues.apache.org/jira/browse/SPARK-27790 if you don't mind.
   
   The problem is the same objections are applicable for new interval types as 
for proposed `TIME` type: 
https://github.com/apache/spark/pull/25678#issuecomment-531325137
   I cannot show potential users for new types besides showing popularity of 
other systems that follow the SQL standard and support those types.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org