MaxGekk commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type. URL: https://github.com/apache/spark/pull/25678#issuecomment-531141943 @rxin how about those real use cases: 1. Update data in parquet files containing `TIME` columns? For example, Snowflake supports [TIME type](https://docs.snowflake.net/manuals/sql-reference/data-types-datetime.html#time), and [Presto](https://prestodb.github.io/docs/current/language/types.html#date-and-time), [BigQuery](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#time-type), [Apache Flink](https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/api/common/typeinfo/Types.html#SQL_TIME) as well. Would it be useful for Spark users to be able to read/write/update parquet files written by other systems? 2. Update tables with `TIME` columns via JDBC. Most of modern Relational DBMS follow the SQL standard and support the `TIME` type. Would it be useful for Spark users to be able to update such table via JDBC? Another argument for the `TIME` type, we cannot say that Spark SQL is compliant with SQL ANSI standard without supporting its basic type. Is Spark SQL going to be compatible with the SQL standard? > In the case of a new data type, it has high overhead to the end user and library developer as well, because now they need to handle this data type in their code. This argument blocks any extensions of Spark SQL type systems, actually forever. It seems implementing the `INTERVAL` type according to the SQL standard has no chance for merging to the upstream: [SPARK-27793: Support SQL day-time INTERVAL type](https://issues.apache.org/jira/browse/SPARK-27793) and [SPARK-27791: Support SQL year-month INTERVAL type](https://issues.apache.org/jira/browse/SPARK-27791). Right?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
