cloud-fan commented on issue #27277: [SPARK-30568][SQL] Invalidate interval type as a field table schema URL: https://github.com/apache/spark/pull/27277#issuecomment-576009823 This does reduce the scope of https://github.com/apache/spark/commit/d67b98ea016e9b714bef68feaac108edd08159c9 to cast only, but we need more than just "keep behavior the same as 2.4", we need to hide interval type from external data sources/catalogs. In 2.4, CREATE TABLE can still has interval type column, even if parser doesn't allow it. This is because we have a java API `spark.catalog.createTable`. Fortunately, this is OK in 2.4, as Hive catalog doesn't allow it and the CREATE TABLE command fails at the end. In 3.0, the catalog becomes an API, and it's possible that we leak interval type column to external catalog implementations. I think it's OK to allow interval type in the parser, as we allow it in `spark.catalog.createTable` already. But it's important to disallow creating table with interval type column like 2.4. We need to add a check in the analyzer.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
