rdblue commented on issue #28026: [SPARK-31257][SQL] Unify create table syntax URL: https://github.com/apache/spark/pull/28026#issuecomment-614372327 @cloud-fan, the meaning of external is dictated by Hive because that's where it came from. Spark is trying to change Hive's meaning of external, while maintaining "compatibility", but you can't do both. Because we now have a catalog abstraction, we're hitting the problem directly: you can't delegate to a catalog _and_ dictate what makes sense in cases specific to that catalog. If you want to drop support for external, then get rid of the keyword. Otherwise, it needs to be passed so that Spark isn't making arbitrary choices about details specific to a catalog. To be clear, I support failing all create statements that use EXTERNAL. That's fine with me, but you can't parse it and not pass it along. For SERDEPROPERTIES and OPTIONS, I fail to see how it is a similar situation. In that case, Spark has introduced a situation where it has clauses mean the same thing, so I implemented them to be passed the same way. We can pass these using separate prefixes if you'd like and let the catalog implementations choose. The problem is that this distinction has zero value for end users. Spark is easier to use and better overall if these are synonyms, but I can't say I care that much about it. Let me know what prefix you would like to use for SERDEPROPERTIES.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
