cloud-fan commented on issue #28026: [SPARK-31257][SQL] Unify create table syntax URL: https://github.com/apache/spark/pull/28026#issuecomment-609838028 I'm a bit worried about merging this `+1661 −1146` huge PR to 3.0 at this stage. I agree that this may be in the right direction to implement Hive as a v2 catalog/data source, but this is way beyond what we need for 3.0. Can we have a more surgical patch that only unifies the CREATE TABLE syntax? Basically, for the items you listed in the PR description. These are still needed: - Unify the create table syntax in the parser by merging Hive and DataSource clauses - Add SerdeInfo and external boolean to statement plans and update AstBuilder to produce them - Add conversion from create statement plan to v1 create plans in ResolveSessionCatalog - Remove SparkSqlParser rules for Hive syntax With one more item: update ResolveCatalogs to fail if Hive specific clauses are specified in the create statement plan for v2 catalogs. These can be done later - Support new statement clauses in ResolveCatalogs conversion to v2 create plans - Add "option." namespace to distinguish SERDEPROPERTIES and OPTIONS in table properties BTW, the `Does this PR introduce any user-facing change?` section looks more like a new feature. Can we focus on the consequence of the syntax unification? For example: 1. `CREATE TABLE ... USING ... PARTITIONED BY (colName colType)` is now supported. 2. `CREATE EXTERNAL TABLE ...` without USING and STORED BY/AS now creates Hive table. 3. ...
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
