lidinghao commented on a change in pull request #25390: [SPARK-28662] [SQL]
Create Hive Partitioned Table DDL should fail when partition column type
missed
URL: https://github.com/apache/spark/pull/25390#discussion_r313013957
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala
##########
@@ -985,7 +985,15 @@ class SparkSqlAstBuilder(conf: SQLConf) extends
AstBuilder(conf) {
} else {
CreateTable(tableDescWithPartitionColNames, mode, Some(q))
}
- case None => CreateTable(tableDesc, mode, None)
+ case None =>
Review comment:
Hi, Yuanjian, I tried some solutions which fix this case in SqlBase.g4
before this PR submitted, but the result isn't good, here is the summary of
all the solutions.
If we try to fix this in SqlBase.g4 , i.e. during syntax analysis phase,
we have to change createHiveTable syntax description to make Antlr split the
analysis path into two branch, one for CTAS which accept syntax that define
partition column without data type and must have a AS query sub-clause at end
of the DDL sentence. Another branch for CT which accept syntax that define
partition column with data type and cannot have a AS query sub-clause at end
of the DDL sentence.
When user use a illegal CT DDL with partition columns data type missed, the
syntax analyzer will match this with first branch, and give a misleading
error messages for user. And if user use CTAS DDL but has partition columns
data type defined, the syntax analyzer will match this with second branch,
also give misleading error messages.
If we fix this in SparkSqlParser, i.e. during semantic analysis phase, we
can not only hand the illegal CTAS&CT DDL case as Antlr, but also give user a
explicit and useful error message.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
