[GitHub] [spark] cloud-fan commented on a change in pull request #24075: [SPARK-26176][SQL] Verify column names for CTAS with `STORED AS`

2019-03-18 Thread GitBox 
cloud-fan commented on a change in pull request #24075: [SPARK-26176][SQL] 
Verify column names for CTAS with `STORED AS`
URL: https://github.com/apache/spark/pull/24075#discussion_r266388981
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
 ##
 @@ -210,7 +209,7 @@ case class RelationConversions(
   case CreateTable(tableDesc, mode, Some(query))
   if DDLUtils.isHiveTable(tableDesc) && 
tableDesc.partitionColumnNames.isEmpty &&
 isConvertible(tableDesc) && 
SQLConf.get.getConf(HiveUtils.CONVERT_METASTORE_CTAS) =>
-DDLUtils.checkDataColNames(tableDesc)
+DDLUtils.checkDataColNames(tableDesc.copy(schema = query.schema))
 
 Review comment:
   Do we need to call it here?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #24075: [SPARK-26176][SQL] Verify column names for CTAS with `STORED AS`

2019-03-18 Thread GitBox 
cloud-fan commented on a change in pull request #24075: [SPARK-26176][SQL] 
Verify column names for CTAS with `STORED AS`
URL: https://github.com/apache/spark/pull/24075#discussion_r266388573
 
 

 ##
 File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala
 ##
 @@ -206,6 +206,8 @@ case class PreprocessTableCreation(sparkSession: 
SparkSession) extends Rule[Logi
 val analyzedQuery = query.get
 val normalizedTable = normalizeCatalogTable(analyzedQuery.schema, 
tableDesc)
 
+DDLUtils.checkDataColNames(tableDesc.copy(schema = 
analyzedQuery.schema))
 
 Review comment:
   did we call this in the else branch?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #24075: [SPARK-26176][SQL] Verify column names for CTAS with `STORED AS`

2019-03-14 Thread GitBox 
cloud-fan commented on a change in pull request #24075: [SPARK-26176][SQL] 
Verify column names for CTAS with `STORED AS`
URL: https://github.com/apache/spark/pull/24075#discussion_r265844898
 
 

 ##
 File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
 ##
 @@ -155,7 +155,7 @@ object HiveAnalysis extends Rule[LogicalPlan] {
   CreateTableCommand(tableDesc, ignoreIfExists = mode == SaveMode.Ignore)
 
 case CreateTable(tableDesc, mode, Some(query)) if 
DDLUtils.isHiveTable(tableDesc) =>
-  DDLUtils.checkDataColNames(tableDesc)
+  DDLUtils.checkDataColNames(tableDesc.copy(schema = query.schema))
 
 Review comment:
   can we unify this check for both data source table and hive serde table?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org