szehon-ho commented on code in PR #50991:
URL: https://github.com/apache/spark/pull/50991#discussion_r2103613204
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala:
##########
@@ -684,11 +684,23 @@ private[spark] class HiveExternalCatalog(conf: SparkConf,
hadoopConf: Configurat
} else {
(oldTableDef.schema, oldTableDef.partitionColumnNames)
}
+
+ // alter column comments
+ val newCommentMap = tableDefinition.schema.map(f => (f.name,
f.getComment())).toMap
+ val newSchemaWithComments = newSchema.copy(
+ fields = newSchema.fields.map { f =>
+ newCommentMap.getOrElse(f.name, None) match {
+ case Some(comment) => f.withComment(comment)
+ case _ => f
+ }
+ }
+ )
+
// // Add old table's owner if we need to restore
val owner =
Option(tableDefinition.owner).filter(_.nonEmpty).getOrElse(oldTableDef.owner)
val newDef = tableDefinition.copy(
storage = newStorage,
- schema = newSchema,
+ schema = newSchemaWithComments,
Review Comment:
@gengliangwang looking at this, actually there is a separate method for
alterting the schema
https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala#L706
maybe we rushed a bit to solution and the problem is the caller
(SessionCatalog) should call this one instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
