Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/15717#discussion_r91863165
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -274,6 +274,63 @@ case class AlterTableUnsetPropertiesCommand(
}
+
+/**
+ * A command to change the columns for a table, only support changing
column comment for now.
+ *
+ * The syntax of using this command in SQL is:
+ * {{{
+ * ALTER TABLE table_identifier
+ * CHANGE [COLUMN] column_old_name column_new_name column_dataType
[COMMENT column_comment]
+ * [FIRST | AFTER column_name];
+ * }}}
+ */
+case class AlterTableChangeColumnsCommand(
+ tableName: TableIdentifier,
+ columns: Map[String, StructField]) extends RunnableCommand {
+
+ // TODO: support change column name/dataType/metadata/position.
+ override def run(sparkSession: SparkSession): Seq[Row] = {
+ val catalog = sparkSession.sessionState.catalog
+ val table = catalog.getTableMetadata(tableName)
+ DDLUtils.verifyAlterTableType(catalog, table, isView = false)
+ // Currently only support changing column comment, throw a Exception
if other fields e.g.
+ // name/dataType are changed.
+ columns.foreach { case (oldColumnName, newColumn) =>
+ val originColumn = table.schema.collectFirst {
+ case field if field.name == oldColumnName => field
+ }
+ val unchanged = originColumn.forall(equalIgnoreComment(_, newColumn))
+ if (!unchanged) {
+ throw new AnalysisException(
+ s"ALTER TABLE CHANGE COLUMN is not supported for changing column
" +
+ s"'${getDesc(originColumn.get)}' to '${getDesc(newColumn)}'")
+ }
+ }
+
+ val newSchema = table.schema.fields.map { field =>
+ // If `columns` contains field name, update the field to the new
column, else respect the
+ // field.
+ columns.get(field.name).getOrElse(field)
--- End diff --
This way will lose the `metadata` field of the column. Let me show you the
way how we add metadata to the column
```Scala
val df = Seq((1, "abc"), (2, "hello")).toDF("a", "b")
val md = new MetadataBuilder().putString("key", "value").build()
val dfWithmeta = df.select('a, 'b.as("b", md))
dfWithmeta.write.saveAsTable("test1")
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
