Maxim Gekk created SPARK-34561: ---------------------------------- Summary: Cannot drop/add columns from/to a dataset of v2 `DESCRIBE TABLE` Key: SPARK-34561 URL: https://issues.apache.org/jira/browse/SPARK-34561 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 3.2.0 Reporter: Maxim Gekk
Dropping a column from a dataset of v2 `DESCRIBE TABLE` fails with: {code:java} Resolved attribute(s) col_name#102,data_type#103 missing from col_name#29,data_type#30,comment#31 in operator !Project [col_name#102, data_type#103]. Attribute(s) with the same name appear in the operation: col_name,data_type. Please check if the right attribute(s) are used.; !Project [col_name#102, data_type#103] +- LocalRelation [col_name#29, data_type#30, comment#31]{code} The code below demonstrates the issue: {code:java} val tbl = s"${catalogAndNamespace}tbl" withTable(tbl) { sql(s"CREATE TABLE $tbl (c0 INT) USING $v2Format") val description = sql(s"DESCRIBE TABLE $tbl") val noComment = description.drop("comment") } {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org