Sudhanva Huruli created SPARK-62033:
---------------------------------------
Summary: COMMENT ON TABLE clears the default collation, and ALTER
TABLE ... DEFAULT COLLATION clears the comment, on the session catalog
Key: SPARK-62033
URL: https://issues.apache.org/jira/browse/SPARK-62033
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.3.0
Reporter: Sudhanva Huruli
A table in the session catalog keeps its comment and its default collation in
dedicated fields, not in its table properties. V2SessionCatalog.alterTable
reads both values back from the table properties after applying the changes, so
a change that sets one of them clears the other.
Steps (spark-sql, default in-memory session catalog; the Hive-backed session
catalog behaves the same):
CREATE TABLE t (a STRING) USING parquet COMMENT 'table comment' DEFAULT
COLLATION UTF8_LCASE;
COMMENT ON TABLE t IS 'new comment';
DESCRIBE TABLE EXTENDED t;
Expected: Comment is 'new comment' and the default collation is still
UTF8_LCASE. Actual: the collation is gone.
The other direction: after the same CREATE, ALTER TABLE t DEFAULT COLLATION
UTF8_BINARY; leaves the collation as UTF8_BINARY but removes the comment.
COMMENT ON TABLE t IS NULL also removes the collation.
ALTER TABLE ... SET TBLPROPERTIES keeps both values, because it runs as the v1
command, which already keeps fields a change does not name. Expected rule for
alterTable: recompute the comment or the collation only when a change sets or
removes that property, and otherwise keep the existing value.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]