[ https://issues.apache.org/jira/browse/SPARK-23525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-23525. --------------------------------- Issue resolved by pull request 20696 [https://github.com/apache/spark/pull/20696] > ALTER TABLE CHANGE COLUMN doesn't work for external hive table > -------------------------------------------------------------- > > Key: SPARK-23525 > URL: https://issues.apache.org/jira/browse/SPARK-23525 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.2.0, 2.3.0 > Reporter: Pavlo Skliar > Assignee: Jiang Xingbo > Priority: Major > Fix For: 2.4.0, 2.3.1 > > > {code:java} > print(spark.sql(""" > SHOW CREATE TABLE test.trends > """).collect()[0].createtab_stmt) > /// OUTPUT > CREATE EXTERNAL TABLE `test`.`trends`(`id` string COMMENT '', `metric` string > COMMENT '', `amount` bigint COMMENT '') > COMMENT '' > PARTITIONED BY (`date` string COMMENT '') > ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' > WITH SERDEPROPERTIES ( > 'serialization.format' = '1' > ) > STORED AS > INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' > OUTPUTFORMAT > 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' > LOCATION 's3://xxxxx/xxxxx/xxxx' > TBLPROPERTIES ( > 'transient_lastDdlTime' = '1519729384', > 'last_modified_time' = '1519645652', > 'last_modified_by' = 'pavlo', > 'last_castor_run_ts' = '1513561658.0' > ) > spark.sql(""" > DESCRIBE test.trends > """).collect() > // OUTPUT > [Row(col_name='id', data_type='string', comment=''), > Row(col_name='metric', data_type='string', comment=''), > Row(col_name='amount', data_type='bigint', comment=''), > Row(col_name='date', data_type='string', comment=''), > Row(col_name='# Partition Information', data_type='', comment=''), > Row(col_name='# col_name', data_type='data_type', comment='comment'), > Row(col_name='date', data_type='string', comment='')] > spark.sql("""alter table test.trends change column id id string comment > 'unique identifier'""") > spark.sql(""" > DESCRIBE test.trends > """).collect() > // OUTPUT > [Row(col_name='id', data_type='string', comment=''), Row(col_name='metric', > data_type='string', comment=''), Row(col_name='amount', data_type='bigint', > comment=''), Row(col_name='date', data_type='string', comment=''), > Row(col_name='# Partition Information', data_type='', comment=''), > Row(col_name='# col_name', data_type='data_type', comment='comment'), > Row(col_name='date', data_type='string', comment='')] > {code} > The strange is that I've assigned comment to the id field from hive > successfully, and it's visible in Hue UI, but it's still not visible in from > spark, and any spark requests doesn't have effect on the comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org