Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/17649
@gatorsmile Hive treats comment simply as a key in the string-string
parameter map, while spark extracts comment from the map as a field in
`CatalogTable`. So the question is, should spark consider both `comment` and
`COMMENT` as table comment?
Here's the results of hive:
```
0: jdbc:hive2://.../> create table src (key int , value string) comment
"initial comment";
No rows affected (1.055 seconds)
0: jdbc:hive2://.../> desc formatted src;
+-------------------------------+-------------------------------------------------------+-----------------------+--+
| col_name | data_type
| comment |
+-------------------------------+-------------------------------------------------------+-----------------------+--+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| key | int
| |
| value | string
| |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | wzh
| NULL |
| Owner: | spark
| NULL |
| CreateTime: | Mon Apr 24 11:43:40 CST 2017
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Protect Mode: | None
| NULL |
| Retention: | 0
| NULL |
| Location: |
hdfs://hacluster/user/hive/warehouse/wzh.db/src | NULL |
| Table Type: | MANAGED_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | comment
| initial comment |
| | transient_lastDdlTime
| 1493005420 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe | NULL |
| InputFormat: |
org.apache.hadoop.hive.ql.io.RCFileInputFormat | NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.RCFileOutputFormat | NULL |
| Compressed: | No
| NULL |
| Num Buckets: | -1
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| Storage Desc Params: | NULL
| NULL |
| | serialization.format
| 1 |
+-------------------------------+-------------------------------------------------------+-----------------------+--+
28 rows selected (0.525 seconds)
0: jdbc:hive2://.../> alter table src set tblproperties("comment"="new
comment", "COMMENT"="NEW COMMENT");
No rows affected (0.62 seconds)
0: jdbc:hive2://.../> desc formatted src;
+-------------------------------+-------------------------------------------------------+-----------------------+--+
| col_name | data_type
| comment |
+-------------------------------+-------------------------------------------------------+-----------------------+--+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| key | int
| |
| value | string
| |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | wzh
| NULL |
| Owner: | spark
| NULL |
| CreateTime: | Mon Apr 24 11:43:40 CST 2017
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Protect Mode: | None
| NULL |
| Retention: | 0
| NULL |
| Location: |
hdfs://hacluster/user/hive/warehouse/wzh.db/src | NULL |
| Table Type: | MANAGED_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | COLUMN_STATS_ACCURATE
| false |
| | COMMENT
| NEW COMMENT |
| | comment
| new comment |
| | last_modified_by
| spark |
| | last_modified_time
| 1493005506 |
| | numFiles
| 0 |
| | numRows
| -1 |
| | rawDataSize
| -1 |
| | totalSize
| 0 |
| | transient_lastDdlTime
| 1493005506 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe | NULL |
| InputFormat: |
org.apache.hadoop.hive.ql.io.RCFileInputFormat | NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.RCFileOutputFormat | NULL |
| Compressed: | No
| NULL |
| Num Buckets: | -1
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| Storage Desc Params: | NULL
| NULL |
| | serialization.format
| 1 |
+-------------------------------+-------------------------------------------------------+-----------------------+--+
36 rows selected (0.337 seconds)
0: jdbc:hive2://.../> alter table src unset tblproperties ("COMMENT");
No rows affected (0.646 seconds)
0: jdbc:hive2://192.168.174.107:21066/> desc formatted src;
+-------------------------------+-------------------------------------------------------+-----------------------+--+
| col_name | data_type
| comment |
+-------------------------------+-------------------------------------------------------+-----------------------+--+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| key | int
| |
| value | string
| |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | wzh
| NULL |
| Owner: | spark
| NULL |
| CreateTime: | Mon Apr 24 11:43:40 CST 2017
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Protect Mode: | None
| NULL |
| Retention: | 0
| NULL |
| Location: |
hdfs://hacluster/user/hive/warehouse/wzh.db/src | NULL |
| Table Type: | MANAGED_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | COLUMN_STATS_ACCURATE
| false |
| | comment
| new comment |
| | last_modified_by
| spark |
| | last_modified_time
| 1493005605 |
| | numFiles
| 0 |
| | numRows
| -1 |
| | rawDataSize
| -1 |
| | totalSize
| 0 |
| | transient_lastDdlTime
| 1493005605 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe | NULL |
| InputFormat: |
org.apache.hadoop.hive.ql.io.RCFileInputFormat | NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.RCFileOutputFormat | NULL |
| Compressed: | No
| NULL |
| Num Buckets: | -1
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| Storage Desc Params: | NULL
| NULL |
| | serialization.format
| 1 |
+-------------------------------+-------------------------------------------------------+-----------------------+--+
35 rows selected (0.552 seconds)
```--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
