[
https://issues.apache.org/jira/browse/HIVE-15995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389925#comment-16389925
]
Vihang Karajgaonkar commented on HIVE-15995:
--------------------------------------------
Hi [~szita] thanks for the updated patch. I was looking at the other queries
which take in {{cascade}} option and I noticed that they have a partition spec
in the query syntax. Eg: add/rename column is under
alterTblPartitionStatementSuffix in HiveParser.g I think it has to do with the
option of adding partition spec to the command. Do you think it makes sense to
add a optional partition spec as well similar to add/replace columns query? I
think column descriptors are also at partition level. Eg. you can do a
{{describe table foo partition (p1=10);}} and hive will show the column
information for that partition. So users can potentially update the columns at
a partition level as well. The current patch is either just the table no
partitions or table with all partitions. There is no way to update columns for
a subset of partitions.
Also, in the qtest can you add a test for a partitioned table? Include describe
table .. partition .. syntax to confirm that partition columns are also getting
updated.
> Syncing metastore table with serde schema
> -----------------------------------------
>
> Key: HIVE-15995
> URL: https://issues.apache.org/jira/browse/HIVE-15995
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 1.2.1, 2.1.0, 3.0.0
> Reporter: Michal Ferlinski
> Assignee: Adam Szita
> Priority: Major
> Attachments: HIVE-15995.1.patch, HIVE-15995.2.patch,
> HIVE-15995.3.patch, HIVE-15995.4.patch, HIVE-15995.patch, cx1.avsc, cx2.avsc
>
>
> Hive enables table schema evolution via properties. For avro e.g. we can
> alter the 'avro.schema.url' property to update table schema to the next
> version. Updating properties however doesn't affect column list stored in
> metastore DB so the table is not in the newest version when returned from
> metastore API. This is problem for tools working with metastore (e.g. Presto).
> To solve this issue I suggest to introduce new DDL statement syncing
> metastore columns with those from serde:
> {code}
> ALTER TABLE user_test1 UPDATE COLUMNS
> {code}
> Note that this is format independent solution.
> To reproduce, follow the instructions below:
> - Create table based on avro schema version 1 (cxv1.avsc)
> {code}
> CREATE EXTERNAL TABLE user_test1
> PARTITIONED BY (dt string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> LOCATION
> '/tmp/schema-evolution/user_test1'
> TBLPROPERTIES ('avro.schema.url'='/tmp/schema-evolution/cx1.avsc');
> {code}
> - Update schema to version 2 (cx2.avsc)
> {code}
> ALTER TABLE user_test1 SET TBLPROPERTIES ('avro.schema.url' =
> '/tmp/schema-evolution/cx2.avsc');
> {code}
> - Print serde columns (top info) and metastore columns (Detailed Table
> Information):
> {code}
> DESCRIBE EXTENDED user_test1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
