[
https://issues.apache.org/jira/browse/IMPALA-12356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765811#comment-17765811
]
Michael Smith commented on IMPALA-12356:
----------------------------------------
Is this a regression, or old behavior?
> Partition created by INSERT will make the next ALTER_PARTITION event on it
> always treated as self-event
> -------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-12356
> URL: https://issues.apache.org/jira/browse/IMPALA-12356
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Venugopal Reddy K
> Priority: Critical
> Labels: ramp-up
>
> In Impala, create a partitioned table and create one partition in it using
> {*}INSERT{*}:
> {noformat}
> create table my_part (i int) partitioned by (p int) stored as parquet;
> insert into my_part partition(p=0) values (0),(1),(2);
> show partitions my_part
> +-------+-------+--------+------+--------------+-------------------+---------+-------------------+---------------------------------------------------+-----------+
> | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format
> | Incremental stats | Location | EC
> Policy |
> +-------+-------+--------+------+--------------+-------------------+---------+-------------------+---------------------------------------------------+-----------+
> | 0 | -1 | 1 | 358B | NOT CACHED | NOT CACHED | PARQUET
> | false | hdfs://localhost:20500/test-warehouse/my_part/p=0 |
> NONE |
> | Total | -1 | 1 | 358B | 0B | |
> | | |
> |
> +-------+-------+--------+------+--------------+-------------------+---------+-------------------+---------------------------------------------------+-----------+
> {noformat}
> In Hive, describe the partition. We can see parameters of
> "impala.events.catalogServiceId" and "impala.events.catalogVersion" added by
> Impala. This is ok.
> {noformat}
> hive> desc formatted my_part partition(p=0);
> +-----------------------------------+----------------------------------------------------+-----------------------------------+
> | col_name | data_type
> | comment |
> +-----------------------------------+----------------------------------------------------+-----------------------------------+
> | i | int
> | |
> | | NULL
> | NULL |
> | # Partition Information | NULL
> | NULL |
> | # col_name | data_type
> | comment |
> | p | int
> | |
> | | NULL
> | NULL |
> | # Detailed Partition Information | NULL
> | NULL |
> | Partition Value: | [0]
> | NULL |
> | Database: | default
> | NULL |
> | Table: | my_part
> | NULL |
> | CreateTime: | Wed Aug 09 15:24:50 CST 2023
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Location: |
> hdfs://localhost:20500/test-warehouse/my_part/p=0 | NULL
> |
> | Partition Parameters: | NULL
> | NULL |
> | | impala.events.catalogServiceId
> | eab33ebb8a14cfd:8b2bdc12df3568df |
> | | impala.events.catalogVersion
> | 1882 |
> | | numFiles
> | 1 |
> | | totalSize
> | 358 |
> | | transient_lastDdlTime
> | 1691565890 |
> | | NULL
> | NULL |
> | # Storage Information | NULL
> | NULL |
> | SerDe Library: |
> org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe | NULL
> |
> | InputFormat: |
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat | NULL
> |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat | NULL
> |
> | Compressed: | No
> | NULL |
> | Num Buckets: | 0
> | NULL |
> | Bucket Columns: | []
> | NULL |
> | Sort Columns: | []
> | NULL |
> +-----------------------------------+----------------------------------------------------+-----------------------------------+
> {noformat}
> Now run an ALTER statement on the partition in Hive, e.g. changing the
> location:
> {code:sql}
> alter table my_part partition(p=0) set location '/tmp';{code}
> Impala will skip the ALTER_PARTITION event since it's considered as a
> self-event. In catalogd logs:
> {noformat}
> I0809 15:30:19.628449 29844 MetastoreEvents.java:628] EventId: 8351549
> EventType: ALTER_PARTITION Incremented events skipped counter to 12
> I0809 15:30:19.628616 29844 MetastoreEvents.java:628] EventId: 8351549
> EventType: ALTER_PARTITION Not processing the event as it is a
> self-event{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
