Sai Hemanth Gantasala created IMPALA-13599:
----------------------------------------------
Summary: Reduce ALTER_PARTITION events fired from Impala
Key: IMPALA-13599
URL: https://issues.apache.org/jira/browse/IMPALA-13599
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Reporter: Sai Hemanth Gantasala
Assignee: Sai Hemanth Gantasala
There are a couple of APIs in Impala that fire ALTER_PARTITION for each
partition.
{code:java}
Alter table foo set cached/uncached;
Drop stats table_name;{code}
These table-level operations are calling HMS APIs alter_partition() thousands
times if there are an equal number of partitions. The side effect is that it
leads to thousands of alter partition events.
We should optimize to call alter_partitions() HMS API only once for these
operations and bulk update the partitions.
P.S: HIVE-27746 - We can ultimately leverage this to have a single
alter_partitions event that can help the event processor catch up with lag
quickly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
